Evaluating the Evidence
This chapter outlines the approach used by this and previous Committees to Review the Health Effects in Vietnam Veterans of Exposure to Herbicides to evaluate the available scientific evidence. A more complete description is found in Chapter 5 of Veterans and Agent Orange: Health Effects of Herbicides Used in Vietnam (hereafter referred to as VAO; IOM, 1994).
CHOICE OF HEALTH OUTCOMES
As discussed in Chapter 1, the Committee was charged with summarizing the strength of the scientific evidence for an association between herbicide exposure during service in the Vietnam War and a variety of diseases or health outcomes. Public Law (PL) 102-4, which led to the committee’s work, however, did not specify particular health outcomes of interest. VAO listed health outcomes addressed in the scientific literature; that list has been amended in the subsequent updates in response to new publications, to requests from the Department of Veterans Affairs (VA) and various veterans’ service organizations, and to concerns of Vietnam veterans and their families. Comments received at public hearings and in written submissions from veterans and other interested persons have been valuable for identifying issues to be pursued in greater depth in the scientific literature.
IDENTIFICATION OF RELEVANT LITERATURE
Mixtures of 2,4-dichlorophenoxyacetic acid (2,4-D), 2,4,5-trichlorophenoxyacetic acid (2,4,5-T), picloram, and cacodylic acid made up the majority of the
herbicides sprayed in Vietnam. At the time of the spraying, 2,3,7,8-tetrachlorodibenzo-p-dioxin (TCDD, one form of dioxin) was an unintended contaminant from the production of 2,4,5-T and was present in Agent Orange and some other herbicide formulations sprayed in Vietnam. Databases, therefore, were searched for the names of those compounds, their synonyms and abbreviations, and their Chemical Abstract Service (CAS) numbers. The evidence indicates that a tissue protein, the aryl hydrocarbon receptor (AhR), mediates the toxicity of TCDD. As such, the AhR also was used as a key word, as were “dioxin,” “Agent Orange,” and “Vietnam veteran.”
As discussed in Chapter 3, one of the herbicides used in Vietnam, cacodylic acid, is an organic form of arsenic, dimethylarsinic acid (DMA). In addition to being synthesized as an herbicide, DMA is a metabolite of inorganic arsenic in humans. DMA was long thought to be a biologically inactive metabolite of inorganic arsenic, but recent evidence suggests that one form—DMAIII—might be responsible for some of the adverse effects of inorganic arsenic. That evidence, however, is not sufficient to support a conclusion that exposure to cacodylic acid results in the same adverse health effects as does exposure to toxic concentrations of inorganic arsenic. Therefore, the literature on the health effects of inorganic arsenic was not considered in this report. Further details on the effects of inorganic arsenic can be found in Arsenic in Drinking Water (NRC, 1999) and Arsenic in Drinking Water: 2001 Update (NRC, 2001). For cacodylic acid and picloram, the search terms were the synonyms for the herbicides’ chemical names and their CAS numbers.
This report concentrates on the evidence published after the completion of work on Veterans and Agent Orange: Update 2002 (IOM, 2003a). For each health outcome, new evidence was reviewed in detail. The conclusions, however, are based on the accumulated evidence, and not just on recently published studies. When statistics have been generated on the same study population over time (as noted in Chapter 4), there will be multiple entries corresponding to successive updates in the summary tables of Chapters 6–9, but only the most comprehensive version of the information on a given population is factored into the committee’s conclusion on any health outcome. A detailed description of the committee’s general approach to the evaluation of scientific evidence is delineated in Chapter 5 of VAO. Later committees have adopted the original committee’s approach.
The information the committee used was compiled from a comprehensive electronic search by keyword of public and commercial databases—biologic, medical, toxicologic, chemical, historical, and regulatory—that provide citations from the scientific literature. In addition, the reference lists of some review and research articles, books, and reports were examined for potentially relevant articles. Literature identification continued through June 1, 2004. That search strategy helped ensure that all potentially relevant articles were identified, however, it also resulted in a large number of non-relevant studies being identified. More than 3,000 citations were identified in those searches, including some
studies that were identified multiple times. For the majority of the citations, it was evident from the abstract that the article did not address health effects in association with exposure to the chemicals of interest. For example, many of the identified citations investigated the efficacy of the herbicides in killing weeds. All studies that discussed the health effects were considered if there was an indication that the herbicides of interest (or any of their components) were investigated. Because of the large number of non-TCDD-like polychlorinated biphenyls (PCBs), epidemiology studies of PCBs were reviewed only if they had data on the TCDD-like activity. More than 550 potentially relevant citations were identified, and each of those articles was retrieved and reviewed for the report.
The committee had three specific tasks: determine whether there is a statistical association between exposure to the herbicides used in Vietnam and health outcomes, determine the degree of increased risk of effects among Vietnam veterans, and determine whether plausible biologic mechanism(s) provide support for a causal relationship with a given health outcome. This section discusses the committee’s approach to each of those tasks.
The issues in determining whether a statistical association exists are detailed in Chapter 5 of VAO. The committee found that the most relevant evidence came from epidemiologic studies—investigations in which large groups of people are studied to identify an association between exposure to a chemical of interest and the occurrence of particular health outcomes. Epidemiologists estimate associations between exposure and outcome in a specific population or group by use of such measures as relative risk, standardized mortality ratio, or odds ratio. Those terms describe the magnitude by which the rate of disease differs between two populations. For example, if the rate in an exposed population doubles relative to a non-exposed population, the relative risk, or rate ratio, is 2. Similarly, if the odds of a health outcome in an exposed population are 1:20 but 1:100 in an unexposed population, then the odds ratio is 5. In this report relative risk refers to the results of cohort studies; odds ratio (an estimate of relative risk) usually refers to the results of case–control studies. (The results of cohort studies sometimes are reported with odds ratios, again to estimate relative risk.) An estimated relative risk greater than 1 indicates a positive association (that is, it is more likely that the outcome will be seen with exposure), whereas values between zero and 1 indicate a negative or inverse association (that is, the outcome is less likely with exposure). A ratio of 1 suggests the absence of association. A statistically significant association is one that would be unlikely to occur by chance if there were truly no association (that is, null hypothesis is true).
Determining whether an estimated association between an exposure and an outcome represents a “real” relationship requires careful scrutiny because there can be more than one explanation for the estimate. Bias is a distortion of the measure of association that results from flawed selection in the assembly of the study population or from error in measurement of the characteristics studied. Confounding is the distortion of the measure of association that results from the failure to recognize or account for some other factor related both to exposure and to outcome. Chance is the degree to which the estimated association might vary across separate samples of the population studied. The width of the confidence interval is used to quantify the likely variability of the exposure-disease association. Even when a relative risk or standardized mortality ratio exceeds 1, a conclusion regarding increased risk must be qualified when the confidence interval is broad. In drawing its conclusions, the committee examined the quantitative estimates of association and evaluated the potential influences of bias, confounding, and chance. When integrating the findings from various studies, the committee considered the degree of statistical significance associated with every estimated risk (a reflection of the magnitude of the observed effect and the power of the study design), rather than simply tallying the “significant” and “non-significant” outcomes as dichotomous items of evidence.
In pursuing the question of statistical association, the committee recognized that an absolute conclusion about the absence of association is unattainable. As in science generally, studies of health effects associated with herbicide exposure cannot demonstrate that a purported effect is impossible or could never occur. Any instrument of observation, even the most excellent epidemiologic study, is limited in its resolving power. In a strict technical sense, therefore, the committee could not prove the absolute absence of an association between a health outcome and exposure to any one of the compounds of interest.
Factors such as consistency of evidence, biological plausibility, temporality, dose-response, and strength of association may be considered when deciding whether an observed statistical association is actually causal. The committee’s charge, however, did not extend to making determinations of causality, so no conclusions regarding cause-and-effect relationships have been made.
Interaction or synergism among the chemicals of interest or with yet other agents is another theoretical concern. Because the committee is not charged with making attribution to one of the chemicals of interest specifically, joint effects among them should be adequately identified by the committee’s approach. The number of combinations of these chemicals with other agents that might be problematic is virtually infinite. Real life experience, as investigated by epidemiology studies, effectively integrates any effect of a target substance over all other possibly detrimental or mitigating exposures that a population might have; it may never be possible to tease apart the contributions of the various factors definitively.
Increased Risk in Vietnam Veterans
When all of the available epidemiologic evidence has been evaluated, it is presumed that Vietnam veterans are at increased risk for a specific health outcome when there is evidence of a positive association between one or more of the chemicals of interest and that outcome. The best measure of potency for the quantification of risk to veterans would be the rate of the outcome in exposed Vietnam veterans compared with the rate in non-exposed veterans, adjusted for the degree to which any other differing factors between exposed and non-exposed veterans might influence those rates. A dose-response relationship established in another human population suitably adjusted for such factors would be similarly suitable.
It is difficult, however, to quantify risk when exposures have not been measured accurately in a population. Fairly accurate TCDD exposure data for Vietnam veterans are available only for a small subgroup enrolled in the Air Force Health Study (Ranch Hand population). Therefore, the absence of reliable measures of exposure to the chemicals of interest among Vietnam veterans limits the committee’s ability to quantify risk of specific diseases in this population.
Plausible Biologic Mechanisms
Chapter 3 details the experimental basis for assessment of biologic plausibility or the extent to which an observed statistical association in epidemiology studies is consistent with biologic or medical knowledge. In other words, would causation of the particular health effect observed make sense based on what is known about how the chemical acts at the tissue, cellular, or molecular level? The relationship between a particular exposure and a specific human health outcome is addressed in the context of research on the effects of those compounds on biologic systems and of evidence from animal studies. In this report, the committee reviews studies that were published after Update 2002 (IOM, 2003a), and considers those and earlier studies in its conclusions about biologic plausibility.
A positive statistical association between exposure and outcome does not necessarily mean that the exposure to the compound is the cause of the health effect. Data from toxicology studies may support or refute a hypothesis that a specific compound can cause a particular disease. Many toxicology studies are conducted with laboratory animals so that variables, including the amount and duration of exposure, can be controlled precisely. Studies that use isolated cells in culture also can be used to elucidate the way a compound alters cellular processes. The objectives of those toxicology studies are to determine what toxic effects are observed at different exposure concentrations and to identify the mechanisms by which the effects are produced. Ultimately, the results of the toxicology studies should be consistent with what is currently known about the human disease process to support a conclusion that the development of the
disease was caused by exposure. That approach is not without shortcomings; for example, the dose of a chemical required to produce an effect in experimental animals is often many times higher than human exposures. (For TCDD, however, effects have been observed in animals having body burdens within a factor of 10 or less of those at the high end of the general population in the industrialized world.) Furthermore, animal and cell culture models do not always accurately mimic human responses. When the epidemiological evidence is strong, the absence of evidence for biologic plausibility from toxicology studies does not rule out the possibility that an association may exist. In fact, such cases often drive new toxicology research.
EVALUATION OF THE EVIDENCE
The associations between exposures to the chemicals of interest and specific health outcomes are determined through an analysis of available epidemiologic studies, informed by an understanding of the toxicology of the chemicals and their exposure pathways. In reaching conclusions, VAO committees consider the nature of the exposures, the nature of the health outcomes, the populations exposed, and the quality of the evidence examined. Some specific issues this and prior committees have considered are addressed below.
The committee reviewed studies of Vietnam veterans and of other populations that might have been exposed to the chemicals of interest. Those studies included cohorts of workers in chemical production and agriculture, populations that reside near sites of environmental contamination, and residents of Vietnam. The committee believes that studies of such non-veteran subjects can help in the assessment of whether the chemicals of interest are associated with particular health outcomes. Some of the studies, especially those of workers in chemical-production plants, provide stronger evidence about health outcomes than do studies of veterans, because industrial exposures are frequently measured sooner after occurrence and are often more thoroughly characterized than has been the case for Vietnam veterans. Furthermore, in the chemical-production plant studies, the magnitudes and duration of exposure to the chemicals were generally greater, and the studies were frequently large enough to examine the health risks among groups of people with different levels of exposure. It is, however, the practice of VAO committees to evaluate all studies according to the same criteria whether or not their subjects are Vietnam veterans and then to weight findings of similar strength and validity equivalently when drawing conclusions.
The committee has concluded that it would be inappropriate to use quantitative techniques, such as meta-analysis, to combine individual study results into a single summary measure of statistical association. The committee reached this
conclusion because of the many differences among studies in their definitions of exposure, health outcomes considered, criteria for defining study populations, correction for confounding factors, and degree of detail in reporting results. The appropriate use of meta-analysis requires more methodologic consistency across studies, especially in the definition of exposure, than is present for the literature reviewed by the committee (Egger et al., 2002; Petitti, 2000). It is more informative to include a detailed discussion of the results from individual studies in appropriate categories (occupational, environmental, and Vietnam veterans), along with a thorough examination of each study’s strengths and weaknesses.
In general, the committee did not consider case reports, case series, or other published studies that lacked control or comparison groups. An exception was made, however, for early-onset transient peripheral neuropathy. Individual case reports were reviewed because the rapid appearance and transient nature of that condition imposes methodologic constraints that might have precluded the application of standard epidemiologic techniques.
Because the effect of Agent Orange in individuals or groups of veterans is evaluated in terms of disease or medical outcome, attention to disease classification was important to the committee in accurately assembling all pertinent data related to a particular endpoint from various investigations prior to integrating the information. The researchers conducting the studies reviewed by the committee faced the same challenge in reliably interpreting the available documentation when assigning a diagnostic label to a given subject and in then grouping those labels for analysis.
Pathologists, clinicians, and epidemiologists use several classification systems, including the International Classification of Diseases (ICD), International Classification of Diseases—Clinical Modification (ICD-CM), and International Classification of Diseases for Oncology (ICD-O). International Classification of Diseases, 10th Edition (ICD-10) is currently used to classify mortality information. The majority of subjects investigated in the studies cited in this update were diagnosed under earlier systems and most of the articles report results using the International Classification of Diseases, 9th Edition (ICD-9), if they use ICD codes at all, so the committee has also employed ICD-9. ICD codes are a hierarchical system for indicating type of disease and site (for example, ICD-9 162 specifies cancers of the lung, trachea, or bronchus; while more exactly 162.2 represents cancer of the main bronchus of the lung; 162.3, cancer of the upper lobe of the lung; and 162.4, cancer of the middle lobe of the lung). In this report, ICD codes appear almost exclusively in the introductory sections of health outcome discussions (particularly for cancers) to specify precisely what endpoint is being addressed. (See Appendix C for cancer groupings with corresponding ICD-9 and ICD-10 codes.)
For a patient to be correctly diagnosed, careful staging of the extent of disease is necessary and a biopsy of the tissue must be analyzed by microscopy, often with special immunohistochemical stains, to confirm a clinical impression.
Unfortunately, many of the epidemiology studies reviewed by this committee did not use the ICD approach to classification of disease and relied instead on clinical impression alone. Death certificate diagnoses are notoriously inaccurate when they are completed by medical officers who are not familiar with the decedent’s medical history. Self-reported diagnoses, which are obtained from survey questionnaires, often are partially or completely inaccurate. For instance, a patient may state that he was treated for stomach cancer when the correct diagnosis could be gastric adenocarcinoma, gastric lymphoma, pancreatic cancer, large bowel cancer, or peritoneal cancer.
Many epidemiologic studies report the disease outcome by organ system. For instance, the term “digestive system” may be used for conditions that are benign or malignant; affecting the esophagus, stomach, liver, pancreas, small bowel, large bowel, or rectum. Therefore, if a report indicated that a cohort has an increased incidence of digestive system cancer, it would be unclear whether the association was attributable to excess cases of esophageal, gastric, hepatic, pancreatic, or intestinal cancers—or to some combination. Such generalization is further complicated by the fact that the cause of cancer may differ at various anatomical sites; for instance, there are strong associations between gastric cancer and Helicobacter pylori infection, between smoking and squamous cell carcinoma of the esophagus, and between chronic hepatitis B infection and liver cancer. Furthermore, a single site may also experience a carcinogenic response to multiple agents.
Interpretation of the epidemiology literature is further complicated because many studies lack information on the latent period (time from exposure to recognition or diagnosis of disease). Issues surrounding the latent period are discussed in detail in Veterans and Agent Orange: Length of Presumptive Period for Association Between Exposure and Respiratory Cancer (IOM, 2004). The latent period must be considered when evaluating whether there is an increased risk of disease. If a study is looking for an increase in cancer, for example, ample time must have passed since exposure to allow cancers to develop.
The committee has also noted concerns expressed by veterans at public meetings regarding cases of multiple health outcomes, such as multiple cancers, in individuals. Little research has been done to address whether the rate of concurrence is greater than would be expected by chance. Simultaneous analysis of multiple health outcomes could potentially provide more insight into the effect of the chemicals of interest in causing multiple health effects, competing risks between various health outcomes, and the interactive effects of some health endpoints on others.
Rare diseases are also difficult to study because it is hard to accumulate enough cases to permit analysis. This often results in whatever cases are observed being included in a broader, less specific category. Thus, epidemiologic data may not be available for assessing whether a certain rare disease is associated with Agent Orange exposure. In some instances, as for chronic lymphocytic leukemia,
VAO committees have reached conclusions on the basis of the data available and the etiology of the disease. VAO committees could not possibly address every rare disease, however, and they simply do not draw any conclusions on diseases that they have not discussed.
Much of the evidence that the committee considered is drawn from studies of populations who were not in Vietnam during the period when Agent Orange and other herbicides were used as defoliants. The most informative studies are well-documented investigations of occupational exposures to TCDD or specific herbicides, such as 2,4-D or 2,4,5-T. In many studies, TCDD exposure is combined with exposures to an array of “dioxin-like” compounds and the herbicides are often analyzed as members of a functional class, which is less informative for the committee’s purpose than individual results for each specific compound. In the real-world experience investigated in epidemiology studies, exposure to multiple possibly toxic chemicals is the rule rather than the exception; for example, farmers or other agricultural populations are likely to be exposed to insecticides and fungicides, as well as to herbicides. In such studies, the committee looked for evidence of health effects that are associated with the specific compounds contained in the chemicals used as defoliants in Vietnam, with consideration to and adjustment for other possibly confounding exposures.
The quality of exposure information in the scientific literature reviewed by this committee spanned a broad range. Some studies relied on interviews or questionnaires to determine the extent and frequency of exposure. Such self-reported information generally carries less weight than would more objective measures of exposure. To the extent that questionnaire-based information can be corroborated or validated by other sources, its strength as evidence of exposure is enhanced. Written records of chemical purchase or production can provide one type of objective information. Even more useful are scientific measurements of exposure. In many occupational studies, for example, workers wear air-sampling instruments that measure the concentration of a contaminant in each worker’s breathing zone. Measurement of chemicals or their products in biologic specimens, such as blood or urine, also can provide reliable indications of exposure for specific periods. Studies that categorize exposure from well-documented environmental sources of contaminants can be important in the identification of exposed populations, but the results of these studies may be inaccurate when individuals with different levels of exposure are assigned to the same general category of exposure. Studies that explore environmental exposure and disease frequency of populations (e.g., states, counties) are known as ecologic studies. Although ecologic studies vary in their ability to specifically link an exposure to a health outcome, most are considered preliminary or “hypothesis generating”
studies because they lack information on exposure and disease on an individual basis and are unable to address potential confounding factors.
Exposure or dose reconstruction is a particularly challenging aspect of exposure assessment for a population such as Vietnam veterans in which few measurements were made during the period of exposure. Much subsequent work has relied on records of herbicide production and use and on military records of troop locations. A recent effort overseen by the Institute of Medicine has developed a new algorithm for application to Vietnam veterans (the “Stellman model”) using records of herbicide applications in Vietnam and revised data about troop movements (IOM, 2003b). The new information holds promise for use in the estimation of what is called exposure opportunity for veterans; that is, estimates of the amount of herbicides (with characteristic TCDD contamination) applied at particular places over particular time periods. Recent studies of veterans known to have been exposed to herbicides in Vietnam have included collection of blood samples and analysis of TCDD in those samples. The readings from those contemporary samples have been used to identify subgroups of Vietnam veterans with relatively high exposures and also to validate estimates from the Stellman model when extrapolated to the present. Although such analysis clearly is valuable, it also must be viewed with caution. In most cases, the measurement of compounds in blood has taken place many years after exposure. There are numerous difficulties in extrapolating back from contemporaneous TCDD tissue concentrations to estimate TCDD (and indirectly herbicide) doses at the time of first exposure or to maximum exposure; the estimates so derived are subject to substantial uncertainty.
Animal and Mechanistic Studies
Animal models used as surrogates for the study of a human disease must reproduce, with some degree of fidelity, the manifestations of the disease in humans. However, a given effect of herbicide exposure in an animal species cannot always be used to establish its occurrence in humans. In addition to possible species differences, there are many factors that affect the ability to extrapolate from studies in animals to health effects in humans. Animals used in experimental studies are most often exposed to purified chemicals and not to mixtures. Even if herbicide formulations or mixtures are used, the conditions of exposure might not realistically reproduce those exposures that occur in the field. Furthermore, Vietnam veterans were likely exposed to other agents—tobacco smoke, therapeutics, drugs, diesel fumes, or alcohol, for example—that may positively or negatively affect the ability of chemicals contained in herbicides to produce a particular adverse health outcome. There have been few, if any, studies either in humans or experimental animals that have examined those interactions.
As discussed in Chapter 3, TCDD, a contaminant of 2,4,5-T, is thought to be responsible for many of the toxic effects of the herbicides used in Vietnam.
Attempts to establish correlations between the effects of TCDD in experimental systems and in humans are particularly problematic because there are known species-, sex-, and endpoint-specific differences in susceptibility to TCDD toxicity. Some data indicate that humans might be more resistant than are other species to TCDD’s toxic effects (Ema et al., 1994; Moriguchi et al., 2003); other data suggest that, for some endpoints, human sensitivity could be the same as or greater than that of some experimental animals (DeVito et al., 1995). Differences in susceptibility may also be affected by variations in the rate at which TCDD is eliminated from the body. (See Chapter 3 for details on the toxicokinetics of TCDD.)
It also is important to consider TCDD’s mode of action when considering species and strain differences. There is a consensus that most, if not all, of the toxic effects of TCDD involve interaction with the AhR, a protein that binds TCDD and other aromatic hydrocarbons with high affinity. Formation of an active complex, involving the intracellular receptor, the ligand (the TCDD molecule), and other proteins is followed by interaction of the activated complex with specific sites on DNA. That interaction can alter the expression of genes involved in the regulation of cellular processes. The development of mice that lack the AhR has helped to establish a definitive association between the AhR and TCDD-mediated toxicity. The affinity of TCDD for the AhR is species- and strain-specific, and responses to binding of the receptor vary among cell types and developmental stages. Also, genetic differences in the properties of the AhR are known to exist in human populations, as they do in laboratory animals. Thus, individuals may be at greater or lesser risk of the toxic effects of TCDD.
Although studying AhR biology in transformed human cell lines minimizes the inherent error associated with species extrapolations, caution must be exercised because it is still not clear to what extent toxicity is affected by the transformation itself or by the conditions under which cell lines are cultured in vitro.
Some studies are more likely to be published than others. This is the concept of publication bias, which has been documented in biomedical research (Song et al., 2000; Stern and Simes, 1997). Most commonly, bias can be introduced when studies that are statistically significant or are otherwise deemed favorable by their authors are selectively submitted for publication. Conversely, “negative” studies, in which the hypothesis being tested is not supported by the study findings, often go unpublished. Therefore, conclusions about associations between exposure and outcome that are based solely on published results could be subject to bias. Despite that, the committee does not believe its conclusions have been unduly affected by publication bias for two reasons: the extensive publicity surrounding the possibility of health effects associated with the herbicides used in Vietnam
has created considerable pressure to publish all findings on the subject, and the many published studies assembled and reviewed contain among their results the full range of possible statistical associations, from convincingly negative to indeterminate to strongly positive.
Role of Judgment
This committee’s process of reaching conclusions about statistical associations involved more than a formulaic application of quantitative procedures to the assembled evidence. First, the committee had to assess the relevance and validity of individual reports. Then, it had to evaluate the possible influences of measurement error, selection bias, confounding, and chance on the reported results. Next, the committee integrated all of the evidence within and across diverse fields of research. Finally, the conclusions drawn were based on consensus within the committee. Those aspects of the committee’s review required thoughtful consideration of alternative approaches at several points and could not be accomplished by adherence to a narrowly prescribed formula.
The realized approach, as described here, was determined to a large extent by the nature of the exposures, of the health outcomes, and of the resultant evidence available for examination; therefore, it has evolved in the course of the work of this and previous VAO committees. The quantitative and qualitative procedures underlying this review have been made as explicit as possible, but ultimately the conclusions about association expressed in this report are based on the committee’s collective judgment. The committee has endeavored to express its judgments as clearly and precisely as the data allowed.
DeVito MJ, Birnbaum LS, Farland WH, Gasiewicz TA. 1995. Comparisons of estimated human body burdens of dioxin-like chemicals and TCDD body burdens in experimentally exposed animals. Environmental Health Perspectives 103(9):820–831.
Egger M, Ebrahim S, Smith GD. 2002. Where now for meta-analysis? International Journal of Epidemiology 31(1):1–5.
Ema M, Ohe N, Suzuki M, Mimura J, Sogawa K, Ikawa S, Fujii-Kuriyama Y. 1994. Dioxin binding activities of polymorphic forms of mouse and human arylhydrocarbon receptors. Journal of Biological Chemistry 269(44):27337–27343.
IOM (Institute of Medicine). 1994. Veterans and Agent Orange: Health Effects of Herbicides Used in Vietnam. Washington, DC: National Academy Press.
IOM. 2003a. Veterans and Agent Orange: Update 2002. Washington, DC: The National Academies Press.
IOM 2003b. Characterizing Exposure of Veterans to Agent Orange and Other Herbicides Used in Vietnam: Final Report. Washington, DC: The National Academies Press.
IOM. 2004. Veterans and Agent Orange: Veterans and Agent Orange: Length of Presumptive Period for Association Between Exposure and Respiratory Cancer. Washington, DC: The National Academies Press.
Moriguchi T, Motohashi H, Hosoya T, Nakajima O, Takahashi S, Ohsako S, Aoki Y, Nishimura N, Tohyama C, Fujii-Kuriyama Y, Yamamoto M. 2003. Distinct response to dioxin in an arylhydrocarbon receptor (AHR)-humanized mouse. Proceedings of the National Academy of Sciences of the United States of America 100(10):5652–5657.
NRC (National Research Council). 1999. Arsenic in Drinking Water. Washington, DC: National Academy Press.
NRC. 2001. Arsenic in Drinking Water: 2001 Update. Washington, DC: National Academy Press.
Petitti DB. 2000. Meta-Analysis, Decision Analysis, and Cost-Effectiveness Analysis: Methods for Quantitative Synthesis in Medicine. New York: Oxford Press.
Song F, Eastwood AJ, Gilbody S, Duley L, Sutton AJ. 2000. Publication and related biases. Health Technology Assessment 4(10):1–115.
Stern JM, Simes RJ. 1997. Publication bias: Evidence of delayed publication in a cohort study of clinical research projects. British Medical Journal 315(7109):640–645.