National Academies Press: OpenBook
« Previous: 1 Introduction
Suggested Citation:"2 Committee's Approach." National Academies of Sciences, Engineering, and Medicine. 2018. Gulf War and Health: Volume 11: Generational Health Effects of Serving in the Gulf War. Washington, DC: The National Academies Press. doi: 10.17226/25162.
×

2

Committee’s Approach

To address the statement of task given in P.L. 144-315, the National Academies of Sciences, Engineering, and Medicine (the National Academies) empaneled a multidisciplinary committee composed of 16 experts from the fields of toxicology, epidemiology, environmental health and occupational medicine, epigenetics, and statistics. The Volume 11 committee began its deliberations with an open public meeting held in Washington, DC, to hear from the Department of Veterans Affairs (VA), interested veterans and veterans’ service organizations, and other stakeholders and to gather information from the National Institute for Environmental Health Sciences and the National Toxicology Program. That input helped the committee understand VA’s needs, appreciate the veterans’ concerns, and be cognizant of complementary efforts already under way.

Given the precedence set by Volumes 6, 7, and 9 in the Gulf War and Health series, the Volume 11 committee included military personnel who had served in the Gulf region after 1991, particularly those who had served after September 11, 2001 (Post-9/11 veterans), in its considerations. This decision was justified because of the overlap of many deployment-related exposures and the anticipated need to proactively address potential health effects in these groups. To understand the potential reproductive and developmental1 effects that might occur in veterans and descendants as a result of deployments to the 1990–1991 Gulf War and the Post-9/11 conflicts, the Volume 11 committee divided its statement of task into two tasks with four activities:

Task 1:

  • Assessment of the current research on possible generational health effects that may be the result of deployment exposures during the 1990–1991 Gulf War and Post-9/11 conflicts (see Chapters 4 through 7); and

___________________

1 The committee considered “reproductive effects” to include changes in reproductive function that may affect fertility and the ability to carry a healthy child through to a term birth. “Developmental effects” is used to represent any outcomes in offspring observed at birth and across the lifecourse. See Box 2-1 for a list of effects.

Suggested Citation:"2 Committee's Approach." National Academies of Sciences, Engineering, and Medicine. 2018. Gulf War and Health: Volume 11: Generational Health Effects of Serving in the Gulf War. Washington, DC: The National Academies Press. doi: 10.17226/25162.
×

Task 2:

  • Development of a framework for monitoring the reproductive health of veterans and the health of their descendants (see Chapter 9);
  • Determination of the feasibility of conducting an epidemiologic study on the reproductive and developmental effects of toxicants in veterans and their descendants and, if feasible, provision of guidance on the design of such a study (see Chapter 9); and
  • Provision of guidance on what research should be conducted to address the data and knowledge gaps, including the use of animal and mechanistic studies (see Chapter 10).

The committee began its work on these tasks by developing an approach to identifying whom might be affected by deployment exposures (i.e., parents, children, grandchildren), what exposures a veteran might have experienced (e.g., lifestyle, toxicants, stress), the characteristics of those exposures (e.g., measuring exposure, the variability of exposures, multiple exposures), and what factors might influence the transmission of effects from parent to descendants (e.g., levels and timing of exposure). The committee then considered how to determine what reproductive and developmental effects may be associated with the toxicants listed in the Gulf War legislation (see Chapter 1, Box 1-1). This determination began with an extensive literature search and review using the evaluation processes established by previous Gulf War and Health committees. The evaluation process, including the categories of association used by the committee to draw its conclusions regarding the strength of the association between an exposure and any reproductive or developmental health effect, is described later in this chapter. The results of the committee’s deliberations for deployment-related exposures, pesticides, combustion products, and solvents are given in Chapters 4 through 7 and summarized in Chapter 8.

The committee’s approach to Task 2 was also driven by the literature as well as by the committee’s expertise. Both the committee’s framework for monitoring veteran and descendant populations for health effects and its guidance on conducting epidemiologic studies that might help link deployment exposures to health outcomes in descendants relied on literature describing studies that are gathering information on veterans’ exposures (deployed and nondeployed) and cataloging their health effects during and after military service. Determining what research is necessary to understand how exposures may contribute to health effects in offspring and future generations was informed by the data reviewed in Chapters 4 through 7—including the genetic and epigenetic literature—that highlighted data and knowledge gaps. The transmission of effects from a veteran to his or her children and possibly grandchildren or great-grandchildren is called generational effects.

EXPOSURE CONSIDERATIONS

To understand the potential impact of deployment exposures on veterans and their descendants, the committee needed to consider any reproductive and developmental outcomes in the broader context of the veterans’ genetic and epigenetic profiles and of their exposures to other stressors (biological, chemical, radiological, psychological, or physiological) they had experienced in their lives before, during, and after deployment. Prior Gulf War and Health volumes have examined the variety of deployment-related exposures (e.g., living conditions, threat of biological and chemical warfare) a veteran might experience in the Persian Gulf region (e.g., IOM, 2006) as well as in Iraq and Afghanistan (e.g., IOM, 2011). Deployment itself has been used as an exposure variable and stressor in several Gulf War and Health reports with deployed veterans being compared with veterans who remained in the United States or who were deployed elsewhere (e.g., Germany, Somalia, Bosnia [see IOM, 2006, 2010; NASEM, 2016]).

Suggested Citation:"2 Committee's Approach." National Academies of Sciences, Engineering, and Medicine. 2018. Gulf War and Health: Volume 11: Generational Health Effects of Serving in the Gulf War. Washington, DC: The National Academies Press. doi: 10.17226/25162.
×

Gulf War and Health, Volume 6 found “that military personnel deployed to a war zone, even if direct combat was not experienced, have the potential for exposure to deployment-related stressors and that the emotional and physical reactions of military personnel to those stressors can vary widely” (IOM, 2008).

Veteran Exposures

Throughout life, a person is exposed to any number of external and internal events and agents, including environmental toxicants and those that are the result of lifestyle choices, that can interact with a person’s genome and epigenome2 to affect health and the occurrence of disease. The combined effects of genetic makeup, lifestyle, and environment, all of which may affect a person from conception to death, set the stage for a person’s response to potentially harmful exposures. Veterans’ exposure during deployment must be considered within the context of their lifetime exposure to adverse events (e.g., psychological and physical stress, accidents, diseases) and toxicants (e.g., drugs, cleaning products, pesticides). Nutrition, smoking, and stress are but a few of the factors that can affect a person’s health. For example, if a veteran was a smoker before deployment and continued to smoke during and after deployment, that exposure may influence his or her response (susceptibility) to other toxicants experienced during deployment.

Military service itself—and not just deployment experiences—may also contribute to a veteran’s exposures. As discussed in Gulf War and Health, Volume 6, on deployment-related stress, deployment to a war zone itself can be a stressor regardless of whether the service member engaged in direct combat. Stresses such as being away from family, seeing casualties, and poor living conditions are among the many factors that may result in or contribute to adverse health effects for deployed service members (IOM, 2008). Certain military occupational specialties that are engaged in prior to or after deployment may involve exposure to toxicants such as solvents for degreasing equipment and chemical-agent-resistant coating paint. These exposures may be compounded during deployment. For example, a mechanic may continue to use solvents for degreasing during deployment but might have additional exposures to mixtures of other toxicants, such as particulate matter from dust storms and combustion products in smoke from burn pits. Determining those additional exposures may be difficult given that the composition of the dust storm and what is disposed of in the pit may vary with time and location. Exposures during military training and those encountered in the civilian environment, such as cleaning products and various other products that are dependent upon lifestyle, hobbies, and social activities, may also affect a veteran’s cumulative exposures and, ultimately, lead to health effects.

Assessing Deployment Exposures

Several important issues must be considered when evaluating the health impacts of stressors and toxicants in veteran populations. These issues include the route of exposure; the magnitude, duration, and frequency of the exposure; possible interactions between multiple single exposures; exposure to mixtures; and the time in the lifecourse at which the exposure occurs. Characterizing veteran exposures is difficult because of the almost complete lack of exposure information collected during U.S. deployments to the Gulf War or Post-9/11 conflicts (IOM, 2011) or to the Vietnam War. Nevertheless, as noted in the report Strategies to Protect the Health of Deployed U.S. Forces: Analytical Framework for Assessing Risks, “Many hazards are specific to military situations, military exposure factors can differ from those relevant to civilians, and stress and extreme environments can affect toxic responses” (NRC, 2000).

___________________

2 See the glossary in Appendix B for definitions of terms.

Suggested Citation:"2 Committee's Approach." National Academies of Sciences, Engineering, and Medicine. 2018. Gulf War and Health: Volume 11: Generational Health Effects of Serving in the Gulf War. Washington, DC: The National Academies Press. doi: 10.17226/25162.
×

Only rarely is environmental sampling conducted by the Department of Defense (DoD) that would help in determining or modeling the levels of a toxicant or toxicants to which veterans might be exposed. Although air or water models may be helpful for estimating exposure for an area, they cannot be used to assess individual exposures. What sampling is conducted tends to be only a snapshot at a given time and location and does not capture the totality of a veteran’s exposure during deployment or even for a particular incident, such as the exposure to sarin gas that may have occurred with the demolition of the Khamisiyah munitions depot in Iraq (IOM, 2006). Furthermore, some veterans’ exposures were specific to deployment to the 1990–1991 Gulf War or Post-9/11 conflicts (e.g., pyridostigmine bromide [PB], burn pits), although others were not (e.g., diesel and solvents).

Route of Exposure

The route of exposure can affect the dose of a toxicant a person receives. It is important to know whether the route of exposure for a veteran in a given situation is comparable to those examined in epidemiologic and occupational or animal studies. For example, if a veteran is typically exposed to a solvent by handling rags soaked in a solvent, but occupational studies show that most workers are exposed via inhalation, will these different routes of exposure affect the potential for reproductive outcomes differently? Additionally, there is the question of whether the doses and the routes of administration used in animal studies are relevant surrogates for human exposures and what extrapolations or uncertainty factors are necessary to compare animal and human exposures.

Multiple Exposures and Chemical Interactions

Gulf War and Post-9/11 veterans were, and continue to be, potentially exposed to numerous toxicants in the environment (co-exposures, e.g., dust and solvents) or together in the same exposure (mixtures, e.g., smoke from burn pits). These exposures include agents used as preventive measures (e.g., PB, vaccines, pesticides, and insecticides), hazards of the natural environment (e.g., sand and endemic infectious agents), job-specific toxicants (e.g., paints, solvents, and diesel fumes), war-related toxicants (e.g., depleted uranium and smoke from burn pits), hazards from cleanup operations (e.g., sarin and cyclosarin), persistent organic pollutants (e.g., lindane and dioxins), and numerous other stressors (e.g., smoking and physical or psychological stress). Thus, deployed veterans have exposures that vary by toxicant or stressor; the frequency, duration, and timing of the dose; individual response; and the theater of operation. The number and combination of toxicants to which a veteran might be exposed make it difficult to determine whether any specific agent or combination of agents is likely to result in reproductive effects in the veterans or developmental effects in their descendants. Adding to the complexity of assessing multiple exposures is that although Gulf War veterans for the most part experienced single deployments to the Persian Gulf area during the war, many Post-9/11 veterans have been deployed more than once and to different areas of southwest Asia, for instance, Iraq and Afghanistan. For extended conflicts in the future, multiple deployments may be common.

Addressing exposure to multiple toxicants has been a subject of much debate and research in the fields of toxicology, environmental and occupational health, and medicine. Although it is possible to assess the toxicity of some mixtures rather than assessing the toxicity of their individual constituents (Elhalwagy and Zaki, 2009), the interactions of toxicants are not, in general, well understood in either humans or animals. However, there are exceptions; for example, the toxic effects of some pesticide mixtures have been studied (Hernández et al., 2013, 2017). In animal studies of Gulf War illness, the effects of mixtures of pesticides and other agents such as PB have been studied (Abdel-Rahman et al., 2004).

Suggested Citation:"2 Committee's Approach." National Academies of Sciences, Engineering, and Medicine. 2018. Gulf War and Health: Volume 11: Generational Health Effects of Serving in the Gulf War. Washington, DC: The National Academies Press. doi: 10.17226/25162.
×

Although many scientific studies evaluate the risks of adverse health effects caused by exposure to a single toxicant, the European Union, the U.S. Environmental Protection Agency (EPA), and the Agency for Toxic Substances and Disease Registry (ATSDR) have developed systematic approaches to evaluating multiple exposures (ATSDR, 2018; EPA, 1986, 2000; EU, 2011). Exposure to multiple agents or stressors may result in interactions that produce an effect that might not have occurred with exposure to a single toxicant, or the interactions might produce a greater, different, or lesser effect than would be caused by a single exposure. Simultaneous exposure to more than one agent might produce not only a variety of adverse effects and severity of those effects; protective or beneficial effects might also occur. Therefore, attributing an effect to one toxicant or a combination of toxicants is difficult, especially in situations where there are many different types of exposures (e.g., chemical, physical, biological). Prior National Academies committees have attempted to determine the toxicity of some mixtures to which veterans were exposed while deployed in southwest Asia. The 2011 IOM report Long-Term Health Consequences of Exposure to Burn Pits in Iraq and Afghanistan reviewed the air sampling done by the DoD and found numerous flaws in the sampling methodology that made it difficult to determine if military personnel stationed at bases with burn pits were likely to have an increased risk of health problems. The Volume 11 committee notes that burn pit emissions were only one poorly defined mixture to which service members might be exposed during their deployment. As a result, the committee did not evaluate all studies of occupational and environmental exposures to the many mixtures in the literature that included a toxicant of interest (such as benzene in the hydrocarbon mixture called BTEX [benzene-toluene-ethylbenzene-xylene]) but instead, generally restricted itself to examining studies that assessed the individual components of a mixture. This decision was made because, given the available information, the committee was unable to determine if a given mixture was analogous to mixtures that might have been present during deployment.

Timing of Exposure

The timing of exposure can refer to a number of different things: when the exposure occurred during a lifetime; when the exposure occurred relative to the manifestation of an effect; how long the exposure lasted (duration); and how frequently the exposure occurred. In this report, the committee focuses on the timing of exposure relative to conception or to a particular window of gamete, fetal, or child development and on the possible manifestation of effects in men, women, fetuses, or children. These effects include direct toxic effects on development and genetic or epigenetic effects.

Exposures that may lead to reproductive effects in men or women can occur at any time. Parental exposures that may affect the development of a fetus or child can occur before conception (during the development or maturation of the germ cell) or during pregnancy (prenatal). Developmental toxicity studies are generally limited to specific windows of development relative to conception and birth (shortly before conception through early life). The timing of exposure is particularly important for understanding inherited genetic and epigenetic effects (see Chapter 3 for more details).

There are only a few rare studies of human populations in which it has been possible to say with certainty when an exposure to a toxicant began and ended. Rather, most exposures are continuous or intermittent over extended periods of time: for example, living in a hot climate or polluted area or being a tobacco user. Other exposures may be infrequent or isolated, such as using a pesticide or an industrial accident. Deployment exposures, on the other hand, occur during a defined time frame of limited duration, usually many months.

Assessing the exposures of female service members poses some unique challenges. Female service members who are pregnant are precluded from being deployed (DoD, 2010). If female veterans are

Suggested Citation:"2 Committee's Approach." National Academies of Sciences, Engineering, and Medicine. 2018. Gulf War and Health: Volume 11: Generational Health Effects of Serving in the Gulf War. Washington, DC: The National Academies Press. doi: 10.17226/25162.
×

pregnant during deployment, they are removed from the theater of operation, so their exposures should be limited to preconception or early pregnancy (after the pregnancy is detected). This is not to say that a female veteran’s military occupational specialty or other duties after she returns from deployment will not expose her to toxicants, but it might be difficult to determine whether any adverse pregnancy outcomes in a veteran or developmental effects in her children could be attributed specifically to her deployment exposures. Given different occupations and military assignments, it is possible for women to serve on active duty during later stages of pregnancy, but not to be deployed (DoD, 2007). An exception to a female service member’s exposure to toxicants being limited to preconception or early pregnancy is her possible exposure during deployment to persistent organic pollutants such as lindane or dioxins. These chemicals may be taken up during deployment and stored in adipose tissue. However, should a woman become pregnant after deployment, these toxicants may be mobilized from the adipose tissue and enter the blood stream or milk of a pregnant or lactating woman, thus exposing her child during pregnancy and during lactation.

The Volume 11 committee found that studies that specifically assessed exposure prior to conception or early in the pregnancy were particularly useful, but they were also relatively rare. Because the timing of exposures in relation to a pregnancy is difficult to determine in epidemiologic studies, the committee took into account those studies where maternal exposures were assessed from preconception through birth (e.g., prenatal), including later in the pregnancy or at the time of birth, even though female veterans were not deployed late in pregnancy.

COMMITTEE’S REVIEW PROCESS

The approach and methods that the committee used to identify, select, and evaluate the scientific and medical literature on Gulf War and Post-9/11 exposures are, for the most part, the same as those used by previous Gulf War and Health committees. The committee also used a similar process to reach conclusions on the strength of the associations between those exposures and reproductive or developmental health effects. The Gulf War legislation was the primary source of the committee’s list of possible toxicants (see Box 1-1), although the committee also considered additional toxicants such as fuels and combustion products, as described in the IOM report Long-Term Health Consequences of Exposure to Burn Pits in Iraq and Afghanistan (2011), a report of concern to Post-9/11 veterans. The committee was asked to assess hexavalent chromium at the request of VA. Exposures relevant to Vietnam veterans are discussed in the most recent update of the Veterans and Agent Orange series (NASEM, 2016b), although dioxins are also considered in this volume because dioxins are among the many combustion products measured in emissions from burn pits in Iraq and Afghanistan.

Identification of the Literature

The Volume 11 committee began its work by overseeing extensive searches of the scientific literature, including published articles, other reports, and government documents. Searches were conducted using five databases: Embase, Medline, Scopus, Proquest, and PubMed. The specific searches targeted studies of (1) the health of Gulf War or Post-9/11 veterans published since Volume 10 without regard for specific toxicants or outcomes; (2) reproductive effects associated with any of the toxicants listed in Box 1-1; and (3) developmental or other health outcomes, including biomarkers of response, in children or grandchildren (F1 or F2) associated with the parental exposures to the toxicants of concern. Studies in both human populations and animal or cellular models were included. No publications that appeared before 1999 were considered in order to focus on data published since the Gulf War and Health series began.

Suggested Citation:"2 Committee's Approach." National Academies of Sciences, Engineering, and Medicine. 2018. Gulf War and Health: Volume 11: Generational Health Effects of Serving in the Gulf War. Washington, DC: The National Academies Press. doi: 10.17226/25162.
×

The searches retrieved more than 29,000 publications on reproductive effects and more than 56,000 publications related to effects in offspring, which were published through January 2018. Publications that came to the committee’s attention after January 2018 were included in the review where appropriate.

Because few studies on reproductive or developmental effects were specific to veterans’ exposures during deployment, the committee decided to approach its review of the health effects by identifying populations that had exposures to the same toxicants as Gulf War and Post-9/11 veterans. For the studies of occupational and environmental cohorts, it is difficult to ascertain how comparable the deployment exposures are to those of the study cohorts in terms of exposure magnitude, duration, frequency, mixtures, and co-exposures and also in terms of population characteristics such as gender, age, ethnicity, and lifestyle. For example, several large cohort studies examined associations between environmental exposures to pesticides and particulate matter and developmental effects on children, such as neurodevelopmental deficits or asthma in childhood (Eskenazi et al., 2007; Tetreault et al., 2016). Those studies examined long-term, continuous exposures throughout the preconception, prenatal, and postnatal periods. These exposures may not accurately reflect exposures during only preconception or early pregnancy periods as experienced by female veterans who were pregnant in theater because late pregnancy and postnatal exposure windows can have a significant impact on development and child health. However, the committee took note of studies that reported trimester-specific exposure associations or that tried to control for effects related to postnatal exposures.

The committee realizes that considerable work is being done to study epigenetic effects in nonmammalian organisms such as fruit flies (Drosophila), zebrafish (Danio rerio), frogs (Xenopus), and roundworms (Caenorhabditis elegans). However, although studies of relevant exposures and reproductive and developmental effects in mammalian models such as rodents and rabbits were included in the review, nonmammalian models were excluded because their immediate relevance to human health is difficult to determine at this time.

In addition to identifying studies on distinct health outcomes such as infertility, birth outcomes, and diseases in offspring, the committee also included studies of biological markers that have been associated with the health outcomes of interest. These studies of biological markers were particularly informative in assessing the potential implications of an association between the exposures of interest and generational effects—for example, Guzick et al. (2001), Khatun et al. (2018), Ombelet et al. (2014), and Patel et al. (2017). The committee considered reproductive and developmental effects as reported in the literature, without a priori decisions on what would be included in each of those areas and regardless of whether it had been identified in an earlier volume of the Gulf War and Health series. Once the literature was screened and a list of effects was generated, the effects were then categorized as reproductive effects, including adverse pregnancy outcomes; developmental effects; and genetic and epigenetic effects, as described in Box 2-1. Because the effects varied somewhat depending on the toxicant, the description of the literature for each toxicant also varies. Although adverse pregnancy outcomes are described throughout this report as a subset of reproductive outcomes, conclusions about those outcomes are made separately, depending on the availability of data. The committee decided not to assess studies on menopause (other than early menopause) or reproductive cancers because they have little relevance for generational effects.

Studies that did not appear to be relevant, based on an assessment of their title and abstract, were deleted from further consideration. The inclusion and exclusion criteria used to select relevant studies are given in Box 2-2. The results were categorized by the toxicants of concern, and specific publication cutoff dates were applied to each exposure based on the latest relevant Gulf War and Health review or a review by another authoritative body such as EPA and ATSDR. Those committee members who had the most familiarity with a particular health outcome or area of expertise reviewed all screened titles

Suggested Citation:"2 Committee's Approach." National Academies of Sciences, Engineering, and Medicine. 2018. Gulf War and Health: Volume 11: Generational Health Effects of Serving in the Gulf War. Washington, DC: The National Academies Press. doi: 10.17226/25162.
×

and abstracts in that area and identified papers for full text retrieval; titles and abstracts of more than 4,000 publications were reviewed.

Evaluation Process

The Volume 11 committee adopted a policy of using only literature that had been published in a peer-reviewed journal or rigorously peer-reviewed government reports or monographs. While the process of peer review by fellow professionals increases the likelihood that higher-quality studies will appear in the

Suggested Citation:"2 Committee's Approach." National Academies of Sciences, Engineering, and Medicine. 2018. Gulf War and Health: Volume 11: Generational Health Effects of Serving in the Gulf War. Washington, DC: The National Academies Press. doi: 10.17226/25162.
×

literature, it does not guarantee the validity or quality of any particular study or the generalizability of its findings. The committee did not collect original data nor did it perform any secondary data analysis.

For a study to be included in the committee’s review, it had to meet specific criteria. It had to (1) have sufficient detail to demonstrate rigorous methodology (for example, it had a control or reference group and adjusted for confounders when needed); (2) have sufficient power to detect an effect; and (3) provide information regarding a persistent health outcome.

Small groups of committee members conducted a preliminary review of the relevant studies for each set of exposures to determine what, if any, relevant information the studies had on the health outcomes of interest. Each study was assigned to a committee member who reviewed it critically for its relevance and quality without preconceived ideas about the strength of the evidence regarding the association between an exposure and a reproductive or developmental health effect. The responsible committee member or members then presented the information from the preliminary screening and categorization to the full committee for discussion.

The information presented for each exposure included a review of the relevant prior Gulf War and Health volumes and their studies and conclusions, the populations used in any new studies, the methods for selecting and evaluating the populations, the study results, and the committee members’ assessments of the strengths and limitations of the study. Because of the variability in the description of the reproductive and developmental effects considered in this report it was impossible for the committee to make a priori assumptions about the usefulness of any study.

Suggested Citation:"2 Committee's Approach." National Academies of Sciences, Engineering, and Medicine. 2018. Gulf War and Health: Volume 11: Generational Health Effects of Serving in the Gulf War. Washington, DC: The National Academies Press. doi: 10.17226/25162.
×

Greater weight was given to studies that were conducted in a manner that reduced sources of error, bias, and confounding. More weight was given to studies in which there was an individual-level exposure assessment or biological assessment of exposure (e.g., urine analysis); in which specific exposures were associated with an occupational title or industry, such as when a job exposure matrix was used to categorize exposure; or in which an assessment was made by an industrial hygienist. Studies that had self-reported exposures were considered less rigorous. For greater detail about epidemiology, biases, confounders and other methodological background, see Chapter 2 of Gulf War and Health, Volume 1 and Volume 8 (IOM, 2000, 2010).

Given the amount of human and animal literature on the reproductive effects of these exposures, the committee also considered reviews from authoritative bodies, such as EPA and ATSDR or other reports from the National Academies. Those reviews follow systematic methodologies, summarize human and animal data for the purposes of identifying human health risks, and undergo rigorous review. The conclusions of these authoritative bodies are presented in this volume and were taken into account by the committee in the context of the time period when they were produced as well as the purpose and approach used by the authoritative bodies. The committee examined any literature that had been published since those reviews, had not been assessed in those reviews, or was of particular interest (such as multigenerational studies), but it relied on the totality of information to reach conclusions. Systematic reviews and meta-analyses in the published literature were also considered by the committee, but they were given no greater weight than any individual study.

After the studies were discussed in a plenary session, the responsible committee member or members drafted the text for that exposure and health outcome. Human and animal data were summarized by outcomes for each exposure. Data and units are presented as reported in the cited studies, except where otherwise noted.

The draft text was reviewed and discussed in further plenary sessions until all committee members reached a consensus on the description of the studies and the conclusions for each health outcome. After this language was agreed upon, the full committee assigned a category of association (discussed later in this chapter) based on the weight of the evidence and expert judgment. It should be noted that the committee did not use a formulaic approach in deciding the number of studies that would be necessary to assign a specific category of association. Rather, the committee’s review consisted of a thoughtful, nuanced, and qualitative consideration of all of the studies as well as expert judgment, and this could not be accomplished by adherence to a narrowly prescribed formula of what data would be required for each category of association or for a particular health outcome. In its evaluation of the evidence, the committee considered a set of overarching concerns—specifically, statistical considerations, the availability of animal and mechanistic studies, the consistency of association, and biologic plausibility—when assigning a category of association to a particular exposure and adverse reproductive or developmental effect.

Statistical Considerations

The strength of an association between an exposure and an outcome is generally estimated quantitatively by using prevalence ratios, relative risks, odds ratios, hazard ratios, or correlation or regression coefficients, depending on the study design. These various measures of the strength of the association are generally referred to as effect sizes. A ratio greater than 1.0 indicates that the outcome variable has occurred more frequently in the exposed group than in the unexposed or control group, and a ratio less than 1.0 indicates that it has occurred less frequently. When the effect size is 1, there is no effect. For correlations (usually denoted with r) and regression coefficients (usually denoted with β), values greater than 0 indicate that the outcome variable increases with exposure and negative values indicate they decrease; when the effect size is 0, there is no effect.

Suggested Citation:"2 Committee's Approach." National Academies of Sciences, Engineering, and Medicine. 2018. Gulf War and Health: Volume 11: Generational Health Effects of Serving in the Gulf War. Washington, DC: The National Academies Press. doi: 10.17226/25162.
×

Effect sizes are typically reported with a confidence interval (CI) to quantify random error. Statistical significance may be represented by a CI or a p-value. A p-value is defined as the probability of observing an effect size as large as or larger than the one observed when there is no effect. In the scientific literature, a result associated with a p-value less than 0.05 is generally referred to as statistically significant. There is a connection between p-value and 95% confidence intervals. Specifically, if the effect size associated with no effect (1 for ratios and 0 for correlation and regression coefficients) is not included in a 95% confidence interval, then the p-value is less than 0.05. If the 95% CI for a risk estimate (such as a risk ratio or odds ratio) includes 1.0, the association is not considered to be statistically significant.

Generally, the greater the effect size, the greater the likelihood that the exposure–disease association is causal and the lower the likelihood that it is due to undetected error, bias, or confounding. Measures of statistical significance, such as p-values, are not indicators of the strength of an association. Small increases in relative risks that are consistent among studies, however, might be evidence of an association, and some forms of extreme bias or confounding can produce a high relative risk. A study must have enough statistical power to be able to detect effects of a certain magnitude, which is especially important in identifying negative results. The committee reviewed the studies in this report with a consideration not only for uncertainty due to sampling variability, measurement error, and biological variability but also for the potential for bias and the authors’ strategies for examining and limiting the effect of each of these on the study’s findings. The committee gave less weight to studies that included only p-values or statements of statistical significance, but no CI. This is because, on their own, p-values provide no information of the effect sizes. With a large enough sample size, a p-value can be close to 0, even when the effect size is practically negligible. Common types of biases found in the Gulf War veteran literature are identified in Box 2-3 and discussed in greater detail in Chapter 9, Statistical Considerations. For greater detail about epidemiology, biases, confounders, and other methodological background, see Chapter 2 of Gulf War and Health, Volume 8 (IOM, 2010).

In its assessment of the scientific literature, the committee was careful to consider the statistical approaches used in the studies it reviewed. In particular, the committee examined whether possible confounding effects were considered and appropriately accounted for when model-based approaches, such as multivariable linear regression, were used. Many of the studies also estimated a large number of associations and ran many statistical tests on a single dataset. For example, in many air pollution studies, the investigators compared multiple outcomes (i.e., a long list of birth defects including multiple levels of specificity from general to specific subcategories) and many measures of exposure (i.e., multiple pollutants and multiple estimates of exposure). In such cases, researchers are likely to find statistically significant associations at the p<0.05 level even by chance alone: that is, 5% of cases will appear to have an association just by chance alone when using p<0.05. There are ways to correct for multiple comparisons in studies, such as a Bonferroni correction (a statistical adjustment for multiple comparisons) which raises the standard of proof needed when an investigator examines a wide array of hypotheses simultaneously by proportionally decreasing the p-value used to evaluate associations.

Animal, Cellular, and Mechanistic Studies

Animal, cellular, and mechanistic studies were included in the committee’s considerations because they provide valuable information on reproductive and developmental effects that may not be evident in human populations or that cannot ethically be studied, such as spontaneous abortion. Furthermore, in animal studies the route, dose, timing of exposure, and mixture or co-exposure composition may be controlled. Specific organ systems can also be examined before effects are clinically apparent and confounders can be carefully controlled.

Suggested Citation:"2 Committee's Approach." National Academies of Sciences, Engineering, and Medicine. 2018. Gulf War and Health: Volume 11: Generational Health Effects of Serving in the Gulf War. Washington, DC: The National Academies Press. doi: 10.17226/25162.
×

Experimental studies are of particular use for studying the generational effects of toxicants. Most laboratory animals reproduce rapidly, and thus generation effects in the F0 (parental), F1, F2, and F3 generations may be studied in a matter of months or a few years, compared with the decades-long maturation and reproductive cycle of humans.

Well-established experimental animal protocols exist that minimize the potential influence of co-exposures, control for background responses, and more clearly characterize dose–response relationships.

Suggested Citation:"2 Committee's Approach." National Academies of Sciences, Engineering, and Medicine. 2018. Gulf War and Health: Volume 11: Generational Health Effects of Serving in the Gulf War. Washington, DC: The National Academies Press. doi: 10.17226/25162.
×

Still, experimental animal studies can be challenging to interpret because most studies evaluate a single strain of a species with a homogeneous genetic background, while humans have a large amount of genetic variability shaping their response to stressors, which makes comparisons problematic. There are also animal studies that have examined the impact of mixed exposures (mixtures and co-exposures), and the committee decided that such studies could help inform some of its deliberations. The committee reviewed mammalian toxicologic studies, as those were deemed most relevant for identifying agents of potential concern for humans. Therefore, the committee referred to experimental animal studies to help provide information on biologic plausibility and the specificity of effects and to fill in gaps where no human data were available.

Animal research may also focus on a toxicant’s mechanism of action (i.e., how a toxicant exerts its deleterious effects at the cellular and molecular levels). Mechanism-of-action (or mechanistic) studies encompass a range of laboratory approaches with whole animals and in vitro systems using tissues or cells from humans or animals. In some cases, the committee used in vitro and/or mechanistic studies to support or enhance its understanding of the plausibility of the results observed in animal models if those models provided evidence of reproductive effects or processes that may lead to molecular or organ damage in fetuses or children. Some mechanistic studies also provided evidence of epigenetic changes that may have effects on cellular processes that might affect reproduction or produce developmental effects.

Consistency of Association

To conclude that an association is consistent requires that the exposure–effect relationship be found regularly in a variety of studies—for example, in more than one study population and with different study methods. However, consistency alone is not sufficient evidence of an association. The committee considered findings that were consistent in direction among different categories of studies to be supportive of an association. It did not require exactly the same magnitude of association to be present in different populations in order to conclude that there was a consistent association. A consistent association could occur when the results of most studies were positive and the differences in measured effects were within the range expected on the basis of sampling error, selection bias, misclassification, confounding, and differences in dose.

Thus, for a health effect to be associated with an agent, there had to be corroboration—that is, replication of findings among multiple studies and populations and under relevant conditions. The greater the degree to which an effect could be consistently reproduced, the more confidence the committee had that there was a true effect.

Biologic Plausibility

Biologic plausibility is based on knowledge of a biologic mechanism by which a toxicant can lead to a health effect. That knowledge is derived from mechanism-of-action or other types of studies—typically animal studies. Biologic plausibility is often difficult to establish and may not exist when an association is first documented. The extent to which all the data in a study are consistent and in line with a biologically plausible mechanism influences the weight attached to the results of that study, as does evidence that the mechanism is similar between the animal or animals under study and humans. This point is especially relevant for the transgenerational impacts that this committee evaluated.

Suggested Citation:"2 Committee's Approach." National Academies of Sciences, Engineering, and Medicine. 2018. Gulf War and Health: Volume 11: Generational Health Effects of Serving in the Gulf War. Washington, DC: The National Academies Press. doi: 10.17226/25162.
×

CATEGORIES OF ASSOCIATION

The committee expressed its judgment of the available data clearly and precisely in the Synthesis and Conclusions section for each toxicant discussed in Chapters 4 through 7. It agreed to use the categories of association that have been established and used by previous Gulf War and Health committees and other National Academies committees (IOM, 2000, 2003, 2005, 2006, 2007, 2008, 2009, 2010). Those categories of association have gained wide acceptance by Congress, government agencies (particularly VA), researchers, and veterans groups.

The five categories below describe different levels of association and present a common message: the validity of an association is likely to vary in line with the extent to which common sources of spurious associations can be ruled out as the reason for the observed association. Accordingly, the criteria for each category express a degree of confidence based on the extent to which sources of error were reduced. The committee discussed the evidence and reached consensus on the categorization of the evidence for each toxicant.

Sufficient Evidence of a Causal Relationship

Evidence is sufficient to conclude that a causal relationship exists between being exposed to a toxicant and a reproductive or developmental effect in humans. The evidence fulfills the criteria for sufficient evidence of a causal association in which chance, bias, and confounding can be ruled out with reasonable confidence. The association is supported by a consideration of the consistency of association and biologic plausibility from experimental studies.

Sufficient Evidence of an Association

Evidence suggests an association, in that a positive association has been observed between an exposure and a reproductive or developmental effect in humans; however, there is some doubt as to the influence of chance, bias, and confounding. Experimental animal and cellular studies either supported or diminished the committee’s decision in making these associations.

Limited/Suggestive Evidence of an Association

Some evidence of an association between exposure and a reproductive or developmental effect in humans exists, but this is limited by the presence of substantial doubt regarding chance, bias, and confounding. The evidence in experimental animal or cellular models was used to strengthen or weaken the associations between exposure and effect.

Inadequate/Insufficient Evidence to Determine Whether an Association Exists

The available studies are of insufficient quantity, quality, validity, consistency, or statistical power to permit a conclusion regarding the presence or absence of an association between an exposure and a reproductive or developmental effect in humans. Evidence in experimental animal studies may have provided additional support for these assessments, but, in the absence of human data, it was not used to strengthen the categorization of the association. In some cases, the body of evidence is too small to permit conclusions, such as when there are no available studies to validate or corroborate the findings of a single study. In other cases, there is evidence from human or animal studies, but the heterogeneity

Suggested Citation:"2 Committee's Approach." National Academies of Sciences, Engineering, and Medicine. 2018. Gulf War and Health: Volume 11: Generational Health Effects of Serving in the Gulf War. Washington, DC: The National Academies Press. doi: 10.17226/25162.
×

of exposures, outcomes, and methods leads to inconsistent findings that preclude the committee from identifying an association between exposure and effect.

Limited/Suggestive Evidence of No Association

There are several adequate studies, covering the full range of levels of exposure that humans are known to encounter, that are consistent in not showing an association between an exposure and a reproductive or developmental effect. A conclusion of no association is inevitably limited to the conditions, levels of exposure, and length of observation covered by the available studies. In addition, the possibility of a very small increase in risk at the levels of exposure studied can never be excluded. Experimental animal studies were examined to support such findings.

REFERENCES

Abdel-Rahman, A., S. Abou-Donia, E. El-Masry, A. Shetty, and M. Abou-Donia. 2004. Stress and combined exposure to low doses of pyridostigmine bromide, DEET, and permethrin produce neurochemical and neuropathological alterations in cerebral cortex, hippocampus, and cerebellum. Journal of Toxicology and Environmental Health A 67(2):163–192.

ATSDR (Agency for Toxic Substances and Disease Registry). 2018. Framework for assessing health impacts of multiple chemicals and other stressors. Atlanta, GA: U.S. Department of Health and Human Services, Public Health Service. https://www.atsdr.cdc.gov/interactionprofiles/ipga.html (accessed August 6, 2018).

DoD (Department of Defense). 2007. Guide to Female Soldier Readiness. U.S. Army Center for Health Promotion and Preventive Medicine Technical Guidance 281. January 23.

DoD. 2010. Department of Defense Instruction Number 6490.07: Deployment-Limiting Medical Conditions for Service Members and DoD Civilian Employees. Under Secretary of Defense for Personnel and Readiness. February 5, 2010.

Elhalwagy, M.E., and N. Zaki. 2009. Comparative study on pesticide mixture of organophosphorus and pyrethroid in commercial formulation. Environmental Toxicology and Pharmacology 28(2):219–224.

EPA (U.S. Environmental Protection Agency). 1986. Guidelines for the health risk assessment of chemical mixtures. EPA/630/R-98/002. September 24. Federal Register 51(185):34014–34025.

EPA. 2000. Supplementary guidance for conducting health risk assessment of chemical mixtures. EPA/630/R-00/002. Washington, DC: Risk Assessment Forum Technical Panel, Environmental Protection Agency.

Eskenazi, B., A.R. Marks, A. Bradman, K. Harley, D.B. Barr, C. Johnson, N. Morga, and N.P. Jewell. 2007. Organophosphate pesticide exposure and neurodevelopment in young Mexican-American children. Environmental Health Perspectives 115(5):792–798.

EU (European Union). 2011. Toxicity and assessment of chemical mixtures. Brussels: Scientific Committee on Health and Environmental Risks, Scientific Committee on Emerging and Newly Identified Health Risks, and Scientific Committee on Consumer Safety, European Commission. http://ec.europa.eu/health/scientific_committees/environmental_risks/docs/scher_o_155.pdf (accessed August 6, 2018).

Guzick, D.S., J.W. Overstreet, P. Factor-Litvak, C.K. Brazil, S.T. Nakajima, C. Coutifaris, S.A. Carson, P. Cisneros, M.P. Steinkampf, J.A. Hill, D. Xu, and D.L. Vogel, for the National Cooperative Reproductive Medicine Network. 2001. Sperm morphology, motility, and concentration in fertile and infertile men. New England Journal of Medicine 345(19):1388–1393.

Hernández, A.F., F. Gil, and M. Lacasaña. 2017. Toxicological interactions of pesticide mixtures: An update. Archives of Toxicology 91(10):3211–3223.

Hernández A.F., T. Parrón, A.M. Tsatsakis, M. Requena, R. Alarcón, and O. López-Guarnido. 2013. Toxic effects of pesticide mixtures at a molecular level: Their relevance to human health. Toxicology 307:136–145.

IOM (Institute of Medicine). 2000. Gulf War and health, volume 1. Depleted uranium, pyridostigmine bromide, sarin, and vaccines. Washington, DC: National Academy Press.

IOM. 2003. Gulf War and health, volume 2: Insecticides and solvents. Washington, DC: The National Academies Press.

IOM. 2005. Gulf War and health, volume 3: Fuels, combustion products, and propellants. Washington, DC: The National Academies Press.

IOM. 2006. Gulf War and health, volume 4: Health effects of serving in the Gulf War. Washington, DC: The National Academies Press.

IOM. 2007. Gulf War and health, volume 5: Infectious diseases. Washington, DC: The National Academies Press.

Suggested Citation:"2 Committee's Approach." National Academies of Sciences, Engineering, and Medicine. 2018. Gulf War and Health: Volume 11: Generational Health Effects of Serving in the Gulf War. Washington, DC: The National Academies Press. doi: 10.17226/25162.
×

IOM. 2008. Gulf War and health, volume 6: Physiologic, psychologic, and psychosocial effects of deployment-related stress. Washington, DC: The National Academies Press.

IOM. 2009. Gulf War and health, volume 7: Long-term consequences of traumatic brain injury. Washington, DC: The National Academies Press.

IOM. 2010. Gulf War and health, volume 8: Update of health effects of serving in the Gulf War. Washington, DC: The National Academies Press.

IOM. 2011. Long-term health consequences of exposure to burn pits in Iraq and Afghanistan. Washington, DC: The National Academies Press.

Khatun A., M.S. Rahman, and M.G. Pang. 2018. Clinical assessment of the male fertility. Obstetrics & Gynecology Science 61(2):179–191.

NASEM (National Academies of Sciences, Engineering, and Medicine). 2016a. Gulf War and health, volume 10: Update of health effects of serving in the Gulf War. Washington, DC: The National Academies Press.

NASEM. 2016b. Veterans and Agent Orange: Update 2014. Washington, DC: The National Academies Press.

NRC (National Research Council). 2000. Strategies to protect the health of deployed U.S. forces: Analytical framework for assessing risks. Washington, DC: National Academy Press.

Ombelet, W., N. Dhont, A. Thijssen, E. Bosmans, and T. Kruger. 2014. Semen quality and prediction of IUI success in male subfertility: A systematic review. Reproductive BioMedicine Online 28(3):300–309.

Patel, A.S., J.Y. Leong, and R. Ramasamy. 2017. Prediction of male infertility by the World Health Organization laboratory manual for assessment of semen analysis: A systematic review. Arab Journal of Urology 16(1):96–102.

Tetreault, L.F., M. Doucet, P. Gamache, M. Fournier, A. Brand, T. Kosatsky, and A. Smargiassi. 2016. Childhood exposure to ambient air pollutants and the onset of asthma: An administrative cohort study in Quebec. Environmental Health Perspectives 124(8):1276–1282.

Suggested Citation:"2 Committee's Approach." National Academies of Sciences, Engineering, and Medicine. 2018. Gulf War and Health: Volume 11: Generational Health Effects of Serving in the Gulf War. Washington, DC: The National Academies Press. doi: 10.17226/25162.
×
Page 25
Suggested Citation:"2 Committee's Approach." National Academies of Sciences, Engineering, and Medicine. 2018. Gulf War and Health: Volume 11: Generational Health Effects of Serving in the Gulf War. Washington, DC: The National Academies Press. doi: 10.17226/25162.
×
Page 26
Suggested Citation:"2 Committee's Approach." National Academies of Sciences, Engineering, and Medicine. 2018. Gulf War and Health: Volume 11: Generational Health Effects of Serving in the Gulf War. Washington, DC: The National Academies Press. doi: 10.17226/25162.
×
Page 27
Suggested Citation:"2 Committee's Approach." National Academies of Sciences, Engineering, and Medicine. 2018. Gulf War and Health: Volume 11: Generational Health Effects of Serving in the Gulf War. Washington, DC: The National Academies Press. doi: 10.17226/25162.
×
Page 28
Suggested Citation:"2 Committee's Approach." National Academies of Sciences, Engineering, and Medicine. 2018. Gulf War and Health: Volume 11: Generational Health Effects of Serving in the Gulf War. Washington, DC: The National Academies Press. doi: 10.17226/25162.
×
Page 29
Suggested Citation:"2 Committee's Approach." National Academies of Sciences, Engineering, and Medicine. 2018. Gulf War and Health: Volume 11: Generational Health Effects of Serving in the Gulf War. Washington, DC: The National Academies Press. doi: 10.17226/25162.
×
Page 30
Suggested Citation:"2 Committee's Approach." National Academies of Sciences, Engineering, and Medicine. 2018. Gulf War and Health: Volume 11: Generational Health Effects of Serving in the Gulf War. Washington, DC: The National Academies Press. doi: 10.17226/25162.
×
Page 31
Suggested Citation:"2 Committee's Approach." National Academies of Sciences, Engineering, and Medicine. 2018. Gulf War and Health: Volume 11: Generational Health Effects of Serving in the Gulf War. Washington, DC: The National Academies Press. doi: 10.17226/25162.
×
Page 32
Suggested Citation:"2 Committee's Approach." National Academies of Sciences, Engineering, and Medicine. 2018. Gulf War and Health: Volume 11: Generational Health Effects of Serving in the Gulf War. Washington, DC: The National Academies Press. doi: 10.17226/25162.
×
Page 33
Suggested Citation:"2 Committee's Approach." National Academies of Sciences, Engineering, and Medicine. 2018. Gulf War and Health: Volume 11: Generational Health Effects of Serving in the Gulf War. Washington, DC: The National Academies Press. doi: 10.17226/25162.
×
Page 34
Suggested Citation:"2 Committee's Approach." National Academies of Sciences, Engineering, and Medicine. 2018. Gulf War and Health: Volume 11: Generational Health Effects of Serving in the Gulf War. Washington, DC: The National Academies Press. doi: 10.17226/25162.
×
Page 35
Suggested Citation:"2 Committee's Approach." National Academies of Sciences, Engineering, and Medicine. 2018. Gulf War and Health: Volume 11: Generational Health Effects of Serving in the Gulf War. Washington, DC: The National Academies Press. doi: 10.17226/25162.
×
Page 36
Suggested Citation:"2 Committee's Approach." National Academies of Sciences, Engineering, and Medicine. 2018. Gulf War and Health: Volume 11: Generational Health Effects of Serving in the Gulf War. Washington, DC: The National Academies Press. doi: 10.17226/25162.
×
Page 37
Suggested Citation:"2 Committee's Approach." National Academies of Sciences, Engineering, and Medicine. 2018. Gulf War and Health: Volume 11: Generational Health Effects of Serving in the Gulf War. Washington, DC: The National Academies Press. doi: 10.17226/25162.
×
Page 38
Suggested Citation:"2 Committee's Approach." National Academies of Sciences, Engineering, and Medicine. 2018. Gulf War and Health: Volume 11: Generational Health Effects of Serving in the Gulf War. Washington, DC: The National Academies Press. doi: 10.17226/25162.
×
Page 39
Suggested Citation:"2 Committee's Approach." National Academies of Sciences, Engineering, and Medicine. 2018. Gulf War and Health: Volume 11: Generational Health Effects of Serving in the Gulf War. Washington, DC: The National Academies Press. doi: 10.17226/25162.
×
Page 40
Next: 3 Introduction to the Study of Generational Effects »
Gulf War and Health: Volume 11: Generational Health Effects of Serving in the Gulf War Get This Book
×
Buy Paperback | $90.00 Buy Ebook | $69.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

For the United States, the 1991 Persian Gulf War was a brief and successful military operation with few injuries and deaths. However, soon after returning from duty, a large number of veterans began reporting health problems they believed were associated with their service in the Gulf. At the request of Congress, the National Academies of Sciences, Engineering, and Medicine has been conducting an ongoing review of the evidence to determine veterans' long-term health problems and potential causes.

Some of the health effects identified by past reports include post-traumatic stress disorders, other mental health disorders, Gulf War illness, respiratory effects, and self-reported sexual dysfunction. Veterans’ concerns regarding the impacts of deployment-related exposures on their health have grown to include potential adverse effects on the health of their children and grandchildren. These concerns now increasingly involve female veterans, as more women join the military and are deployed to war zones and areas that pose potential hazards.

Gulf War and Health: Volume 11 evaluates the scientific and medical literature on reproductive and developmental effects and health outcomes associated with Gulf War and Post-9/11 exposures, and designates research areas requiring further scientific study on potential health effects in the descendants of veterans of any era.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!