members read each study critically and considered its relevance and quality. The committee did not collect original data, nor did it perform any secondary data analysis.

It should be noted that our available scientific tools—toxicology and epidemiology—are inadequate to illuminate clearly the human health effects of individual components of complex mixtures of the type experienced by Gulf War veterans. In many cases, the committee found “inadequate/insufficient evidence of an association” between the exposure of concern and a health outcome; that may have been due to a lack of clear evidence because of the inadequacy of those tools rather than to the absence of effects.

Inclusion Criteria

The committee’s next step, after securing the full text of some 800 epidemiologic studies, was to determine which studies would be included in the review as primary or support studies. Therefore, the committee developed inclusion criteria. For a study to be included in the committee’s review it had to meet specified criteria, for example, methodologic rigor, identification of class or agent, specificity of health outcome, an exposure assessment, and in some cases, an exposure-free interval. For relevance to the Gulf War veterans, the committee focused on long-term health outcomes that persist after exposure ceases.

Considerations in Assessing the Strength of the Evidence

The committee’s process of reaching conclusions about the various agents and their potential for adverse health outcomes was collective and interactive. Once a study was included in the review because it met the committee’s criteria, there were several considerations in assessing the strength of an association. They are patterned after those introduced by Hill in 1971 and include strength of the evidence for an association, dose-response relationship, temporal relationship, consistency of association, specificity of association, and biologic plausibility. The committee also considered whether alternative explanations or errors—such as bias, confounding, and chance1—might account for the finding of an association.

The committee’s final judgment, therefore, is based on a balance between the strength of support of an association and the degree of exclusion of alternatives. The evaluation of evidence to reach conclusions about statistical associations goes beyond quantitative procedures; several stages during the review required thoughtful consideration and judgment and could not always be accomplished by adherence to a prescribed formula.

The approach described here evolved throughout the process of review and was determined in important respects by the nature of the evidence, exposures, and health outcomes being examined. Both quantitative and qualitative aspects of the process were important to the overall review. Ultimately, the conclusions expressed in this report are based on the committee’s collective judgment.


The committee classified the evidence of an association between exposure to a specific agent and a specific health outcome into five categories. The categories have been developed by


Chance refers to sampling variability.

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement