in the peer-reviewed literature with additional effort. Researchers working in state agencies need both incentives and support to publish study findings. Many studies reviewed by the committee would be of great interest to researchers in other areas. The committee encourages activities, such as the Agency for Toxic Substances and Disease Registry (ATSDR) State Environmental Health Information Clearinghouse, to make these gray-literature reports available to others.

Public-health agencies need a combination of speed and low cost in their research efforts. Thus they use readily available data for populations and geographic regions, and many gray-literature studies used an ecologic design. The disadvantages of this approach are not fully and widely understood; in the literature reviewed for this report, many studies had significant shortcomings. More research on methods, improvements in the methods already available, and better training are needed for scientists using this approach. It would be beneficial to develop guidelines for typical health-department studies to assist practitioners. These might be versions of existing texts used by academic epidemiologists.

Another common limitation in these studies was their inappropriate use of exposure and health-outcome databases that do not provide adequate information for this type of health study. While some of the databases need to be and could be improved, researchers need greater sophistication about their limitations.

Yet another problem is the lack of nonexposed or less-exposed persons as controls for many studies. Often, regional rates were compared with national rates, which can give misleading results because the national rates do not take into account confounders (e.g., race or socioeconomic status) in the study population. Appropriate comparison groups must be used to correct for possible confounders.

The weakest aspects of most of these studies was their use of imprecise measures of exposure or small study populations, so type II errors may often occur (failing to reject the null hypothesis when it should be rejected). There were often unavoidable limitations in the size of the exposed population. Creative approaches (e.g., meta-analysis) may help to overcome this problem. Also, it may be possible to group related health outcomes in ways that increase statistical power. Many adverse reproductive outcomes are uncommon, so that even a large relative risk may be difficult to detect.

Conclusions and Recommendations Concerning the Gray Literature

We conclude that most studies and reports in the gray literature have serious limitations, such as lack of adequate exposure information, that

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement