Summary
Over the last few decades, people have debated the benefits and hazards of fluoride exposure. The Centers for Disease Control and Prevention has recognized water fluoridation as one of the greatest public-health achievements, and others have claimed that fluoride exposure causes various adverse health effects. In 2006, the National Academies of Sciences, Engineering, and Medicine (the National Academies) reviewed the scientific literature on the health effects of fluoride exposure and concluded that chronic fluoride exposure can cause enamel fluorosis and weakening of bone that could increase the risk of fracture. Studies of the potential neurotoxicity of fluoride exposure lacked sufficient detail and did not allow definitive conclusions. However, the National Academies report concluded that the consistency of the results on neurotoxicity warranted further investigation. Since the 2006 report, several epidemiologic studies of fluoride exposure and neurodevelopmental and cognitive effects have been published. That research and a nomination from the Fluoride Action Network (FAN) prompted the National Toxicology Program (NTP) to conduct a systematic review of the evidence of adverse neurodevelopmental and cognitive effects of fluoride exposure. NTP’s conclusions are summarized in the monograph Systematic Review of Fluoride Exposure and Neurodevelopmental and Cognitive Health Effects.1
To ensure the integrity of its report, NTP asked the National Academies to review the monograph. As a result of that request, the National Academies convened the present committee. It is important to note that the committee was tasked with reviewing the monograph and focused its efforts on evaluating whether evidence as presented in the monograph supported NTP’s conclusions. Thus, it did not conduct its own independent evaluation of the evidence, and it did not conduct a data audit (an independent review of all the data reported in the monograph to identify omissions or errors in reporting). However, it did review some key literature to enable its review of the monograph. The committee’s findings and suggestions for improvements are contained in this report; some overarching findings concerning methods, assessment of animal and human evidence, and NTP’s hazard conclusion are provided here.
METHODS AND COMMUNICATION
The protocol for the systematic review described in the monograph was published on NTP’s Web site in June 2017 and made available for public com-
___________________
1 Referred to hereafter as the monograph.
ment. It was reviewed several times by technical advisers selected for their expertise on this topic. In general, it describes the overall systematic-review process and clearly outlines modifications that were made during the review. Thus, NTP appears to be adhering to best practices for systematic reviews with respect to the availability and documentation of such a protocol before initiation of a review.
The committee, however, identified several issues associated with the protocol. First, the role of the Office of Health Assessment and Translation (OHAT) handbook in developing the protocol is unclear. The protocol scarcely refers to the OHAT handbook and does not discuss the role of the handbook in its development. That ambiguity leads to concerns about the lack of detail in the protocol and about apparent conflicts between methodologic approaches in the protocol and the handbook itself.
Second, important details are missing from the protocol, including information on the strategy used to update the experimental animal literature, expertise and experience of review team members, and the planned conduct of statistical analyses. It does not provide explicit exclusion and inclusion criteria for study selection, which are critical for transparency of the process and reproducibility of the findings. It also does not provide justification for some of its decisions, for example, regarding screening parameters or what information to make publicly available, such as the list of excluded studies.
Third, there are some inconsistencies in the details provided in the protocol and the methods ultimately implemented in the monograph, including how mechanistic data would be considered, how the outcome assessment would be conducted, and which confounders were identified as critical covariates. Those discrepancies are troubling because inconsistencies between the protocol and the monograph raise questions about how the process was actually conducted, about what changes were made, and about when and why modifications were implemented.
The committee found some issues associated with data presentation and communication of various aspects of the process that are discussed further in the context of the evaluation of the animal and human evidence. One particular aspect of communication needs to be emphasized here. Many people are interested in whether water fluoridation to prevent tooth decay poses a threat to human neurodevelopment and cognition. Although the monograph provides some discussion of dose–response relationships, NTP did not conduct a formal dose–response assessment and needs to state clearly that the monograph is not designed to be informative regarding decisions about fluoride concentrations for water fluoridation.
ANIMAL EVIDENCE
The monograph presents a systematic review of animal studies of fluoride exposure related to learning and memory that were published from 2015 to August
20, 2019, as an update to the NTP systematic review published in 2016. Examination of the animal studies published since 2015 led NTP to conclude that the animal data are inadequate to support conclusions on human cognitive effects.
The committee has serious concerns about the risk-of-bias evaluations of the animal literature and whether they identified important threats to internal validity that are specific to neurobehavioral outcomes in animal tests. The guidance in the protocol touched on some of the threats, but insufficient details appear to have been provided to ensure a rigorous, consistent evaluation of neurobehavioral studies. Specifically, the committee had concerns about the risk-of-bias evaluations for attrition, outcome assessment, and statistical analyses. It also found one element—maternal, fetal, and pup toxicity—that did not appear to have been adequately captured in the risk-of-bias criteria. Although severe postnatal toxicity was mentioned in risk-of-bias evaluations of some studies, it is unclear whether maternal, fetal, and pup toxicity was routinely assessed for all studies. Such effects can seriously confound interpretation of neurodevelopmental effects. Overall, the committee found that some studies cited in the monograph had severe methodologic shortcomings that could warrant exclusion from the body of evidence.
NTP justifies its conclusion that the animal evidence is inadequate on the grounds that it is not possible to separate cognitive effects from effects on locomotor activity. Although locomotor activity can affect learning and memory outcomes, it has been demonstrated many times that the presumed influence of locomotor activity on learning and memory does not occur. Thus, the committee does not agree with NTP’s rationale for dismissing the animal evidence and finds that it is a mistake to dismiss studies of learning and memory because of minor, brief locomotor-activity changes or when other assessments can rule out confounding locomotor effects in cognitive assessments.
Given the serious concerns raised by the committee in the present report, NTP will need to decide whether it should reanalyze the animal evidence. The committee cautions, however, that given the poor quality of the animal studies that it reviewed, revising the systematic review to address the concerns highlighted might not affect the ultimate finding that the animal evidence is inadequate to inform conclusions about fluoride exposure and neurodevelopmental and cognitive effects in humans.
HUMAN EVIDENCE
NTP based its conclusion in the monograph primarily on human evidence. NTP considered the human evidence to be “relatively robust” and evaluated the association of fluoride exposure and neurodevelopmental and cognitive effects as reported in 82 publications. Although it evaluated all publications, its confidence in its conclusion is primarily based on the studies that were rated as having a lower risk of bias; NTP concluded that studies rated with a higher risk of bias did not
affect its confidence in its hazard conclusion. The committee had substantive concerns regarding NTP’s evaluation of the human evidence as noted below.
The strategy used for the literature search indicated that NTP used FAN as a source to identify relevant literature. The process by which FAN identified and selected studies is unclear, and that uncertainty raises the question of whether the process could have led to a biased selection of studies. Such a concern raises the need for a formal evaluation of any potential bias that might have been introduced into the literature-search process. Another issue with the literature is that it appears that multiple publications are based on a single study and thus should not be considered independent studies. That lack of independence needs to be addressed in evaluating the findings and conclusions.
Several issues in the evaluation of risk of bias of individual studies were identified. First, there appeared to be inconsistent application of the risk-of-bias criteria across studies, perhaps stemming from differences in the approaches presented in the protocol and monograph. Second, the committee identified many cases in which NTP’s evaluation of confounding was insufficient, difficult to understand, or applied inconsistently across studies. NTP should develop clear criteria that are defined in the protocol to identify critical confounders and, if these are not consistently applied to individual studies, explain why some potential confounders are considered to be of greater importance in some studies and not others. NTP should also address critical aspects of confounding, such as magnitude and directionality. Third, NTP noted the possibility of exposure misclassification in several cases but did not discuss its likely magnitude and direction and did not discuss it in the context of whether a given study reported an association. The failure to address exposure misclassification thoroughly and consistently raises the question of whether NTP’s evaluations were sufficient and supported its conclusion. Fourth, it is imperative to protect examiners from information about exposure that could bias their administration and interpretation of outcome assessments, especially when they are assessing cognition or other neurobehavioral outcomes in human studies. Several studies reviewed by NTP did include information on techniques of blinding of examiners, but many did not. Because failure to blind examiners might result in a high risk of bias of study results and conclusions, NTP should consider this aspect more carefully when assessing the risk of bias of human studies. Fifth, NTP in some cases classified studies as having a low risk of bias when the measure of the neurodevelopmental and cognitive outcome was seriously flawed. Given the importance of that outcome in determining whether fluoride is hazardous, its proper measurement should be considered more carefully. Finally, the committee is concerned that the studies included in the systematic review did not undergo rigorous statistical review. That flaw is problematic because some of the studies identified as having low risk of bias did not adequately account for the hierarchical structure of their data or had errors in their summary statistics—faults that compromised their internal validity.
The committee also identified several issues with the analysis, summary, and presentation of the data. A key conclusion of the monograph is that the results of the epidemiologic studies consistently show a positive association. Although
the desire to provide a simple summary of a complex array of evidence is understandable, doing so requires comparing studies that have similar parameters, and this was not done in the monograph. In fact, the studies that are reviewed in the monograph used various measures of fluoride exposure and analytic techniques and evaluated neurodevelopmental and cognitive outcomes at different developmental times. The committee recognizes that drawing conclusions always requires aggregating or summarizing data that have some degree of heterogeneity among other considerations, but the monograph should juxtapose results across broadly comparable studies and use that information to provide a text summary of the patterns observed. If comparing “like to like” results yields consistent results across all measures, ages, exposure sources, statistical approaches, and exposure ranges, taking random error into account, that will indeed warrant a statement that results consistently show adverse effects. However, the monograph does not provide the evidence in a manner that leads to that conclusion. The committee notes that NTP did not conduct a meta-analysis. Given that meta-analysis is a useful tool for aggregating and summarizing data and analyzing comparable studies, the committee strongly recommends that NTP reconsider its decision not to perform one.
Lastly, the discussion section of the monograph provides an informal assessment of the evidence with regard to exposure and concludes that adverse health effects are observed largely in association with exposures above those associated with water fluoridation. The basis of that conclusion is not apparent and seems to contradict the earlier assertion that nearly all the studies are positive, including ones that evaluated groups exposed to lower concentrations. More important, as noted above, this discussion gives a false impression that NTP conducted a formal dose–response assessment. NTP should be clear that the monograph cannot be used to assess what concentrations of fluoride are safe.
NTP CONCLUSION
The monograph “concludes that fluoride is presumed to be a cognitive neurodevelopmental hazard to humans. This conclusion is based on a consistent pattern of findings in human studies across several different populations showing that higher fluoride exposure is associated with decreased IQ or other cognitive impairments in children.” The committee was tasked with assessing whether NTP satisfactorily supports its conclusion. Given the issues raised by the committee regarding the analysis of various aspects of some studies and the analysis, summary, and presentation of the data in the monograph, the committee does not find that NTP has adequately supported its conclusion. That finding does not mean that the conclusion is incorrect; rather, further analysis or reanalysis as noted in the present report is needed to support conclusions in the monograph.
