A critical review of the literature is a three-part process. First, multiple databases are searched for information on the dietary supplement ingredient and other substances with similar taxonomical, structural, or functional properties. Such searches are broad-based, and include information on safety and biological activity of the ingredients, including human data, animal data, and in vitro data.
Second, each primary research paper is reviewed for internal consistency; for example, are proper methodologies used? Do data fit the conclusions? Are the associations real? Is appropriate information included? Are there chance, bias, confounding variables, a lack of coherence, or other significant internal issues or limitations that should be taken into account?
Third, the external consistency of the research papers must be judged as a group. Are the studies coherent as a whole? Is there strength in the associations, general agreement, etc.? Studies can then be sorted into those that suggest that there is little risk of illness or injury when consuming the supplement ingredient, those that indicate a relevant concern for risk of illness or injury, and those that have equivocal results. Each should then be examined for flaws and strengths in accordance with the principles and concepts discussed in the subsequent chapters on each general category of data (Chapters 4 through 7).
Focused Versus Broad-Based Evaluation. An integrative evaluation might be reactive to the signal and focused in nature, in that it is conducted to examine a specific moderate- or high-level concern about an ingredient, or it might be more proactive and broad-based, in that it is looking for any risk associated with use of the dietary supplement ingredient. As described above in the description of the signal detection component, a proactive integrative evaluation might be initiated simply because a large percentage of the population is using the ingredient, rather than as a reaction to a particular safety concern.
The amount of information gathered depends on the nature of the harmful effect that is the focus of concern. If a focused evaluation is conducted, it is assumed that less information will be reviewed. However, the relative importance of an individual study is established in conjunction with an evaluation of other relevant literature. Clearly, data or information outside the primary safety concern may include information that has a direct bearing on the overall evaluation of the safety concern identified in the signal component. Thus a comprehensive review can provide information that may raise concerns in other areas not relevant to the focus, but which should not be ignored in a safety monograph.