table journals and historic information. Of course, all published reviews cannot claim to be current after they are published, but some will be updated with new information, if available. Evaluations that use resources that do not consider recent data should be supplemented with information available after publication of the review to determine if new data might have affected safety conclusions.
Are the primary sources cited accurately and completely? Accurate citations are one indication that the original literature is being used and support the accuracy of the interpretation of the data. Incomplete or incorrectly cited references take away from the credibility and usefulness of the resource.
Is the review well balanced and objective? Is there a conflict of interest (financial or otherwise) relative to the outcome of the safety evaluation? Conflicts of interest should lead the user to be more skeptical in interpreting the review’s assessment of safety as unbiased and appropriate. In considering the objectivity of a particular evaluation, the starting assumptions should be identified. For example, there is a difference between concluding that a substance can be consumed without safety concerns because the data reviewed included relevant safety information and it was determined to be of little concern, compared with concluding that a substance can be consumed without concern but not having any data that provided information related to safety. Guidelines for authors of scientific journals include the importance of disclosing financial relationships (Campbell, 2001), underscoring the importance of full disclosure relative to the information included in the review
Do the authors have the depth and breadth of expertise necessary to assess the primary sources, weigh the data, and make conclusions as to the adequacy of the data for safety assessment? Persons knowledgeable in safety assessment should be included in the evaluations. In addition, particular types of expertise may be needed depending on the safety issues being considered for the particular supplement.
Was the authored information peer-reviewed by knowledgeable experts? A good safety review will utilize the expertise of a variety of appropriate experts. Peer review is an essential component and, as such, the review should be critically reviewed by experts from a variety of disciplines. As the National Research Council (NRC, 1998) has stated, “External experts often can be more open, frank, and challenging to the status quo than internal reviewers, who may feel constrained by organizational concerns. Evaluation by external reviewers thus can enhance the credibility of the peer review process by avoiding both the reality and the appearance of conflict of interest.”
If appropriate, is variability of specific preparations addressed or acknowledged? Do reviewers address or acknowledge the limitations in