ratio of 3 or more generally represents a strong association, although choice of this cut-point is debatable (Stolley, 1990; Temple, 1999). Similarly, as a rule of thumb, a relative risk or odds ratio of 2 or less generally represents a weak association (Temple, 1999).
In addition to the relative risk or odds ratio values, a measurement of uncertainty (e.g., standard error) is needed to properly assess the level of concern that should be attached to a finding. Confidence intervals and p values use both the estimated relative risk or odds ratio and the standard error of these estimates in their calculations. In most medical, biological, and health services literature, a p value (the probability of the observed data if the null hypothesis is true) of 0.05 or less is considered statistically significant. This cut-point translates into incorrectly rejecting the null hypothesis 1 time or fewer out of 20, on average. Although a p value of ≤ 0.05 is commonly used to determine when a result warrants attention, in interpreting studies regarding safety, a cutoff point of p < 0.05 is often not appropriate because of its implications. Dietary supplements are regulated similarly to foods and are presumed to be safe (the null hypothesis). A p value under this null hypothesis reflects the probability of the observed data when assuming that the dietary supplement is safe. A p value of 0.05 means that on average, one would incorrectly reject the assumption that a supplement is safe 1 time out of 20. For serious adverse events, when there is a high prevalence of use or when the supplement is used in special populations, a p value greater that 0.05 might raise the level of concern substantially. A p value of 0.10 would mean that the probability of the observed number of adverse events is 0.10 if one assumes that the supplement is safe (i.e., on average one expects to see this number of adverse events 1 time out of 10 if the supplement is safe). This type of finding could be enough to raise the concern level. Knowledge that at this level one incorrectly rejects the hypothesis of safety 1 out of 10 times on average should enable this information to be appropriately integrated with other types of information (e.g., animal data).
Finally, confidence intervals of relative risks or odds ratios may be more useful than p values in interpreting results. A 95 percent confidence interval typically is used and means that 95 percent of the calculated confidence intervals are expected to contain the true relative risk if the estimation were repeated a large number of times in similar study settings. However, as with p values, the choice of the value 95 percent should be used with full understanding of its consequences and meaning. Studies where a 95 percent confidence interval for a relative risk or an odds ratio covers 1 (thus indicating lower significance statistically) may still offer important information about safety. A recent example where it was decided that there existed a high safety concern even without having statistical significance is