National Academies Press: OpenBook

Proposed Framework for Evaluating the Safety of Dietary Supplements -- For Comment (2002)

Chapter: 5 Framework Steps One and Two: Screening/Flagging and Priority Setting

« Previous: 4 Factors Considered in Screening, Setting Priorities, and Safety Evaluation
Suggested Citation:"5 Framework Steps One and Two: Screening/Flagging and Priority Setting." Institute of Medicine. 2002. Proposed Framework for Evaluating the Safety of Dietary Supplements -- For Comment. Washington, DC: The National Academies Press. doi: 10.17226/10456.
×

5
Framework Steps One and Two: Screening/Flagging and Priority Setting

The proposed framework begins the evaluation of the safety of dietary supplement ingredients with a process of screening each ingredient and then flagging those that should receive highest priority for an in-depth critical safety evaluation. Using the factors described in Chapter 4, it is possible to utilize readily available information resources to aid in identifying those dietary supplement ingredients that warrant further evaluation and to prioritize them for the evaluation process. In this chapter the proposed general approach to setting priorities for review is described, followed by a discussion of how screening and priority setting can be done. It is assumed that the screening/flagging and priority-setting steps will be completed by Food and Drug Administration (FDA) staff, but FDA may instead choose to contract these processes out to a suitable scientifically based organization.

GENERAL APPROACH

The initial goal of the framework is to organize the factors described in chapter 4 into a structure that enables regulators to focus on the supplement ingredients that require the most attention. A three-step framework is proposed to focus such efforts. The three-step process consists of (1) screening and flagging based on readily available information, (2) priority setting based on analysis of available information, and (3) critically evaluating the data regarding safety of supplement ingredients. It is designed to help FDA first make initial judgments that reduce the large number of ingredients currently in the U.S. marketplace to a manageable number. The ingredients are then categorized by priority based on the greatest likelihood of potential harm. This will allow FDA to focus on conducting in-depth critical safety assessments for ingredients determined to be of high priority (Step 3, described in Chapter 6).

As was discussed in Chapter 4, data regarding key factors are used in one or more of the steps of the framework, but there are differences in how the different factors, modifiers, and new ingredient status are considered in each of the three steps. For example, new ingredient status is assigned an integral place in the screening/flagging step, but it plays no role in the subsequent safety assessment. In contrast, there are also factors that are not explicitly considered as discrete factors in the screening/flagging step but which may play an integral role in the priority-setting or critical safety evaluation steps.

Suggested Citation:"5 Framework Steps One and Two: Screening/Flagging and Priority Setting." Institute of Medicine. 2002. Proposed Framework for Evaluating the Safety of Dietary Supplements -- For Comment. Washington, DC: The National Academies Press. doi: 10.17226/10456.
×

In summary, the goal of Step One, screening/flagging, is to identify those supplement ingredients that could possibly cause harm. The goal of Step Two, priority setting, is to determine which of the ingredients identified in the screening/flagging step are of the highest concern and should therefore be placed at highest priority for a full evaluation. The goal of Step Three, critical safety evaluation (discussed in Chapter 6), is to provide a detailed review of what is known about the theoretical and demonstrated safety of a dietary supplement ingredient that would allow FDA to determine if further action is needed in regulating the ingredient in the marketplace.

STEP ONE: SCREENING/FLAGGING

The screening/flagging step was developed on the premise that it is not feasible for FDA to immediately search extensively for information about every dietary supplement ingredient. Readily available information can be used to flag substances that warrant further attention while maintaining enough sensitivity to minimize false negatives and not omit any items with potential safety concerns.

To flag substances warranting some level of attention, “yes or no” questions were developed to identify ingredients that should move forward to Step Two, the priority- setting step. A “yes” answer to any of the following questions flags the dietary supplement ingredient and moves it on to Step Two.

  1. Has a 75-day new ingredient notification been filed with FDA?

  2. Are there serious adverse events reported through MedWatch, poison control centers, or clinical studies that illustrate a pattern in terms of the type of incident reported, that are well-documented in the medical literature, or that may be plausibly linked to the dietary supplement ingredient? Or, does the number of serious adverse events reported appear high compared to the ingredient’s prevalence of use? Or, does it seem plausible that particular subpopulations are particularly susceptible to serious adverse events reported for this dietary supplement ingredient?

  3. Has the ingredient been brought to FDA’s attention because of concerns other than new ingredient status or human adverse event data described above? (A preliminary evaluation of concerns that have come to FDA’s attention will allow PDA to determine which of these ingredients should move into the priority setting process.)

The following are examples of “other concerns” that may bring an ingredient to FDA’s attention for screening and possibly flagging, but this list is by no means exhaustive:

  • Safety concerns from other groups or organizations that have evaluated substances currently on the market as dietary supplement ingredients. Examples of such safety concerns include compounds previously evaluated as drugs (including over-the-counter drugs), regulatory actions from other governments (e.g., Commission E), usage levels greater than the tolerable upper intake level values set by the Dietary Reference Intakes process (IOM, 1998), and information released by the Office of Dietary Supplements, National Institutes of Health.

  • Strong evidence of serious interactions with prescription drugs.

  • Evidence that the ingredient mimics hormonally active compounds.

Suggested Citation:"5 Framework Steps One and Two: Screening/Flagging and Priority Setting." Institute of Medicine. 2002. Proposed Framework for Evaluating the Safety of Dietary Supplements -- For Comment. Washington, DC: The National Academies Press. doi: 10.17226/10456.
×
  • Communications, such as letters describing potential public health problems associated with a particular ingredient, brought to FDA’s attention via consumers, consumer representative organizations, or scientific organizations. Likewise, this type of concern may come to FDA’s attention through media coverage of a public health problem or new experimental data.

In some cases, reports coming to FDA through any of these mechanisms may indicate that contamination or adulteration of a dietary supplement ingredient may be associated with serious adverse events. Such cases should be directed to the section of FDA handling Good Manufacturing Practice issues for dietary supplement ingredients, rather than moving the ingredient forward to the priority-setting process in this framework.

In keeping with the philosophy that the screening/flagging step should be relatively simple and straightforward, answering the three screening questions does not involve evaluation or weighting of the evidence. A “yes” to any of the questions is sufficient to move the ingredient to the next step. The rationale for the questions is explained below.

New Ingredient Status

New ingredient status is the most straightforward of the screening questions. The reviewer asks if the ingredient is a new ingredient, marketed after the Dietary Supplement Health and Education Act (DSHEA) was enacted (after October 1994). According to DSHEA (see Chapter 1), FDA is to be notified of all ingredients new to the United States 75 days before they are marketed, so this information is readily available. As described in Chapter 4, the rationale for flagging all new ingredients and moving them to the priority-setting process is that (1) they are less likely to be associated with a history of safe use, and (2) there has been little opportunity for serious adverse events to surface, at least in the United States. There is also likely to be less scientific research on these ingredients, although this may not always be the case.

Human Data

Human data is the first key factor considered. In the screening/flagging step, upon learning of serious adverse events from a number of sources such as FDA’s MedWatch reporting system, poison control center databases, published clinical studies and case reports, or secondary reviews of safety information, FDA flags the dietary supplement ingredient involved and moves it forward to the priority-setting process. Adverse events that warrant consideration are those that are “serious,” as defined in Chapter 4. FDA makes an initial judgment about the reported events, but does not attempt to determine if there is causation or to validate the reports at this stage in the framework. To determine which ingredients are moved forward to the priority-setting step, FDA looks tor possible patterns in the type of events reported, for well-documented reports, and for reports that suggest a linkage between the ingredient and the event is at least plausible. Importantly, at this screening step, the reviewer asks whether the data suggest a possible problem rather than focusing on making a definitive judgment. While this strategy has the potential to overestimate ingredients causing possible harm, it is important to be inclusive at this point in the process.

In this preliminary evaluation of human data on serious adverse events, some consideration should be given to the usage patterns of the ingredient in question. Ingredients for which the number of serious adverse events relative to the prevalence of use seems high should be moved

Suggested Citation:"5 Framework Steps One and Two: Screening/Flagging and Priority Setting." Institute of Medicine. 2002. Proposed Framework for Evaluating the Safety of Dietary Supplements -- For Comment. Washington, DC: The National Academies Press. doi: 10.17226/10456.
×

forward. Likewise, the pattern of vulnerable group use should also be considered if data regarding their use come forward. If an ingredient is particularly marketed to, or preferentially used by, a particular subpopulation, and human data suggest particular susceptibility of this subpopulation to the biological action of the ingredient, it should be moved forward to the priority-setting step.

Other Concerns

The third screening category, “other concerns,” recognizes that information regarding safety will come to FDA’s attention for reasons other than new ingredient status or serious adverse events in humans. As discussed earlier, it is important for FDA to take advantage of sources of information that do not require detailed and time-consuming searches. Consumer protection organizations and the media bring forward information that they believe warrants FDA attention. This screening question allows FDA to consider this input and other concerns that come to its attention without judging the quality until it is examined more thoroughly in the priority-setting step.

FDA may also learn about other possible safety problems by perusing the safety concerns of other evaluative groups. For example, information can be gleaned about ingredients previously considered by other governments (e.g., Commission E in Germany), organizations conducting secondary reviews (e.g., the World Health Organization [WHO, 1999]), and texts describing historical patterns of use. The accuracy and reliability of the different secondary sources as arbiters of safety information may vary considerably, but when concerns are raised they warrant consideration.

It is not possible to list all the possible concerns that warrant moving an ingredient forward for consideration in the priority-setting step, but there are a number of concerns that definitely should flag a dietary supplement ingredient when FDA becomes aware of them. These include ingredients that mimic hormonally active compounds and ingredients that may interact with prescription drugs.

In summary, the screening questions given above are designed to use readily available information sources to flag ingredients for consideration in the priority-setting step. Information pertaining to serious adverse events in humans will require FDA to actively and regularly search primary information sources, but new ingredients and other concerns will either come to FDA’s attention directly or will require only minimal information-gathering activities.

STEP TWO: PRIORITY-SETTING PROCESS

The goal of the priority-setting process is to identify those dietary supplement ingredients that require the most immediate attention of FDA for a more in-depth safety evaluation. The priority-setting process differs from the initial screening process in four fundamental ways:

  • additional factors are considered;

  • additional information about each factor is obtained through more active searching;

  • a more evaluative judgment about the strength of the evidence and the level of potential harm is made; and

  • the different factors are weighted differently, based on their comparative importance.

Suggested Citation:"5 Framework Steps One and Two: Screening/Flagging and Priority Setting." Institute of Medicine. 2002. Proposed Framework for Evaluating the Safety of Dietary Supplements -- For Comment. Washington, DC: The National Academies Press. doi: 10.17226/10456.
×

Thus, the most fundamental difference is that as a dietary supplement advances at each step in the process, it requires more information and there must necessarily be more evaluative input.

The factors to be used in the priority-setting process include:

  • human data;

  • animal data;

  • biological activity of structurally related and taxonomically related substances;

  • in vitro data suggesting potential risk of harmful effects; and

  • prevalence of use.

It is assumed that FDA staff with enough experience to make initial judgments regarding the level of evidence of possible risk and seriousness of potential effects will conduct this activity. It is possible, however, that FDA may want to contract with a suitable scientifically based organization to conduct the priority-setting process.

General Description of How to Score Information

In the priority-setting process (Step Two), a sorting matrix can be used to categorize the variety of dissimilar ingredients according to their relative priority for review. For each of the supplement ingredients flagged in the screening process, the information for each of the four key factors (human data, animal data, data about the bioactivity of related substances, and in vitro data) is reviewed and assigned a score. These scores are 0, 1, 2, 3, or NAD when no appropriate data are available to evaluate the information. The four scores (one for each factor) for each ingredient are entered into a matrix (see Table 5–1).

The numerical scores are designed to reflect a judgment of the potential seriousness, and therefore relevance, of the physiological effect and the evidence of possible risk, which is derived from both the quantity and quality of the evidence reviewed. Specific guidelines are outlined for each factor later in the following section, but in general, the following scoring guidelines are employed, as illustrated in Figure 5–1:

  • A score of 3 is assigned for each factor where the data suggest both a potentially serious and very relevant harm and where there is a strong evidence of possible risk, or there is strong evidence suggesting a possible serious drug interaction.

FIGURE 5–1 Scoring system.

Suggested Citation:"5 Framework Steps One and Two: Screening/Flagging and Priority Setting." Institute of Medicine. 2002. Proposed Framework for Evaluating the Safety of Dietary Supplements -- For Comment. Washington, DC: The National Academies Press. doi: 10.17226/10456.
×
  • A score of 0 is assigned when there is strong evidence that there is no serious harm.

  • A score of NAD is assigned when no appropriate data are available.

Scores of 1 and 2 are not explicitly defined but result from a judgment of the potential seriousness of the physiological effect and the evidence of possible risk. This is illustrated in Figure 5–1. For background information on each of the factors, including discussions of what constitutes strong evidence of possible risk, see Chapter 4.

As the information about a number of ingredients is scored, more cells of the priority-setting matrix are filled, as shown in Table 5–1 for fictitious ingredients. The matrix serves as an increasingly useful informational database for classifying information about the different ingredients. The matrix structure also enables the ingredients to be sorted based on their relative priority for further in-depth review. This sorting is described below, following the description of the relative priority of scores for each factor.

TABLE 5–1 Matrix of Scores Used in Establishing Relative Priority Among Dietary Supplements

Ingredient Name

Human Data

Animal Data

Biological Activity of Structurally Related or Taxonomically Related Substances

In Vitro Data

Yellow plant extract

3

1

2

2

Vitamin X

2

NAD

2

NADa

Animal tissue

2

1

1

1

a NAD=no appropriate data.

Which Scores Indicate Higher Priority

A number of different sorting schemes could be developed to produce a ranking of ingredients categorized in order of priority for a full safety evaluation. The committee, recognizing that weights assigned to different factors could easily be arbitrary, deliberately chose not to assign explicit quantitative weights to the factors other than hierarchical ranking. In the proposed scheme, ingredients are ranked and categorized into priority groups by a sorting mechanism that reflects the hierarchical value of the different key factors. When available, concerns raised by human data are weighted more heavily than animal data, and are thus given higher priority. Concerns raised by either human or animal data are given greater weight than concerns raised by bioactivity of related substances or in vitro data, which are weighted equally.

Scores of 3 represent greater concern and therefore rank higher than scores of 2, 1, or 0. NAD scores always rank higher than 0. Because a score of 0 indicates that there is evidence suggesting no serious harm, and a NAD score indicates that there is no evidence, it is clear that NAD scores represent more reason for concern than scores of 0.

Does an NAD score cause more concern than some evidence of harm, as would be indicated by a score of 2 or 1? In the model presented here, a score of NAD is sorted as warranting more concern than a score of 1, but less concern than a score of 2 (e.g., as if it is assigned a value of 1.5). There are two exceptions to this rule: if either the data about bioactivity of related substances or animal data are scored as a 3, then a NAD score for human data is sorted as if it falls between scores of 2 and 3 (e.g., as if it is assigned a value of 2.5). How NAD scores are sorted compared to the numerical scores is summarized here:

Suggested Citation:"5 Framework Steps One and Two: Screening/Flagging and Priority Setting." Institute of Medicine. 2002. Proposed Framework for Evaluating the Safety of Dietary Supplements -- For Comment. Washington, DC: The National Academies Press. doi: 10.17226/10456.
×
  • human data: 3, 2, NAD, 1, 0 if neither animal data nor bioactivity of related substances are scored as 3; or

    3, NAD, 2, 1, 0 if either animal data or bioactivity of related substances are scored as 3

  • animal data: 3, 2, NAD, 1, 0

  • bioactivity of structurally related or taxonomically related substances: 3, 2, NAD, 1, 0

  • in vitro evidence: 3, 2, NAD, 1, 0.

The sorting methodology can be further illustrated with examples. The following list indicates how an ingredient with a score of NAD in animal data would be sorted in comparison to three other ingredients. The ingredient with a score of NAD is sorted as if the NAD had a value of 1.5:

2–2–2–3

2–2–2–3

2-NAD-2–3

2–1–2–3

The following list illustrates how an ingredient with a score of NAD for human data and a 3 for animal data is sorted compared to three other ingredients. Because the animal data is a 3, the NAD value is given more weight and the ingredient with a score of NAD is sorted as if the NAD had a value of 2.5:

2–3–3–3

2-NAD-3–3

2–2–3–3

2–1–3–3

To further illustrate the sorting methodology, all possible scoring combinations are listed in order of priority in Appendix C.

Ranking Ingredients Within the Matrix: Using Scores to Sort Ingredients Into Priority Groups

The proposed mechanism for categorizing dietary supplement ingredients considers the scores for each factor to sort ingredients into priority categories named Priority Group I, Priority Group II, Priority Group III, and so on. These priority groups are illustrated in Table 5–2. The proposed sorting mechanism reflects the hierarchy of the different types of relevant scientific data reviewed (i.e., human data>animal data>data about bioactivity or structurally related and taxonomically related substances=in vitro data). It places highest priority on the ingredients for which there is strong human and animal evidence of possible risk of serious adverse events or serious drug interactions (i.e., scores of 3). Next priority is given to ingredients for which there is strong human data evidence of possible risk of serious adverse events or serious drug interactions, and then to ingredients for which there is strong evidence from animal studies of possible risk of serious adverse events or serious drug interactions, and so on.

This rationale is numerically reflected in the sorting of scores. First, dietary supplement ingredients with scores of 3 in the human data and animal data factors are grouped into Priority

Suggested Citation:"5 Framework Steps One and Two: Screening/Flagging and Priority Setting." Institute of Medicine. 2002. Proposed Framework for Evaluating the Safety of Dietary Supplements -- For Comment. Washington, DC: The National Academies Press. doi: 10.17226/10456.
×

TABLE 5–2 Matrix for Priority Establishment Based on Factor Analysis

Priority Group

Human Data

Animal Data

Bioactivity of Structurally Related or Taxonomically Related Substances

In Vitro Data

Number of Combinations (Total=625)

Characteristics of Priority Group

I

3

3

 

 

25

Two 3s in first two factors

II

3

 

 

 

100

3 in human data

III

 

3

 

 

100

3 in animal data

IV

 

 

3

 

144

One or two 3s in structure/ taxonomy or in vitro factors

 

 

 

 

3

 

V

 

 

 

 

256

No 3s in any key factor

Group I. There are 25 different score combinations that would be included in this priority group. Within Priority Group I, the 25 score combinations are ranked according to their scores in the bioactivity of related substances and in vitro data. These details are apparent in the in-depth scheme provided in Appendix C, which lists all possible combinations of scores and ranks them by priority.

Priority Group II includes ingredients with a score of 3 in the human data factor and less than 3 in the animal data factor. The 100 different score combinations within Priority Group II are sorted based on the scores in animal data, bioactivity of related substances, and in vitro data. Priority Group III includes ingredients that scored a 3 in the animal data section and less than 3 in the human data factor. Priority Group IV includes ingredients that have a score of 3 in the bioactivity of related substances factor or the in vitro factor. Finally, Priority Group V ingredients are ingredients that did not score a 3 in any of the categories. Within Priority Group V, ingredients are ranked as described above; that is, more weight is placed on human data, less on animal data, and even less on data about bioactivity of related substances and in vitro data. If, with use of this system, Priority Group V is found to encompass too many ingredients to be helpful, it could be further divided into subgroups V-1, V-2, and so on, following the pattern for defining the other priority groups.

All theoretically possible scores and how they fit into the different Priority Groups are listed in Appendix C. A number of these combinations of scores, or composite scores, are unlikely to occur because the combinations of data they represent are expected to occur infrequently, if at all. For example, the composite score of 0–0–0–3 (Priority Group IV) would be unlikely to occur very often, given the conflicting nature of the data that must exist to derive this score. That is, a 0–0–0–3 indicates that there are strong human data, animal data, and data about related substances, implying the ingredient causes no serious harm, but there is also strong in vitro evidence of harm in a highly predictive in vitro assay and evidence that harmful ingredients may reach sites of action where they can cause harm. At first glance the current approach might seem to inappropriately allow the in vitro data alone to place this theoretical ingredient in the Priority Group IV, but if this scenario does exist, the higher Priority Group IV classification provides an

Suggested Citation:"5 Framework Steps One and Two: Screening/Flagging and Priority Setting." Institute of Medicine. 2002. Proposed Framework for Evaluating the Safety of Dietary Supplements -- For Comment. Washington, DC: The National Academies Press. doi: 10.17226/10456.
×

opportunity for FDA to consider the disparity presented by the information before considering ingredients in Priority Group V with scores of 2s but no 3s.

Prevalence of Use: A Modifier of Sorting Within Priority Groups

Within each priority group, the prevalence of use is considered as a modifier of the available data. Ingredients with relatively high prevalence of use are shifted to the top of the ranked list within each priority group, so that they will receive attention first before those in the same priority group. “Relative prevalence of use” cannot be precisely defined, but it is suggested that lists of sales data and surveys about the use of particular ingredients be consulted, as described in Chapter 4.

Specifics on Assigning a Score for Each Factor

More information about assigning a score to each of the ingredients is provided below. Some examples for what is considered a higher priority (a score of 3) and what is considered a lower priority (a score of 1) are presented, recognizing that judgment comes into the process of scoring. Although judgment is important in setting scores, the scoring process enables the reviewers to consider the different factors independently and individually for each ingredient, taking advantage of all the data that are available. The system also enables FDA to develop an overall ranking for each ingredient—a ranking that is dynamic and can shift when more information becomes available. Finally, one could imagine that the composite scores of ingredients (such as 3–1–2–2 for the yellow plant extract in Table 5–1) might be used internally to summarize the preliminary information collected in the priority-setting step.

Scoring Human Data

Human data are considered from a different perspective in the priority-setting process, as compared to how human data are considered in the screening/flagging step. As with all the factors, the information is scored based on the potential seriousness of the harm and the evidence of possible risk. In the screening step, however, the effort is focused on examining evidence of only serious adverse events (as defined in Chapter 4), while in the priority-setting step all evidence of adverse events in humans is considered. Another difference is that in the priority-setting step, more time is invested in assessing the evidence that suggests adverse events. It will also be necessary to consider the human data for ingredients that were flagged for reasons other than evidence of serious adverse events in humans.

As shown in Figure 5–1, evidence of possible risk is considered in scoring human data. At this point in the framework, sufficient resources are unlikely to be available to determine causation, but the general guidelines outlined in Chapter 4 for considering causation should guide the judgment of the data quality and quantity.

The seriousness of the adverse effects or potential interactions is also important in scoring. In addition to considering the seriousness of potential harm to the general population and the strength of the data suggesting potential harm to the general population, it is also important to consider how particular subpopulations might be particularly vulnerable to adverse effects from the ingredient (as described in Chapter 4).

The following descriptions should be used as guidelines to score the human data, bearing in mind that some judgment is involved in scoring:

Suggested Citation:"5 Framework Steps One and Two: Screening/Flagging and Priority Setting." Institute of Medicine. 2002. Proposed Framework for Evaluating the Safety of Dietary Supplements -- For Comment. Washington, DC: The National Academies Press. doi: 10.17226/10456.
×
  • A score of 3 should be given if there is strong evidence that there is a possible risk of potentially serious adverse effects or potentially serious drug interactions. For example, well-documented cases of potentially serious adverse events in the medical literature, a strong pattern of similar potentially serious adverse events in MedWatch, action against a dietary supplement ingredient by another country’s regulatory authority, clinical studies indicating potentially serious drug interactions, or multiple -well-documented case reports of potentially serious drug interactions.

  • Scores of 1 and 2 fall between 3 and 0 and thus are notably more subjective. Using Figure 5–1 as a guide, a score of 2 would be appropriate in a situation where the evidence of possible risk is limited but the potential harm is serious. Likewise, a score of 1 might be appropriate in situations when there is some evidence of possible risk, but the potential risk does not appear to be very serious.

  • A score of 0 should be given if there is strong evidence suggesting no potential serious harm and no potential serious drug interactions (in most cases it is not anticipated that information about historical use would provide strong enough evidence to warrant a score of0).

  • A score of NAD should be given if there is no appropriate data available to evaluate.

Scoring Animal Data

The consideration of animal data in the priority-setting step is very different from the consideration of animal data in the screening/flagging step. In the screening/flagging step, animal data are only considered if they come to FDA’s attention. In contrast, FDA needs to actively look for the data in the priority-setting step. In this step, the scientific literature should be systematically searched for evidence of harmful effects in animal studies for all flagged ingredients. Sources for primary scientific literature include IBIDS, MedLine, Toxline, and other scientific literature databases. Other databases that focus particularly on natural ingredients (e.g., NAPRALERT) may also be useful for searching for evidence of adverse effects in animals. Finally, it is important to consult secondary or tertiary reviews that may cite older or foreign data that might otherwise be difficult to uncover using databases that do not capture pre-1960s or non-English literature. Reviews that may be helpful include those listed in Table 4–1.

After gathering information, the evaluation of animal data should be concerned with the same two components considered for human data: the evidence of possible risk, of which the quality and quantity of the data are components, and the seriousness of the harm. While all whole animal experiments may be informative, the nature of the experimental design, the quality of the methodology, and the statistical significance of the results need to be taken into consideration in scoring the evidence. When considering seriousness of harm, animal studies that predict serious harm or death warrant more attention, and thus higher scores, than those that predict mild, self-limiting effects on humans. It is not practical to list all animal data that predict serious effects in humans, but it is important to note that certain types of animal data should be considered serious because associations between these effects and consumption of particular ingredients are likely to be much more evident in animals than in humans. These effects include evidence of cancer, reproductive system effects, or developmental toxicity effects, including teratogenicity or other harm to fetuses.

Ingredients for which data indicate serious effects are scored higher than those with potentially less serious effects. Likewise, ingredients for which the evidence of risk is stronger

Suggested Citation:"5 Framework Steps One and Two: Screening/Flagging and Priority Setting." Institute of Medicine. 2002. Proposed Framework for Evaluating the Safety of Dietary Supplements -- For Comment. Washington, DC: The National Academies Press. doi: 10.17226/10456.
×

are scored higher than those with weaker evidence of possible risk. As discussed in Chapter 4, the strongest animal data result from experiments in which the ingredient is orally administered in a form similar to that used by humans. This characteristic is reflected in the scoring definitions below. As with human data, it is also important to consider how vulnerable subpopulations may be particularly susceptible to adverse effects observed in animals. The scoring guidelines outlined for animal data are analogous to those given for scoring human data:

  • A score of 3 should be given if there is strong evidence that there is a possible risk of potentially serious adverse effects or potentially serious drug interactions. Strong evidence is generated by experiments in which the ingredient is orally administered in a form similar to that used by humans. Serious harm includes effects that would eventually be reported as serious adverse events in humans and those effects that would not be readily detected from general human use or clinical trials.

  • As outlined earlier for human data. Figure 5–1 provides general guidance in judging whether a score of 2 or 1 is appropriate. A score of 2 would be appropriate in a situation where the evidence of potential risk is limited but the potential harm is serious.

  • A score of 0 should be given if there is strong evidence suggesting no potential serious harm and no potential serious drug interactions.

  • A score of NAD should be given if there are no appropriate data available to evaluate.

Scoring Data on the Biological Activity of Structurally Related and Taxonomically Related Substances

A dietary supplement ingredient may be structurally related or taxonomically related to substances with biological activity that cause concern. As with animal data, this type of data will be considered in the screening/flagging step if it comes to FDA’s attention. In the priority-setting step, if the chemical structure of a dietary supplement ingredient or its component chemical compounds is known, FDA should actively look for information to determine if structurally related or taxonomically related substances are of known toxicological concern. This type of information can be gathered from sources such as Medicinal Chemistry Reviews or other journals. Chemicals that act as agonists or antagonists at the same receptors or other biological targets are likely to produce similar effects and should be considered as related chemicals, even if they are not structurally related in the strictest sense.

As described in Chapter 4, the plant genus itself may also provide clues about adverse effects. In the case of botanical dietary supplement ingredients, several key sources of information about plants should be consulted in the priority-setting stage. Texts that list poisonous plants by plant family, genus, and species are helpful in identifying harmful ingredients. Poisonous Plants of the United States and Canada (Kingsbury, 1964) lists egregious examples of harmful plants and also contains information helpful in identifying related plants. Additionally, several databases (e.g., NAPRALERT) provide pharmacological and toxicological information about plant compounds, which is helpful in identifying potentially dangerous substances by the taxa of botanicals.

When scoring information about the biological activity of structurally related or taxonomically related substances, several considerations are important:

Suggested Citation:"5 Framework Steps One and Two: Screening/Flagging and Priority Setting." Institute of Medicine. 2002. Proposed Framework for Evaluating the Safety of Dietary Supplements -- For Comment. Washington, DC: The National Academies Press. doi: 10.17226/10456.
×
  • the ingredient’s similarity in structure or taxonomic relatedness to a known harmful substance;

  • the seriousness of harm caused by ingesting related substances; and

  • the strength of the evidence suggesting that the related substance does cause harm.

Taken together, these components allow the data about the biological activity of related substances to be scored (see below). As in the case of other factors, scoring of this factor should take into account if and how a predicted adverse effect might particularly affect vulnerable subpopulations.

Scoring for the biological activity of related substances is analogous to the scoring schemes outlined for the factors above:

  • A score of 3 should be given if the ingredient has the same or similar structure, or putative biological target (e.g., receptor) as a known compound that causes potentially serious adverse effects or potentially serious drug interactions. Or, the plant-derived dietary supplement ingredient is of a related species and same genus as a plant for which there is strong evidence of possible risk for potentially serious adverse effects or potentially serious drug interactions.

  • As outlined earlier for human and animal data, Figure 5–1 provides general guidance in judging whether a score of 2 or 1 is appropriate. A score of 2 would be appropriate in a situation where the evidence that a related structure or plant causes harm is limited, but the potential harm is serious. Likewise, a score of 1 might be appropriate in situations when there is some evidence of possible risk, but the risk does not appear to be potentially very serious.

  • A score of 0 should be given if there is strong evidence suggesting no structurally related or taxonomically related ingredient that causes serious adverse events or drug interactions.

  • A score of NAD should be given if there is no available evidence about the structure of the substance or its taxonomy, or the biological activity of structurally related or taxonomically related substances.

Scoring In Vitro Data

In vitro data collected on flagged ingredients are scored in the priority-setting step in the same manner as the other factors described above. Information about in vitro effects can be obtained from the same sources as those for animal data (see Table 4–1). These sources include literature databases, other databases, and secondary and tertiary reviews.

In vitro data are considered in terms of the evidence of possible risk and the potential seriousness of harm suggested by the data. Predictive value is also considered for in vitro data; assays that strongly correlate with animal or human outcomes that are very harmful or serious are scored higher.

Scoring for in vitro data is as follows:

  • A score of 3 should be given if there is strong in vitro evidence that there is a possible risk of potentially serious adverse effects or potentially serious drug interactions. Strong in vitro evidence consists of (a) data from a validated assay that strongly predicts in vivo

Suggested Citation:"5 Framework Steps One and Two: Screening/Flagging and Priority Setting." Institute of Medicine. 2002. Proposed Framework for Evaluating the Safety of Dietary Supplements -- For Comment. Washington, DC: The National Academies Press. doi: 10.17226/10456.
×

outcomes that are potentially serious or predictive of potentially serious drug interactions and (b) evidence that the potentially harmful substances are bioavailable. Examples of results from validated assays may include inhibition of mitochondrial oxidative phosphorylation, substantial cytochrome P450 inhibition, and evidence of slowed cardiac repolarization.

  • As outlined for human and animal data, Figure 5–1 provides general guidance in judging whether a score of 2 or 1 is appropriate. A score of 2 might be appropriate when the evidence of possible risk is limited but the potential harm is serious. Similarly, a score of 1 might be appropriate in situations where there is some evidence of possible risk, but the potential harm does not appear to be serious.

  • A score of 0 should be given if there is strong evidence suggesting no potentially serious harm and no potentially serious drug interactions.

  • A score of NAD should be given if there is no appropriate data available related to in vitro assessment.

SUMMARY

In summary, each type of information available (human data, animal data, data about the biological activity of structurally related or taxonomically related substances, and in vitro data) is scored. These scores are derived by considering what the available information reveals about both the evidence of possible risk and the potential seriousness of harm. These scores are used to sort the dietary supplement ingredients into priority categories based on their priority for further safety evaluation. The result of the priority sorting may be modified by information on the prevalence of use.

REFERENCES

IOM (Institute of Medicine). 1998. Dietary Reference Intakes: A Risk Assessment Model for Establishing Upper Intake Levels for Nutrients. Washington, DC:National Academy Press.


Kingsbury JM. 1964. Poisonous Plants of the United States and Canada. Englewood Cliffs, NJ: Prentice-Hall.


WHO (World Health Organization). 1999. WHO Monographs on Selected Medicinal Plants, Volume I. Geneva: WHO.

Suggested Citation:"5 Framework Steps One and Two: Screening/Flagging and Priority Setting." Institute of Medicine. 2002. Proposed Framework for Evaluating the Safety of Dietary Supplements -- For Comment. Washington, DC: The National Academies Press. doi: 10.17226/10456.
×

This page intentionally left blank.

Suggested Citation:"5 Framework Steps One and Two: Screening/Flagging and Priority Setting." Institute of Medicine. 2002. Proposed Framework for Evaluating the Safety of Dietary Supplements -- For Comment. Washington, DC: The National Academies Press. doi: 10.17226/10456.
×
Page 77
Suggested Citation:"5 Framework Steps One and Two: Screening/Flagging and Priority Setting." Institute of Medicine. 2002. Proposed Framework for Evaluating the Safety of Dietary Supplements -- For Comment. Washington, DC: The National Academies Press. doi: 10.17226/10456.
×
Page 78
Suggested Citation:"5 Framework Steps One and Two: Screening/Flagging and Priority Setting." Institute of Medicine. 2002. Proposed Framework for Evaluating the Safety of Dietary Supplements -- For Comment. Washington, DC: The National Academies Press. doi: 10.17226/10456.
×
Page 79
Suggested Citation:"5 Framework Steps One and Two: Screening/Flagging and Priority Setting." Institute of Medicine. 2002. Proposed Framework for Evaluating the Safety of Dietary Supplements -- For Comment. Washington, DC: The National Academies Press. doi: 10.17226/10456.
×
Page 80
Suggested Citation:"5 Framework Steps One and Two: Screening/Flagging and Priority Setting." Institute of Medicine. 2002. Proposed Framework for Evaluating the Safety of Dietary Supplements -- For Comment. Washington, DC: The National Academies Press. doi: 10.17226/10456.
×
Page 81
Suggested Citation:"5 Framework Steps One and Two: Screening/Flagging and Priority Setting." Institute of Medicine. 2002. Proposed Framework for Evaluating the Safety of Dietary Supplements -- For Comment. Washington, DC: The National Academies Press. doi: 10.17226/10456.
×
Page 82
Suggested Citation:"5 Framework Steps One and Two: Screening/Flagging and Priority Setting." Institute of Medicine. 2002. Proposed Framework for Evaluating the Safety of Dietary Supplements -- For Comment. Washington, DC: The National Academies Press. doi: 10.17226/10456.
×
Page 83
Suggested Citation:"5 Framework Steps One and Two: Screening/Flagging and Priority Setting." Institute of Medicine. 2002. Proposed Framework for Evaluating the Safety of Dietary Supplements -- For Comment. Washington, DC: The National Academies Press. doi: 10.17226/10456.
×
Page 84
Suggested Citation:"5 Framework Steps One and Two: Screening/Flagging and Priority Setting." Institute of Medicine. 2002. Proposed Framework for Evaluating the Safety of Dietary Supplements -- For Comment. Washington, DC: The National Academies Press. doi: 10.17226/10456.
×
Page 85
Suggested Citation:"5 Framework Steps One and Two: Screening/Flagging and Priority Setting." Institute of Medicine. 2002. Proposed Framework for Evaluating the Safety of Dietary Supplements -- For Comment. Washington, DC: The National Academies Press. doi: 10.17226/10456.
×
Page 86
Suggested Citation:"5 Framework Steps One and Two: Screening/Flagging and Priority Setting." Institute of Medicine. 2002. Proposed Framework for Evaluating the Safety of Dietary Supplements -- For Comment. Washington, DC: The National Academies Press. doi: 10.17226/10456.
×
Page 87
Suggested Citation:"5 Framework Steps One and Two: Screening/Flagging and Priority Setting." Institute of Medicine. 2002. Proposed Framework for Evaluating the Safety of Dietary Supplements -- For Comment. Washington, DC: The National Academies Press. doi: 10.17226/10456.
×
Page 88
Suggested Citation:"5 Framework Steps One and Two: Screening/Flagging and Priority Setting." Institute of Medicine. 2002. Proposed Framework for Evaluating the Safety of Dietary Supplements -- For Comment. Washington, DC: The National Academies Press. doi: 10.17226/10456.
×
Page 89
Suggested Citation:"5 Framework Steps One and Two: Screening/Flagging and Priority Setting." Institute of Medicine. 2002. Proposed Framework for Evaluating the Safety of Dietary Supplements -- For Comment. Washington, DC: The National Academies Press. doi: 10.17226/10456.
×
Page 90
Next: 6 Framework Step 3: Critical Safety Evaluation of Dietary Supplement Ingredients »
Proposed Framework for Evaluating the Safety of Dietary Supplements -- For Comment Get This Book
×
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF
  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!