The Risk Characterization chapter of the Food Safety and Inspection Service (FSIS) draft risk assessment integrates and applies the modeling work done in the three modules of the exposure assessment (Production, Slaughter, and Preparation) and the dose-response assessment presented in “Hazard Characterization.” The analyses characterize the risk associated with E. coli O157:H7 exposure of individuals, the community, and the population for different exposure periods. The effect of seasonal variations in exposure, and therefore risk, is examined with differences in risk to young children. The results of some sample sensitivity analyses are also presented to demonstrate potential policy applications of the model.
In the discussion below, the committee offers observations, suggestions, and recommendations related to the major sections of the draft chapter. Many of the important points it wishes to make have already been addressed in the previous chapters or are more appropriately discussed in the Modeling Approach and Implementation chapter that follows, so this chapter is brief.
DEFINITION OF KEY TERMS
The chapter’s definitions are clearly spelled out and helpful in interpreting the material. However, as mentioned elsewhere in this review, some of the draft report’s definitions are not the standard ones used in the scientific literature or other quantitative risk assessments. Others include terminology that may confuse readers. Because microbial risk assessment is a relatively new field, it is desirable to promote consistency and clarity
in expression. We present below a number of specific suggestions regarding defined and undefined terms.
“Typical” individual risk is the term applied to the risk posed for someone who purchases ground beef that is contaminated with E. coli O157:H7 organisms at the median concentration, who stores and cooks that product in a way that is consistent with the median growth and cooking distributions, and who consumes a single serving. Because the definition does not describe a “typical” exposure, the committee suggests that that word be changed, perhaps to hypothetical or illustrative.
Duration of exposure is defined in the draft as “the length of time (e.g., per serving, per annum, or lifetime) for which a risk estimate was assessed.” The definition is confusing. Relating a serving to a duration of exposure is awkward. The committee suggests that this term, which is used only twice in text, be dropped.
The term dose is defined as the number of E. coli O157:H7 in a single serving of ground beef. That is consistent with the draft Food and Drug Administration (FDA)-USDA Listeria risk assessment (2001). However, in chemical risk assessments, dose is usually defined as grams per kilogram of body weight of a subject. The committee suggests that the report’s definition of dose draw attention to the different meaning of this term in microbial risk assessments.
The draft cites sensitivity analysis as “the quantitative process of identifying factors (model inputs) in the farm-to-table continuum that contribute to the occurrence of E. coli O157:H7 in ground beef or the subsequent risk of illness” (p. 130). That definition is tangentially related to the more traditional understanding of the term: the quantification of the effects of changes in model inputs on model outputs. However, it fails to capture the sense in which most risk analysts apply it: the systematic investigation of whether and to what extent changes in model inputs across a plausible range of values affect model outputs. The committee suggests that the final risk assessment adopt the more common definition of sensitivity analysis. The related terms correlation analysis and dependency analysis should also be more clearly and completely documented on the basis of established definitions.
The “risk of illness” is addressed throughout the chapter, but no formal definition of the conditions covered by illness is provided. As the draft notes (pp. 22–23), ingestion of E. coli O157:H7 can result in a wide array of outcomes, including asymptomatic infection, abdominal cramps, nonbloody diarrhea, hemorrhagic colitis, hemolytic uremic syndrome (HUS), and thrombotic thrombocytopenic purpura (TTP). The chapter needs a clear definition, and, depending on the conditions covered, risk of infection may be a more appropriate term. Furthermore, “annual risk” and “risk per serving” are both referred to as risk of illness in the chapter. The
committee suggests that the text clearly state what is being reported in each instance and, if necessary, use different terms for these metrics to minimize confusion.
In later sections of the chapter, the risk of “severe illness” is evaluated. The term is not defined, although the text (p. 135) refers to hemorrhagic colitis (called “bloody diarrhea”), HUS, and death. The draft’s Hazard Characterization chapter lists hospitalization, HUS, TTP, and death in a discussion of “severe clinical outcomes” (p. 121). The committee suggests that the terminology be standardized in the risk assessment and that appropriate definitions be added to this and other sections.
In general, the committee suggests that—where possible—the final risk assessment adopt the definitions established by one of the major organizations that have already published glossaries and that alternative expressions be used in other circumstances. The World Health Organization (WHO, 1995), Codex Alimentarius Commission (1999), and International Life Sciences Institute (ILSI, 2000) have developed sets of definitions that are potential starting points for aligning terminology. The draft FDA-USDA Listeria risk assessment (2001) also contains a well-written extended glossary. The committee recognizes that not all terms can be aligned, but it believes that in many cases it would be less confusing if terms were used in a standard way.
The committee also suggests that all the terms in this section and the rest of the draft be provided with general definitions before they are applied specifically to the O157:H7 risk assessment, as is done for the draft’s definition of risk.
RISK OF ILLNESS FROM E. COLI O157:H7
The term risk of illness is used in this section to describe the probability of illness from consuming ground beef with different doses of O157:H7. That risk is estimated as the product of the probability of exposure and the probability of illness associated with a given dose. The committee is concerned that the results of this analysis, as presented in the tables and figures in the draft, may be misunderstood by some lay readers. Because the probability of exposure to very large numbers of organisms is relatively small, the risk of illness from such exposure is concomitantly small. The committee is concerned that some may interpret the presentation to mean that the model predicts that one is less likely to become ill from massive doses of E. coli O157:H7 than from much smaller exposures. The committee recommends that these analyses, if retained in the final risk assessment, be expressed as a cumulative probability of illness, given the projected frequency and levels of contamination of ground beef.
The committee also observes that it would be helpful if a discussion were added to this section that clearly indicates how the modeling of infection differs from the modeling of illness.
Risk of Illness for an Individual
In this subsection, per-serving, annual, and lifetime risks are provided for a “typical” person, on the basis of the point estimates of the model’s outputs and the median of the derived dose-response relationship. As noted above, this is in reality a hypothetical person with an exposure scenario that could be considered illustrative.
Table 5-1 in the FSIS draft, which presents the information used in this section, is particularly useful in identifying the location of data used in the assessment. The addition of a side-by-side table that summarizes the individual hypothetical risks (per serving, annual, and lifetime) and the associated assumptions in the “typical” exposure scenario would provide additional clarity and bring home the point that a hypothetical individual risk is being calculated.
More generally, the committee believes that the draft’s focus on a “typical” person is—from public health and policy perspectives—misguided. It is desirable to avoid all E. coli O157:H7 infections, but attention needs to be centered on the more severe outcomes of infection. That means examining the determinants of high exposure in the general population and any exposure in the subpopulations thought to be most vulnerable to complications: children and the elderly.1 Although some attempt is made to address the special vulnerabilities of children later in the chapter, the analysis is cursory and does not address whether differences in consumption patterns might affect risk. There is no separate consideration of the elderly in the draft.
The committee recommends that the Risk Characterization chapter be refocused to concentrate on the analysis of severe illnesses associated with E. coli O157:H7 infection, the subpopulations known or thought to be most vulnerable to them, and the interventions that might have the greatest effect in preventing them.
Risk of Illness for a Community—Simulated Outbreak
This subsection is intended to provide an example of how the presence of E. coli O157:H7 in ground beef might contribute to an outbreak. Like the hypothetical individual risk, this risk characterization is based on a prescribed scenario of exposure—an example of what might happen if contamination occurred and were spread at a community level. The risk metric in this section is risk per serving, and the analysis estimates the number of illnesses as a function of assumptions regarding the number of servings consumed by each person, the total number of contaminated servings, and the reduction in contamination from cooking.
This brief “what-if” analysis illustrates how an outbreak might occur and provides interesting outputs. However, the committee believes that it is appropriate to address more time and attention to this scenario in the final risk assessment because of the great public interest in community outbreaks. At a minimum, the committee suggests that a summary table of assumptions in the community-exposure scenario and calculations of risk be added to improve the clarity of the presentation. This addition will permit the reader to relate estimates of risk per serving to the number of people who could become ill.
The committee also suggests that the assessment address the conditions in the exposure continuum that are likely to lead to outbreaks. Additional examples that demonstrate the effects of storage, handling, and cooking methods on the potential for an outbreak (for example, an analysis of the effect of improper cooking on the risk of a community outbreak) would be helpful, as would a discussion of how likely the various scenarios are. And the committee recommends that the risk assessment focus its evaluation of community risk on more severe outcomes and effects on vulnerable subpopulations.
Risk of Illness for the US Population
This subsection summarizes modeling results that are likely to be used often for public health decision-making, and it evaluates the overall risk of illness, given the population distribution of exposure and the dose-response function. The risk characterization for the US population and, as noted above, selected vulnerable subpopulations should thus be the central output of the risk assessment.
The committee notes that the authors indicate that they will address other populations’ risk variability as more data become available. It supports that plan and recommends that additional analyses be presented in the final report based on available data to test the effects of different as
sumptions and model parameters. The analyses could include estimates of risk associated with different storage and cooking practices.
The statement that the population risk estimate from this report is “comparable” with risks estimated by other investigators (p. 135) is overstated. The estimated 9.6 × 10−7 annual risk of illness in the draft risk assessment is 2 orders of magnitude lower than that reported by Cassin et al. in 1998 (5.1 × 10−5 for adults), and the upper bound of this risk assessment is 3 orders of magnitude lower than the upper bound of Marks and colleagues’ in 1998 (10−7 versus 10−4). Chapter 7 of the present review provides additional discussion of this point.
The explanation of differences between the number of cases per year from the draft model and that based on surveillance data is incomplete. Attributes of data used in the risk assessment might contribute to the differences; for example, the estimate of ground-beef servings consumed per year could be an underestimate.
Given the anchoring approach used in the draft, the committee questions the informativeness of the estimation of the risk of severe illness presented at the end of the subsection because the model is already adjusted to conform to obs erved levels of illness. Furthermore, as noted elsewhere, the risk of severe illness is highly age-dependent. Either age dependence should be addressed in this subsection, or the analysis should be dropped. Again, these points are addressed in greater detail in Chapter 7 of this review.
For clarity, the committee suggests that the X axes of Figures 5-1 and 5-2—labeled “Dose”—be relabeled “Number of E. coli O157:H7 per Serving” to relate them more directly to the data and analysis summarized in Table 5-2 in the draft risk assessment.
POPULATION RISK BY SEASON, AGE, AND LOCATION
This is an important section, and the authors have clearly noted that. It is less clear whether the appropriate metric has been applied.
Intuition suggests a pronounced seasonal effect in preparation, handling, and consumption of ground beef associated with fair-weather activities, such as outdoor grilling, picnicking, and camping. The committee suggests that if such seasonal factors can be explicitly addressed, they should be. If data are insufficient to allow their effects to be analyzed, seasonal effects beyond the changes in E. coli O157:H7 prevalence now addressed need to be recognized, and the resulting limitations acknowledged.
As noted above, the analysis of the risks to children is cursory at best, and the special risks to the elderly are not evaluated. If this analysis is to
be used to set policy that affects public health, it needs to address the special vulnerabilities of these sections of the population.
The committee agrees that there are likely to be great differences in food-preparation practices between hotels, restaurants, and institutions (HRI) and home environments, and it points out that practices in the fast-food mass-retail establishments—where a considerable proportion of ground-beef meals are prepared and consumed—may differ from those of other HRI businesses. The draft risk assessment correctly notes this as a research need. HRI businesses and trade associations may well already have the information needed to allow analysis of the issue, and the committee recommends that their input be solicited.
Sensitivity analysis is an important aspect of a risk characterization. The committee recommends that a discussion of the types of analyses that are available within the Risk Characterization Module and a rationale for the selection of the most appropriate analysis for a given situation be included. In the case of correlation analysis, the degree of uncertainty in the model inputs should be clearly stated, in addition to the presentation of the correlation outputs.
The dependence analysis presented in this section appears to arbitrarily introduce a 50% change in the model input’s assumptions (that is, feedlot prevalence, steam pasteurization, and the like). If there is any rationale for the new assumptions, it should be stated. Otherwise, it should be made clear that they are being presented for illustrative purposes only.
On p. 149 of the draft, preparation scenarios 2 and 3 are based on cooking to 5-log reduction. The reader would benefit from an explanation of the motivation for picking this level of reduction, given that the draft report indicates (in a footnote on p. 131) that a 5.5-log reduction is the median of the cooking distribution.
Finally, the committee recommends that a sensitivity analysis for the overall risk model be included, in addition to the sensitivity analysis of the individual modules in the exposure continuum.
The committee does not believe that it is appropriate to offer specific comments regarding the analyses presented in the “Conclusions” section of the Risk Characterization chapter. It believes that it is premature for the draft risk assessment to draw the inferences contained in this section and that its inclusion at this stage of the assessment process conveys an un
warranted confidence in the validity of the model’s output. It is more appropriate to include such material after the model has been refined. Alternatively, the level of uncertainty in these conclusions needs to be much more forcefully stated.
The committee believes that recommendations regarding the potential applications of the model’s various outputs would be a useful addition to the final risk assessment. The authors should make clear what they believe the model can and cannot be used for. As discussed in Chapter 7 of this review, they should devote particular attention to how anchoring can affect whether particular applications or inferences are appropriate.
The committee notes, as it has elsewhere, that the impact of cross contamination is not assessed in this draft chapter and that characterizations of uncertainty and variability in the results are either weak or absent.
The committee commends the authors for their considerable work in the development of a comprehensive chapter on risk characterization. Although there is much additional work to do to complete the risk characterization, the critical topics have been identified, and initial work presented for each.
Cassin MH, Lammerding AM, Todd EC, Ross W, McColl RS. 1998. Quantitative risk assessment of Escherichia coli O157:H7 in ground beef hamburgers. International Journal of Food Microbiology 41:21–44.
Codex Alimentarius Commission. 1999. Principles and Guidelines for the Conduct of Microbiological Risk Assessment. CAC/GL-30. Food and Agricultural Organization of the United Nations.
FDA/USDA (Food and Drug Administration/US Department of Agriculture). 2001. Draft Assessment of the Relative Risk to Public Health from Foodborne Listeria monocytogenes Among Selected Categories of Ready-to-Eat Foods. FDA/Center for Food Safety and Applied Nutrition, USDA/Food Safety and Inspection Service, Centers for Disease Control and Prevention, January 2001. http://www.foodsafety.gov/~dms/lmrisk.html.
ILSI (International Life Sciences Institute). 2000. Revised Framework for Microbial Risk Assessment — An ILSI Risk Science Institute Workshop report. Washington, DC: International Life Sciences Institute.
Marks HM, Coleman ME, Lin CTJ, Roberts T. 1998. Topics in risk assessment: Dynamic flow tree process. Risk Analysis 18:309–328.
WHO (World Health Organization). 1995. Application of Risk Analysis to Food Standards Issues-The report of the Joint FAO/WHO Expert Consultation, 13–17 March 1995 (WHO/FNU/FOS/95.3).