While the US food supply is widely considered to be among the safest in the world, foodborne diseases are still thought to cause some 76 million illnesses, 325,000 hospitalizations, and 5,000 deaths in the country each year (Mead et al., 1999). A 2001 analysis by the US Department of Agriculture (USDA) estimated the annual medical costs, productivity losses, and value of premature deaths due to exposure to five common foodborne pathogens1 at $6.9 billion (ERS, 2001).
In the face of such a major public health problem, USDA’s Food Safety and Inspection Service (FSIS) is formulating risk assessments to identify important foodborne hazards; evaluate potential strategies to prevent, reduce, or eliminate those hazards; assess the effects of different mitigation strategies; and identify research needs. These risk assessments, in brief, empirically characterize the determinants of the presence or level of microbial contamination in vulnerable foodstuffs at various points leading up to consumption.
One of the initial efforts in the undertaking is a risk assessment of the public health impact of E. coli O157:H7 in ground beef (USDA-FSIS, 2001).2 A draft report describing this work was released for comment on
The five pathogens are Campylobacter, nontyphoidal Salmonella, Escherichia coli O157:H7, E. coli non-O157:H7 STEC, and Listeria monocytogenes.
The complete text of the draft risk assessment may be found on the FSIS web site (www.fsis.usda.gov). At the time this report was completed, the URL for the draft was http://www.fsis.usda.gov/OPPDE/rdad/FRPubs/00-023N/00-023NReport.pdf.
September 7, 2001. In addition to soliciting public input, FSIS asked the Institute of Medicine (IOM) to convene a committee of experts to review the draft and offer recommendations and suggestions for consideration as the agency finalizes the document. This report presents the results of that review.
E. coli O157:H7
E. coli serotype O157:H7 is a rare variety of E. coli, a normal inhabitant of the intestines of all animals, including humans (FDA, 2002). The pathogen produces large quantities of one or more related potent toxins, called Shiga toxins, that cause severe damage to the lining of the intestine and to other target organs, such as the kidneys. E. coli O157:H7 was first recognized as a cause of illness in 1982 during an outbreak of severe bloody diarrhea that was later traced to contaminated hamburgers (CDC, 1982). It has since been implicated in a number of outbreaks of intestinal distress. The most severe outcome in the general population is typically hemorrhagic colitis, a prominent symptom of which is bloody diarrhea. Life-threatening complications, however, sometimes ensue. Some victims, particularly the very young, may develop hemolytic uremic syndrome (HUS). HUS, which is characterized by renal failure and hemolytic anemia, occurs in up to 15% of hemorrhagic colitis victims and can lead to permanent loss of kidney function. In the elderly, the combination of HUS with fever and neurologic dysfunction is characteristic of thrombotic thrombocytopenic purpura (TTP). Left untreated, TTP has a mortality of about 95%; however, early diagnosis and treatment yield a survival rate of 80–90% (Abumuhor and Kearns, 2002). Overall, the Centers for Disease Control and Prevention estimates that E. coli O157:H7 is responsible for some 73,500 cases of infection, 2,150 hospitalizations, and 61 deaths in the United States each year (Mead et al., 1999).
Eating meat, particularly ground beef, that has not been cooked sufficiently to kill E. coli O157:H7 is thought to be the primary cause of infection. Cross contamination—which occurs when harmful bacteria in raw beef or its juices are spread to other foods through contact with cutting boards, utensils, and the like—also accounts for illnesses. Among other known sources of infection are consumption of contaminated sprouts, lettuce, salami, and unpasteurized milk and fruit juice; swimming in or drinking contaminated water; and contact with the stools of infected animals or people. FSIS has classified E. coli O157:H7 as an adulterant in raw ground beef, thus banning the sale of any ground beef contaminated with it.3
The Food Safety and Inspection Service
FSIS is the public health agency in the USDA. It is responsible for ensuring that the nation’s commercial supply of meat, poultry, and egg products is safe, wholesome, and correctly labeled and packaged.
On July 25, 1996, FSIS issued its landmark rule, Pathogen Reduction; Hazard Analysis and Critical Control Point (HACCP) Systems (USDA-FSIS, 1996). The rule addresses foodborne illness associated with meat and poultry products by focusing more attention on the prevention and reduction of microbial pathogens on raw products that can cause illness. FSIS is formulating microbial risk assessments to help to inform efficient risk-management policy decisions and identify future research needs.
The application of risk assessment techniques to microbial pathogens poses challenges that differ from those of chemical, environmental, or toxicologic risk assessments. Notably, bacterial populations are living things that can grow, spread, or die, depending on the characteristics of their environment. FSIS is working with other agencies and institutions to develop appropriate quantitative risk assessment methods and to support studies to fill data gaps and enhance the precision and reduce the uncertainty in risk characterizations of microbial pathogens.
Charge to the Committee
USDA asked the IOM committee to provide comments on the draft E. coli O157:H7 risk assessment for consideration as they finalized the document. The charge directed the committee to include evaluations of the overarching logical structure of the model, the validity and appropriateness of the input data used, the reasonableness of the assumptions made, the reasonableness of the anchoring approach (that is, the adjustment of simulation outputs of the model to be more compatible with observed data), and the model’s mathematics and equations. It also asked the committee to consider whether risks had been appropriately characterized and whether key sources of variability and uncertainty, critical assumptions, and important data gaps had been identified and characterized.
ORGANIZATION AND FRAMEWORK
The format of this report follows, in part, the organization of the FSIS draft risk assessment. The major topics addressed are
a summary of the content of the draft risk assessment (Chapter 1);
reviews of the three modules of the exposure assessment—Production, Slaughter, and Preparation—that characterize the nature and deter
a review of the Hazard Characterization, the section of the assessment that relates exposure to the pathogen to various health outcomes (Chapter 5); and
an examination of the Risk Characterization, the part of the assessment that uses the model to generate risk estimates and provides a means for examining which steps in the process are most influential in determining the model’s outputs (Chapter 6).
The committee’s comments on the overall approach taken for constructing and implementing the model appear in Chapter 7. The FSIS draft also includes a chapter (“Hazard Identification”) that summarizes the microbiologic and epidemiologic evidence used in the analysis. The committee’s comments regarding this content are in the reviews of later chapters where it is referred to.
An appendix to the report contains an independent review by Edmund Crouch—presented to the committee at a public meeting in February 2002—that provides additional comments on the model’s variables and the software implementation of the analysis.
COMMENTS REGARDING THE DRAFT RISK ASSESSMENT
The committee conducted a thorough, science-based examination of the content of the draft—mindful of the fact that it was a work-inprogress—and generated a number of comments focused on the subjects delineated in the charge. Because the report contains specific observations on numerous individual components of a complex assessment, it is not possible to cogently and concisely list all of the comments here. The sections below are a synopsis of the committee’s major findings. Chapters 2–7 detail the reasoning underlying these conclusions and present the committee’s complete findings.
At the outset, it should be said that the effort underlying the FSIS draft risk assessment is impressive. The authors of the report have undertaken an extraordinary task of collection, analysis, and integration of information that far exceeds the scope and breadth of prior assessments of E. coli O157:H7. They faced a number of substantial methodologic hurdles peculiar to microbial risk assessment whose solutions have not been described in textbooks or published elsewhere. It is thus appropriate that they interrupt their effort to allow for peer review and to reassess their solutions to these very challenging issues.
In that regard, the committee commends the draft’s authors on the mag
nitude of their effort and the principles behind it. It wishes to make clear that many criticisms of this model would probably apply to most, if not all, microbial risk assessment efforts previously and currently undertaken.
The Production Module of the draft risk assessment models E. coli O157:H7 in cows, bulls, steers, and heifers from the farm through transit to the slaughter plant. The committee’s principle comment on this section pertains to the use of fecal prevalence as the sole output of the module. It notes that the concentration at which an animal sheds pathogen is also important—one animal shedding 105 colony forming units (CFU) of E. coli O157:H7/g of feces would yield the same number of cells as 1,000 animals shedding 102 CFU/g, yet both are considered to contribute equally in a model in which only prevalence is factored in. The exclusive use of fecal prevalence also requires the assumption that most carcass contamination in later stages of processing occurs directly from the gastrointestinal tract of slaughtered animals, although circumstantial evidence suggests that the hide is also an important source. The decision to use fecal prevalence appears to have been necessitated by the paucity of information on other indicators. The committee thus recommends that the risk assessment acknowledge forthrightly that fecal prevalence is being used as a proxy to characterize several interrelated variables that are poorly understood and on which data are sparse, and that some carcass contamination is derived from contaminated hides. It also recommends that an impact assessment of animals shedding E. coli O157:H7 at high and low concentrations be conducted.
The committee raises some questions regarding whether data from disparate studies have been properly combined and whether prevalence has been correctly calculated in all cases, notably in the determination of within-herd prevalence. These instances, detailed in Chapter 2, can be addressed by more clearly defining the intent of the variables estimated in the module and changing the inputs and variable equations as appropriate.
The Slaughter Module estimates the prevalence of E. coli O157:H7 at each step of the slaughter-plant process, starting with live cattle entering the plant and ending with packaged meat product that is ready for shipment. The committee’s primary comments regarding this module also have to do with the lack of available information. The draft risk assessment correctly notes that published data on the prevalence and cell density of E. coli O157:H7 during the slaughter and fabrication processes are
scarce. In addition, data on the surface area contaminated and the extent of cross contamination are lacking.
The draft recognizes the need for more research to obtain additional information on the contribution of the hide to carcass contamination; the prevalence, extent, and density of E. coli O157:H7 contamination on carcasses after dehiding; the contribution of cross contamination to product contamination; the effect of carcass decontamination and chilling on increases or decreases in E. coli O157:H7 organisms; and the influence of fabrication activities on redistribution of contamination in meat cuts and trimmings.
The committee observes that the lack of publicly available data regarding crucial steps in the slaughter process, the variability of the operations modeled in the module, and the potential unpredictability of the effects of some activities on contamination during slaughter and carcass fabrication complicate modeling and limit the module’s forecasting capacity. The draft module relies heavily on the results of only one study (Elder et al., 2000), makes major assumptions regarding some variables, and readjusts some inputs to fit the expected outcomes. Although such practices are often necessary in model development, the committee recommends that these difficulties and deficiencies be more strongly emphasized in discussions of the outcomes calculated by the model and that the need for more data for model improvement be highlighted throughout the final assessment. Furthermore, the committee recommends that the final assessment stress the potential influence of slaughter plant activities on cross contamination, on the level of contamination, and on the extent of carcass or trim surface area contaminated. It should make clear that the effects of these activities, although important, might be difficult to characterize empirically. The committee recommends that the authors add a discussion of the appropriate and inappropriate applications of the Slaughter Module in its present state of development—in particular, whether the module is ready to be used to draw conclusions about which factors modeled in it are most important in influencing the occurrence and extent of E. coli O157:H7 contamination in ground beef and the possible effects of interventions.
The Preparation Module estimates the incidence and scope of E. coli O157:H7 contamination in a serving of cooked ground beef by modeling the conditions under which it is ground, transported, stored, handled, and cooked. A central issue for the committee in its review of this module is the lack of factoring of the contributing influence of cross contamination on human illness. Cross contamination during preparation results
when E. coli O157:H7 is transmitted from contaminated ground beef to such vehicles as other foods, food preparation and processing surfaces, and food handlers. Because of the highly infectious nature of the pathogen, vehicles cross-contaminated through direct or indirect exposure are likely to be important sources of human illness.
The draft clearly notes that exposures from cross contamination are outside the scope of the assessment, and the committee understands and respects the decision of the modelers to establish reasonable bounds on the reach of their work—a necessary part of any risk assessment. The committee observes, however, that cross contamination during preparation is an established, important risk factor; that the lack of data concerning its impact is no more severe than the lack of data for some other parts of the draft model; and that further attention to cross contamination will help to lay the groundwork for an analysis and better identify the data gaps that need to be filled by future research. The value of the risk assessment in informing public health policy and supporting regulatory interventions will be increased if it is able to factor in the effect of cross contamination on E. coli O157:H7 infections and perhaps address the influence of interventions. Just as important, the committee is concerned that the draft risk assessment may foster the incorrect impression that proper cooking of ground beef will prevent all E. coli O157:H7 infections associated with ground beef. If the model is used to simulate the impact of various interventions on human health outcomes, the omission of cross contamination could unintentionally omit from consideration interventions that could have a material effect on infection.
The committee thus suggests that consideration be given to factoring cross contamination in the model. If that is not possible, it recommends that the final risk assessment more clearly highlight the role of cross contamination in E. coli O157:H7 infection and emphasize the limitations in the model engendered by a decision to not factor it.
The committee notes that although practices for storage, handling, and cooking of ground beef in the home, at fast-food restaurants, and in other retail facilities (called HRI—hotels, restaurants, and institutions—in the draft) vary considerably, the model does not differentiate among them. It suggests that the revised model account for these differences and model each location separately.
Data availability is an issue in this module, as elsewhere. In the draft’s calculations of the annual number of raw ground-beef servings, some estimates are unsound because a very small number of observations are linearly extrapolated to represent the entire population. The committee recommends that the authors acknowledge that they do not have adequate information on the consumption of raw ground beef and
suggests that alternatives that reflect the uncertainty in the numbers be pursued while better data are sought.
Hazard Characterization Modules
The draft’s Hazard Characterization Module describes a method to estimate the number of symptomatic infections resulting from the consumption of cooked ground beef contaminated with E. coli O157:H7. As it is not possible to test this relationship directly, upper and lower boundaries for it are established by using data from similar pathogens. Overall, the committee believes that the draft chapter’s authors did an elegant job in generating a dose-response function. The available literature strongly supports the relevance of their decision to use data on Shigella dysenteriae 1 for the upper limit of the bracket. The data further argue, though, that the E. coli O157:H7 dose-response function is likely to be very close to that of Shigella and that arguably it will be more appropriate to use dose-response data from experimental challenges with Shigella administered with buffer. The use of enteropathogenic E. coli (EPEC) dose-response data as the lower limit is reasonable but somewhat more problematic because studies of its effects in humans do not generally reflect real-world exposures. The committee believes that the EPEC dose-response function is a conservative choice and suggests that if the bounding approach is used in the final risk assessment, consideration should be given to alternatives that might reflect E. coli O157:H7 pathogenicity better.
More generally, although the draft’s discussions of the baseline number of E. coli O157:H7 infections and adjustments for underdiagnosis and underreporting are scientifically sound, they do not account for non-O157:H7 serotypes of enterohemorrhagic E. coli (EHEC). The virulence properties that allow O157:H7 to cause hemorrhagic colitis, HUS, and TTP are common to a broader category of pathogens. The draft indicates that because E. coli O157:H7 is the most important serotype in the United States from a public health standpoint and because there is a paucity of epidemiologic data on non-O157 serotypes, the risk assessment is limited to E. coli O157:H7. The committee acknowledges that decision but points out its implication: whatever risk to the public health of the United States is attributed to O157:H7 as a ground-beef contaminant by the risk assessment will be an underestimate of the overall risk because other EHEC serotypes also cause disease. Because non-O157:H7 serotypes contribute to the EHEC disease burden—particularly as a cause of HUS—the committee suggests that the decision to exclude these serotypes be revisited. If the final risk assessment is limited to O157:H7, the committee recommends that this decision and its implications for the model be explicitly discussed in the “Hazard Characterization” chapter.
The draft’s risk characterization integrates and applies the modeling work done in the three exposure assessment modules (Production, Slaughter, and Preparation) and the dose-response assessment presented in the “Hazard Characterization” chapter to generate analyses of the risk associated with E. coli O157:H7 exposure for individuals, the community, and the US population. It is premature to draw conclusions regarding the results of these analyses, because the model is in draft form, but the committee does have some suggestions regarding the form and focus of this work when the model is final.
In particular, the committee believes that the draft’s focus on a “typical” individual is—from a public health and policy perspective—misguided. It is desirable to avoid all E. coli O157:H7 infections, but attention needs to be centered on the more severe outcomes of infection. The committee therefore recommends that the risk characterization be refocused to concentrate on the analysis of severe illnesses associated with E. coli O157:H7 infection, the subpopulations known or thought to be most vulnerable to them, and the interventions that might have the greatest impact on preventing them.
The committee questions the informativeness of the risk estimates generated with the model as it is now structured because it is already adjusted to conform to observed levels of illness. It suggests that a discussion of potential applications of the model’s various outputs would be a useful addition to the final assessment. The authors should make clear what they believe the model can and cannot be used for and should address how the structure of the model affects whether particular applications or inferences are appropriate.
Modeling Approach and Implementation
Two major issues were identified in the committee’s overall review of how the draft model was constructed and implemented.
The first is that the risk assessment is not structured in the form classically used in such work. A risk assessment is typically an effort directed at providing an estimate of risk through the collection of evidence and the application of mathematical tools, and the risk estimate is usually a dependent output of the model. The draft alters that arrangement by deriving the exposure assessment and the population risk estimates from separate sources and then inferring an E. coli O157:H7 dose-response relationship that is mathematically compatible with the calculated exposure assessment and the distribution of population risk estimates.
The committee recommends that the final report more clearly communicate the nature of, the rationale for, and the impact of the deviation
from the standard approach and that the authors consider relabeling the product as a system risk model to avoid implying that the model generates an estimate of risk independent of those derived from epidemiologic estimates. It also recommends that the authors re-evaluate the approach taken to infer the dose-response relationship in light of the possibility that this parameter is not the greatest source of uncertainty in the model and in consideration of other comments expressed in this review.
At several points in the development of the draft model, algorithms are invoked to adjust the simulation outputs to be more compatible with observed data. That technique, called anchoring in the report, is well-founded in health risk assessment and the related field of environmental modeling. But its use poses three problems that the committee believes should be more completely addressed. First, by censoring some simulation outcomes, valuable information on low-probability adverse events may be lost. Second, the rationale underlying the choice and management of censored values is not well articulated. Third, the ability to validate the model through comparison with observed events or the output of other E. coli O157:H7 risk assessments is compromised. The authors should consider removing the current algorithms for calculating dose-response parameters and replacing them with model elements based on evidence that is independent of O157:H7 epidemiologic data. That will allow for a stable evidence base and provide for some limited validation of model estimates with epidemiologic data. If no independent data are available, a formal statistically based updating algorithm could be used.
The other major issue is the transparency with which the draft model is presented—the extent to which “the rationale, the logic of development, constraints, assumptions, value judgements, decisions, limitations and uncertainties of the expressed determination are fully and systematically stated, documented, and accessible for review” (Codex Alimentarius Commission, 1999). Appendix C to the draft (released as a supplement on October 9, 2001)—a partial list of the model equations and code—is a good start in this effort. However, as noted in several instances in the committee’s review, the final assessment needs to make explicit the major assumptions underlying the model. Details of all the variables and equations used must be clearly listed, including data sources, units of measure, and distribution shapes as appropriate. And the analysis environment (now a spreadsheet with macros to automate the simulation process, implemented with and without “add-in” software to generate some statistical distributions) needs to be documented in a fashion that allows other professionals to more easily track the flow of the equations and trace errors. The credibility of the risk assessment will be greatly enhanced if its structure and inner workings are made more clearly visible to the outside world.
The management of uncertainty (a deficiency in the knowledge base)
and variability (the dispersion or distribution of some known quantity) is a challenge in all risk assessments. The committee’s review notes specific instances in which a thorough effort to characterize quantities by, for example, reconstructing regression scatters or modeling sampling error of empirical distributions would be appropriate.
A somewhat related question is the management of intervariable dependences. The draft treats all variables in a given equation as uncorrelated with one another. That assumption makes implementation of simulations more straightforward but does so at the possible expense of precision. The committee suggests that the authors evaluate available empirical information related to possible correlations among the variables and, in the absence of relevant data, carefully consider whether independence assumptions are most appropriate for the model.
Two unifying concepts underlie many of the committee’s comments on the draft risk assessment. Both are universal in the field of risk assessment. The first concept is that the risk assessment will be improved by making the inner workings of the analysis more explicit so that others may understand it better. Model building is a complex process, and the more accessible it is to others, the better the opportunities for its use and improvement. The second is that the lack of data on major components of the process being modeled hampers the construction of an informative risk assessment. Risk assessments are intended to collect and format knowledge in a way that is useful to decision-makers. Identification of data gaps is thus a strength of the process. The data deficiencies identified in the FSIS risk assessment should serve as the foundation for a delineation of research priorities to be promoted or pursued so that the model and E. coli O157:H7 policy decisions can be improved in the future.
Abumuhor I, Kearns EH. 2002. Thrombotic Thrombocytopenic Purpura. The Virtual Health Care Team, School of Health Professions and the School of Medicine, University of Missouri, Columbia. www.vhct.org/case2300/morbidity.shtml, accessed March 14, 2002.
CDC (Centers for Disease Control). 1982. Epidemiologic Notes and Reports: Isolation of E. coli O157:H7 from Sporadic Cases of Hemorrhagic Colitis—United States. Morbidity and Mortality Weekly Report 31(43):580, 585.
Codex Alimentarius Commission. 1999. Principles and Guidelines for the Conduct of Microbiological Risk Assessment. CAC/GL-30. Food and Agricultural Organization of the United Nations.
Elder RO, Keen JE, Siragusa GR, Barkocy-Gallagher GA, Koohmaraie M, Laegreid WW. 2000. Correlation of enterohemorrhagic Escherichia coli O157 prevalence in feces, hides, and carcasses of beef cattle during processing. Proceedings of the National Academy of Sciences 97(7):2999–3003.
ERS (Economic Research Service), US Department of Agriculture. 2001. ERS Estimates Foodborne Disease Costs at $6.9 Billion per Year. www.ers.usda.gov/Emphases/SafeFood/features.htm, accessed July 30, 2002.
FDA (Food and Drug Administration) 2002. Escherichia coli O157:H7. Center for Food Safety & Applied Nutrition Foodborne Pathogenic Microorganisms and Natural Toxins Handbook. vm.cfsan.fda.gov/~mow/chap15.html, accessed March 14, 2002.
Mead PS, Slutsker L, Dietz V, McCaig LF, Bresee JS, Shapiro C, Griffin PM, Tauxe RV.1999. Food-related illness and death in the United States. Emerging Infectious Disease 5(5):607–625.
USDA-FSIS (US Department of Agriculture, Food Safety and Inspection Service). 1996. Pathogen Reduction; Hazard Analysis and Critical Control Point (HACCP) Systems; Final Rule. Part II, 9 CFR Part 304, et al. Federal Register: July 25, 1996 61(144):38805– 38956.
USDA-FSIS. 2001. Draft Risk Assessment of the Public Health Impact of Escherichia coli O157:H7 in ground beef. September 7, 2001. [Draft report Appendix C dated October 9, 2001; Appendix D undated but released October 31, 2001.]