. "Letter Report
." Review of Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: Letter Report. Washington, DC: The National Academies Press, 2009.
The following HTML text is provided to enhance online
readability. Many aspects of typography translate only awkwardly to HTML.
Please use the page image
as the authoritative form to ensure accuracy.
Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report
INSTITUTE OF MEDICINE OF THE NATIONAL ACADEMIES
Committee on Review of the Use of Process Control Indicators in theFSIS Public Health Risk-Based Inspection System
March 13, 2009
Carol Maczka, Ph.D.
Assistant Administrator
USDA Food Safety and Inspection Service
Office of Data Integration and Food Protection
South Agriculture Building 1400 Independence Avenue, S.W., Room 3130 Washington, DC 20250
Dear Dr. Maczka,
At the request of the Food Safety and Inspection Service (FSIS), the Institute of Medicine (IOM)—under the auspices of the Standing Committee on the Use of Public Health Data in FSIS Food Safety Programs—established the Committee on Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System to review criteria developed by FSIS for ranking establishments based on relative risk. The body of this letter report provides the committee’s findings and recommendations regarding whether FSIS has adequately defined and identified indicators of process control that will be used to rank establishments and allocate agency inspection resources to protect public health. Specifically, the committee has evaluated how FSIS is proposing to use its available data to develop risk-based criteria for ranking establishments, as described in the technical report Public Health Risk-BasedInspection System for Processing and Slaughter (PHRBIS; FSIS, 2008b).
SUMMARY
Overall, the committee finds FSIS’s commitment to developing a risk-based inspection system commendable and agrees with the general concept of using process control indicators as part of an algorithm to rank establishments in different levels of inspection. The committee also encourages FSIS to continue to provide the rationale and scientific evi-
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole. Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 1
Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report
INSTITUTE OF MEDICINE OF THE NATIONAL ACADEMIES
Committee on Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System
March 13, 2009
Carol Maczka, Ph.D.
Assistant Administrator
USDA Food Safety and Inspection Service
Office of Data Integration and Food Protection
South Agriculture Building 1400 Independence Avenue, S.W., Room 3130 Washington, DC 20250
Dear Dr. Maczka,
At the request of the Food Safety and Inspection Service (FSIS), the Institute of Medicine (IOM)—under the auspices of the Standing Committee on the Use of Public Health Data in FSIS Food Safety Programs—established the Committee on Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System to review criteria developed by FSIS for ranking establishments based on relative risk. The body of this letter report provides the committee’s findings and recommendations regarding whether FSIS has adequately defined and identified indicators of process control that will be used to rank establishments and allocate agency inspection resources to protect public health. Specifically, the committee has evaluated how FSIS is proposing to use its available data to develop risk-based criteria for ranking establishments, as described in the technical report Public Health Risk-Based Inspection System for Processing and Slaughter (PHRBIS; FSIS, 2008b).
SUMMARY
Overall, the committee finds FSIS’s commitment to developing a risk-based inspection system commendable and agrees with the general concept of using process control indicators as part of an algorithm to rank establishments in different levels of inspection. The committee also encourages FSIS to continue to provide the rationale and scientific evi-
OCR for page 2
Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report
dence serving as the basis for the proposed system and praises FSIS for its resilience as it improves the proposal with public comments. In general, the committee found it a challenge to evaluate the adequacy of indicators of process control to rank establishments and allocate agency inspection resources without a clear understanding of the rationale for the general approach. The committee’s deliberations, based on its review of the report PHRBIS, open meetings, and personal communications with FSIS, resulted in the following findings:
The proposed inspection system consists of two components: one based on process control indicators and a second based on public health impact. The committee was tasked to review only the first component, but found it difficult to completely exclude deliberations on indicators of public health impact.
The report PHRBIS lacks details that are crucial to its evaluation. For example, the description of the algorithm, the scientific basis for the algorithm, the scientific basis for the use of the process indicators, the description and analysis of data, and the use of the process control indicator algorithm as it is integrated into the overall inspection system are not clearly articulated in the FSIS technical report.
The specific activities assigned to the three levels of inspection are not explicated. Likewise, the process of decision making to transfer a plant into a different level of inspection (LOI) (e.g., from LOI 2 to LOI 1) is not well defined. Further, it is not clear for how long or how frequently a plant in category LOI 2 or LOI 3 will be subject to an in-depth inspection or how these LOI designations relate to current regulatory requirements.
Key terms of the algorithm, such as “process control indicators,” are not well defined. In addition, the proposed algorithm assigns the same weight to all process indicators, even though they vary in their ability to predict loss of process control. For example, some indicators may predict future loss of control (e.g., the rate of health-related noncompliance records [NRs]), but others might only reflect past loss of control (e.g., recalls). For some foods, no adequate process control indicator is proposed.
The statistical analysis that was conducted to find associations between proposed process control indicators—lift analysis—is a data-mining tool appropriate for use in finding initial associations among events that occur infrequently. However, the identi-
OCR for page 3
Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report
fication of process control indicators to properly categorize plants based on risk to public health requires more complex statistical analysis as well as data that have been collected for the purpose of identifying such indicators.
Although there are limitations on the use of pathogenic organisms or Salmonella verification testing results as indicators of process control (e.g., infrequency of events), the committee concludes that the use of such testing to categorize plants in different levels of inspection is appropriate, if the recommendations stated in this report are followed.
FSIS currently tests each product class for different microorganisms, for different purposes, and with different underlying assumptions. The applicability of these data to the FSIS algorithm is dependent on the specific protocols, assumptions, and statistical characteristics of each testing program. The FSIS technical report did not provide in-depth consideration of the statistics that underlie the specific microbiological testing protocols employed and the assumptions made when using such data (e.g., the magnitude of type I and type II errors).
The use of the rate of NR receipt as an indicator of process control is promising but presents limitations based on the nature of the NRs (e.g., they document failure to comply with a regulation but are not always associated with a loss of process control or a public health hazard; NRs are subjective in nature; statistical analysis was conducted by aggregating data from all facilities, which might have biased the results).
Other proposed process control indicators also present limitations. The use of public health-related recalls, enforcement actions, and outbreaks to rank establishments in different levels of inspection has been justified based on potential direct public health risk, a valid risk-management decision criterion. However, the initial data analysis has not provided scientific support for these decision criteria as predictive of a loss of process control or for their association with other indicators.
The deliberations of the committee resulted in recommendations for improvement in the areas listed below that should be followed prior to implementing this algorithm:
OCR for page 4
Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report
Definition of key terms used in developing the algorithm, specifically, pointing out the limitations and consequences of using such terms in the context of the proposal;
Design of the algorithm, by conducting a risk-ranking activity to better identify process control indicators and their relative importance;
Collection or retrieval of additional data for the purpose of confirming the current process control indicators as well as exploring the use of new potential process control indicators to improve the sensitivity of the algorithm; and
Development of procedures to validate the algorithm.
CHARGE TO THE COMMITTEE
Responding to the request of the Food Safety and Inspection Service of the U.S. Department of Agriculture (USDA), the Institute of Medicine of the National Academies appointed the nine-member ad hoc Committee on Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System. Committee members provided expertise in meat and poultry microbiology, molecular biology methods, design and operation of processing establishments, risk analysis and decision-making tools, meat and poultry inspection, and foodborne disease epidemiology and public health. The committee met three times during the course of its work. The first meeting (Appendix A: Meeting Agendas) was held on November 6-7, 2008, in conjunction with a public data-gathering session with FSIS representatives, who provided program background and an in-depth description of the committee’s task (Box 1). The committee’s second meeting on December 17-18, 2008, also included Dr. Artur Dubrawski, of Carnegie Mellon University, and Dr. Marc Huckabee and Dr. Curtis Travis, consultants to FSIS from Science Applications International Corporation, who conducted the statistical analysis. During an open session of that meeting, these invitees responded to the committee’s questions about the statistical analysis of the data on process control indicators that were used by FSIS to establish the proposed risk-based algorithm. In addition to discussions with FSIS representatives and consultants, the committee formally requested data and information from FSIS through the Freedom of Information Act, as suggested by FSIS representatives. The committee deliberated on the following process control indicators and the data analysis approaches used by FSIS to evaluate their potential inclusion in the algorithm:
OCR for page 5
Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report
Salmonella verification testing in raw meat and poultry
Pathogen testing in ready-to-eat (RTE) meat and poultry (Salmonella enterica, Listeria monocytogenes, and Escherichia coli O157:H7) and raw ground beef and its components (E. coli O157:H7)
Noncompliance records
Enforcement actions
Class I and II recalls
Pulsed field gel electrophoresis (PFGE) patterns of Salmonella serovars of particular human health concern for isolates derived from the raw meat and poultry Salmonella verification testing program
System for Tracking E. coli O157:H7 Positive Suppliers (STEPS)
The committee also discussed the potential use of other indicators that were not included in the FSIS proposal. Findings and recommendations were drafted. A third committee meeting was held on January 13-14, 2009, in Washington, DC, to finalize its findings and recommendations and to prepare the report for external review.
The committee reviewed the data and statistical analysis (Appendixes D and E of the technical report Public Health Risk-Based Inspection System for Processing and Slaughter [FSIS, 2008b]) provided for the proposed indicators listed above. Appendix D of that report includes a description of the data used; Appendix E describes the data analysis that was conducted and the conclusions derived thus far. Appendix D and E also include limitations of the data and analysis and the rationale for the design of the algorithm.
At the request of FSIS and because another National Academy of Sciences (NAS) committee (Committee on Review of the Food Safety and Inspection Service [FSIS] Risk-Based Approach to Public Health Attribution) was assigned the task, data on volume and food attribution were not reviewed by this committee. FSIS noted that this algorithm would undergo improvements during the committee’s deliberation and, therefore, the proposal should be considered preliminary; since the publication of its technical report, FSIS has slightly modified the selection of process indicators. The committee based its deliberations on the updated version of the algorithm that was presented at its meeting on November 6-7, 2008 (see the indicators of process control and levels of inspection
OCR for page 6
Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report
in Appendix B of this report). The committee additionally reviewed several other FSIS reports, such as the 2008 technical report on poultry slaughter provided to the National Advisory Committee on Meat and Poultry Inspection (NACMPI) (FSIS, 2008a), to better understand the evolution of the FSIS algorithm.
This letter report begins with a background description of the FSIS initiative of a risk-based inspection system. Overall recommendations and findings are followed by recommendations for each specific process control indicator reviewed. The agenda of the workshop held on November 6-7, 2008, and the agenda for the open session of the second meeting are provided in Appendix A. Appendix B lists the indicators of process control corresponding to each level of inspection. Appendixes C and D contain a list of acronyms and a glossary, respectively. Appendix E lists the committee members’ biosketches.
BOX-1
Statement of Task
An ad hoc committee will review whether the Food Safety and Inspection Service (FSIS) has adequately defined and identified indicators of process control to protect public health that will be used to rank establishments and allocate agency inspection resources. Specifically, the committee will evaluate how FSIS is proposing to use its available data to develop a relative risk ranking of establishments described in the technical report Public Health Risk-Based Inspection System for Processing and Slaughter, publicly posted at http://www.fsis.usda.gov/Regulations_&_Policies/National_Advisory_Committe e_on_Meat_&_Poultry/index.asp.
BACKGROUND
Public Health Risk-Based Inspection System for Processing and Slaughter
The Food Safety and Inspection Service, the USDA agency responsible for ensuring the safety of meat, poultry, and egg products, has examined a number of strategies to develop a risk-based food safety system. Examples include the development and implementation of the Pathogen Reduction; Hazard Analysis and Critical Control Point (PR/HACCP) Systems; Final Rule in 1996 (FSIS, 1996), the development of microbiological performance standards (FSIS, 1999), and requirements for pathogen testing of ready-to-eat foods (Requirements for specific classes of product. 2008. 9 CFR Part 430).
OCR for page 7
Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report
In January 1997, President Clinton announced a Food Safety Initiative to reduce the incidence of foodborne disease in the United States. Among other changes, government agencies in charge of ensuring food safety were directed to improve inspections and enforce HACCP compliance in establishments that process meat and poultry (FDA-USDA-EPA-CDC, 1997). It was anticipated that implementation of the HACCP system would be accompanied by concurrent changes in inspection procedures. In 2003, the IOM Committee on Review of the Use of Scientific Criteria and Performance Standards for Safe Food found that the inspection of FSIS-regulated establishments relied largely on visual and organoleptic observations rather than on risk to public health (IOM, 2003). Although these are important and necessary elements of a plant survey, an improved, risk-based inspection system would assign levels of inspection to establishments according to the magnitude of their product’s risk to the public’s health. Other organizations, including the National Academies (NRC, 1987; IOM, 1990) and the Government Accountability Office (GAO, 1992), have previously emphasized the need for a risk-based inspection system for meat and poultry products.
In 2006, FSIS initiated the development of a risk-based inspection system. In its technical report Public Health Risk-Based Inspection System for Processing and Slaughter (hereafter referred to as PHRBIS) (FSIS, 2008b), FSIS proposes a decision-making tool to rank establishments according to their risk to public health by categorizing them first acccording to their level of process control and then by the impact on public health of the food produced. In addition, FSIS intends to upgrade several other elements of the proposed inspection system. For example, FSIS plans to strengthen its information technology system to enable inspection personnel to enter data on hazard analysis and make subsequent decisions in a more integrated and objective manner (FSIS, 2008b). FSIS also plans to train its inspection force (inspectors and supervisors) in effective use of the proposed inspection system tools. For example, in addition to continuing routine inspection training, a group of in-plant inspectors will receive training to enhance their understanding of establishment food safety systems, including HACCP plans or sanitary requirements. Supervisors will also be trained to use a more streamlined inspection review process (E. Dreyling, FSIS, personal communication, December 13, 2008).
As FSIS describes in its technical report PHRBIS, the proposed tool has evolved with input from stakeholder groups as well as the USDA’s National Advisory Committee for Meat and Poultry Inspection. An important innovation of this current proposal is to rely, where possible, on
OCR for page 8
Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report
data collected in conjunction with FSIS’s regulatory programs (FSIS, 2008b). The ultimate aim is the production of an effective tool for achieving the Healthy People 2010 goals of reducing foodborne disease caused by Salmonella, Escherichia coli O157:H7, and Listeria monocytogenes (HHS, 2000). FSIS concludes that to protect public health most effectively, inspection resources have to be allocated based on the degree of risk to public health presented by each processing plant. Therefore, a key element of the risk-based inspection system is an algorithm for categorizing slaughter and processing plants according to risk so that inspection efforts are focused on those establishments having the greatest impact on public health (FSIS, 2008b). The algorithm consists of two consecutive steps to rank an establishment’s risk: a first component to determine the establishment’s level of process control (i.e., identifying attributes that indicate whether the establishment is maintaining control) and a second component to quantify public health impact (i.e., the volume of the commodity produced at the establishment together with public health attribution of the food produced) (FSIS, 2008b). The committee was charged with reviewing the scientific basis of and rationale for the first component of the algorithm—the data and data analysis that were used by FSIS to identify indicators for categorizing establishments according to their level of process control. A second NAS ad hoc committee (Committee on Review of the Food Safety and Inspection Service [FSIS] Risk-Based Approach to Public Health Attribution) was charged with reviewing the second component, the public health attribution system. Because the two components are closely related (e.g., the volume of production in an establishment influences the sampling plans for pathogen testing programs that FSIS proposes to use to indicate process control) and included in an overall inspection system, this committee found it difficult to completely exclude deliberations on indicators of public health impact.
OVERALL FINDINGS AND RECOMMENDATIONS
This section of the report provides overall findings and recommendations related to strengthening the proposed FSIS risk-based decision tools for ranking establishments. It is followed by a section that provides more specific recommendations for each proposed indicator. Prior to implementing the algorithm, the recommendations in this report should be followed.
OCR for page 9
Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report
General Approach
The committee concurs that a risk-based approach to inspection is essential and commends FSIS for undertaking such a daunting and controversial endeavor. The committee found that the development and use of a model (algorithm) to categorize establishments based on risk can ultimately be a systematic approach to realizing and implementing decision criteria in a transparent, predictable manner. However, the committee found it challenging to comprehend the framework, concepts, and rationale that FSIS applied in several segments of the proposed model. The descriptions of the algorithm, the scientific basis for the selection of the proposed process indicators, the analysis of data, and the use of the process control indicator algorithm as it is integrated in the overall inspection system were not clearly stated in the technical report PHRBIS that was provided to the committee. For example, FSIS uses the term “algorithm” to describe its decision-making tool to categorize plants into levels of inspection. As shown in Table 1, there are various definitions of the term algorithm.
However, in the context of a risk-based system, the term algorithm implies a mathematical model. Since FSIS did not construct a mathematical model, it would be more precise to use the designation decision tool or framework. To avoid confusion for the reader, the committee decided to retain the term algorithm for the purposes of this report.
TABLE 1 Definitions of Algorithm
Definition
Source
A set of rules for solving a problem in a finite number of steps, (e.g., finding the greatest common divisor) A procedure for solving a mathematical problem (e.g., finding the greatest common divisor) in a finite number of steps that frequently involves repetition of an operation; broadly: a step-by-step procedure for solving a problem or accomplishing some end especially by computer
http://dictionary.reference.com/
http://www.merriam-webster.com/
A precise rule (or set of rules) specifying how to solve some problem
http://www.websters-online-dictionary.org/
OCR for page 10
Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report
Mathematics. A process, or set of rules, usually expressed in algebraic notation, now used especially in computing, machine translation, and linguistics
Medicine. A step-by-step procedure for reaching a clinical decision or diagnosis, often set out in the form of a flow chart, in which the answer to each question determines the next question to be asked
Oxford English Dictionary, 2nd edition, 1982
Any special method of solving a certain kind of problem; specifically, the repetitive calculations used in finding the greatest common divisor of two numbers
Webster’s New World Dictionary, 2nd college edition, 1982
Finding 1: Although the use of a model to categorize plants in levels of inspection is appropriate, the descriptions of the algorithm, the scientific basis for the use of the process indicators, the description and analysis of data, and the use of the process control indicator algorithm as it is integrated into the overall inspection system are not clearly articulated in the FSIS technical report.
RECOMMENDATION 1: The committee recommends that in addition to the improvements in data collection and analysis presented below, FSIS revise its proposal to improve the transparency and clarity of the description of the overall inspection system—in particular, the process control indicator algorithm, its scientific basis, and the type and analysis of data used. Further, FSIS should consider tailoring the proposal to its target audiences (e.g., plant managers, FSIS inspectors and supervisors, FSIS managers and scientists, outside expert panels) and providing them with supplemental information or reports.
Definitions of Process Control and Process Control Indicators
The FSIS (2008b) report does not adequately define various terms that are key to evaluating the proposed inspection system (e.g., algorithm, process control, process control indicator). The ambiguous use of these terms hampered the ability of the committee to understand the use of data and could result in misinterpretations and unnecessary disputes in the future. To avoid confusing the reader and for the purposes of this report, however, the committee opted to retain the terms process control and process control indicators while also pointing out the ambiguity of their usage.
OCR for page 11
Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report
The committee offers more clearly defined key terms and explains its interpretation of those terms for the purposes of this report. The concept of process control, which applies to all manufacturing companies and can be used broadly to address both quality and safety issues, is used in the context of the current report as a means to quantify how well an establishment is employing control measures to minimize pathogen contamination. Examples of definitions of process control are shown in Table 2.
TABLE 2 Definitions of Process Control
Definition
Source
At certain points in the processing of a food, control measures can be applied to prevent an unacceptable increase in a hazard, eliminate it, or reduce it to an acceptable level
Scientific Criteria to Ensure Safe Food (IOM, 2003, p. 94)
Activities involved in ensuring a process is predictable, stable, and consistently operating at the target level of performance with only normal variation
BusinessDictionary.com
(http://www.business dictionary.com/)
The inspection of work-in-progress to provide feedback on, and correct, a production process. First developed as a mechanical feedback mechanism, process control is now widely used to monitor and maintain the quality of output
bnet.com
(http://www.bnet.com/)
Method by which the input flow of processing plants is automatically controlled and regulated by various output sensor measurements. Process control can also describe the method of keeping processes within specified boundaries and minimizing variation within a process
Chemicals-technology.com
(http://www.chemicals-technology.com/glossary/)
The active changing of a process based on the results of process monitoring. Once the process monitoring tools have detected an out-of-control situation, the person responsible for the process makes a change to bring the process back into control
NIST/SEMATECH e-Handbook of Statistical Methods
(http://www.itl.nist.gov/div898/handbook/pmc/section1/pmc13.htm)
The automated control of a process. Process control is used extensively in oil refining, chemical processing, electrical generation, and the food and beverage industries where the creation of a product is based on a continuous series of processes being applied to raw materials
PCMag.com (http://www.pcmag.com/encyclopedia_term/0,2542,t=process+control&i=49753,00.asp)
In its 2003 report Scientific Criteria to Ensure Safe Food, IOM evaluated the use of scientific criteria and standards in food safety regulations (IOM, 2003). That report defines various terms used in food
OCR for page 50
Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report
recalls is further limited by their lack of specificity as well as their lack of timeliness (they may occur too late to identify issues in food processing); their use is reactive to a past food safety problem, not necessarily predictive of a future problem.
As FSIS states, the statistical analysis that estimates the ability of recalls to predict a loss of process control should allow for differentiation between recalls that have a public health impact and those that do not. A public health-related recall is often based on a laboratory test result—that is, the result of a microbiological test. Because of the limitations of sampling (size, frequency, etc.) and the sporadic nature of contamination, the isolation of a foodborne pathogen may not indicate a breakdown in process control. In some situations, especially related to E. coli O157:H7 in ground beef, a recall is based on failure to hold a product that has been tested for a pathogen, not a failure of the control process.
Finding 12: The use of public health-related recalls to rank establishments in different levels of inspection has been justified based on potential direct public health risk, a valid risk-management decision criterion. However, the initial data analysis has not provided scientific support for this decision criterion as being predictive of a loss of process control or for its association with other indicators.
RECOMMENDATION 14: Only health-related product recalls should be included in the model for ranking public health risks and assigning inspection resources. FSIS should continue to conduct assessments and take regulatory enforcement actions in plants following a recall.
STEPS Database
Use and Scientific Evidence
The System for Tracking E. coli O157:H7 Positive Suppliers database identifies suppliers of trim to grinding operations whose ground beef product tests positive for E. coli O157:H7. FSIS proposes to use this database to categorize LOIs for supplier establishments in the following manner (Dreyling, 2008; FSIS, 2008b; E. Dreyling, FSIS, personal communication, February 18, 2009):
OCR for page 51
Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report
LOI 1: Establishment has not been cited in the STEPS database more than once; or if it has, any related FSA and follow-up sampling has been completed more than 120 days previously, any related enforcement actions are closed, and establishment meets all other criteria for LOI 1.
LOI 2: For an establishment in the STEPS database more than once, any related FSA and follow-up sampling has been completed in the previous 120 days, and all related enforcement actions are deferred or in abeyance.
LOI 3: Establishment was in the STEPS database more than once within the previous 120 days.
The justification for using the STEPS database is that grinding operations lacking an E. coli O157:H7 intervention step that would decrease the likelihood of the presence of pathogens need a process control step to ensure that incoming trim products are not already contaminated with pathogens.
Committee’s Discussion
The committee recognizes the potential benefits of this approach and would be interested in seeing the details and data supporting it. As discussed above in relation to testing ground beef for E. coli O157:H7, FSIS should assess the role of testing trim for E. coli O157:H7 as a risk determinant.
Foodborne Disease Outbreaks
A foodborne outbreak is the occurrence of two or more cases of a similar illness resulting from the ingestion of a common food. FSIS proposes to use foodborne disease outbreaks to categorize establishments in levels of inspections as follows (Dreyling, 2008; FSIS, 2008b; E. Dreyling, FSIS, personal communication, December 13, 2008):
LOI 1: An establishment has not been linked to an outbreak; or if it has, any related FSA and follow-up sampling has been completed more than 120 days previously, any related enforcement actions are closed, and establishment meets all other criteria for LOI 1.
OCR for page 52
Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report
LOI 2: For an establishment that was linked to an outbreak, any related FSA and follow-up sampling has been completed in the previous 120 days, and all related enforcement actions are deferred or in abeyance.
LOI 3: Human illness was linked to an FSIS-regulated product from the establishment.
Finding 13: The use of foodborne disease outbreaks to rank establishments in different levels of inspection has been justified based on their potential direct public health risk, a valid risk-management decision criterion. However, the initial data analysis has not provided scientific support for use of this decision criterion to predict loss of process control or for its association with other indicators.
RECOMMENDATION 15: The committee recommends including foodborne disease outbreaks in the algorithm to categorize plants in levels of inspection. The committee also strongly recommends that FSIS systematically work with other appropriate federal and state agencies to routinely disseminate public reports of the results of the investigations into the plant and process failures associated with these outbreaks.
Salmonella Serotypes of Human Health Concern
If FSIS is planning to use specific serotypes as indicators of process control, serotypes not often linked to human health should also be considered. Since the potential application of serotype evaluation to the designation of facilities as LOI 1, 2, or 3 is dependent on establishing a clear relationship between individual serotypes and disease attribution, any recommendations by this ad hoc committee await the findings of the NAS Committee on Review of the Food Safety and Inspection Service (FSIS) Risk-Based Approach to Public Health Attribution.
Consumer Complaints
The use of consumer complaints as a potential indicator of process control was analyzed by FSIS and then dismissed due to the challenge of overcoming its limitations (FSIS, 2008b). The committee agrees that the process currently employed to collect and analyze consumer complaints is not appropriate for use as an indicator of an establishment’s need for a
OCR for page 53
Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report
higher level of inspection. In addition to having no significant associations with other potential indicators (see Appendix E in FSIS, 2008b), consumer complaints may often incorrectly associate a food with an adverse health effect.
OTHER POTENTIAL INDICATORS OF PROCESS CONTROL
Microbial Test Results
The primary goal of process control for raw meat and poultry products is to limit the presence of fecal contamination, the source of enteric pathogenic microorganisms. Both Salmonella and generic E. coli are indicators of fecal contamination and, as such, indicators of loss of process control, and both were targeted by FSIS in the pathogen reduction HACCP regulation. The ideal process indicator is one that is present at sufficient levels and frequency to be measured on a routine basis. For some commodities (e.g., beef carcasses), Salmonella is currently found so rarely that its usefulness as an indicator is limited. It is envisioned that for other commodities where it is currently useful, Salmonella could become equally rare in the future. FSIS would benefit from identifying alternative microbial indicators that could augment current indicators of fecal contamination on a commodity-specific basis.
Data on generic E. coli are collected by individual plants on a regular basis, but are not used by FSIS. Establishments are not required to send such data to FSIS, only to make them available if requested. According to FSIS, there are two limitations to the collection of data on generic E. coli that prevent FSIS from using them as indicators (C. Travis, Science Applications International Corporation, personal communication, December 13, 2008). One is FSIS’s concern about the comparability of data resulting from a variety of different testing methods. Although the PR/HACCP regulation states that validated methods of testing should be used, there is no required single standard testing methodology or sampling procedure. It would appear that this could readily be corrected if FSIS articulated the specific methodological requirements (e.g., sensitivity, specificity, reproducibility, repeatability) of its current standard methods and its expectation that similar performance would be achieved by alternative validated methods. The second limitation is that the agency does not currently have the information technology capability to
OCR for page 54
Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report
retrieve and process such data efficiently. It is worth noting that in its May 2008 technical report to the NACMPI on poultry slaughter, FSIS presented convincing evidence of the potential utility of generic E. coli as a process control indicator and suggested that it was considering a new performance standard based on its use (FSIS, 2008a). These findings were based on a detailed study of various potential indicators of process control performed by FSIS and ARS. FSIS indicated that it has analyzed a 2006-2008 generic E. coli data set from its baseline program, but because this data set is small, the results were inconclusive (FSIS, 2008a).
Indicator organisms in RTE foods are used not as indicators of fecal contamination but rather as indicators of other control measures, such as the adequacy of the microbicidal step, prevention of recontamination, and maintenance of proper storage conditions (e.g., refrigeration). The FSIS algorithm does not currently include the use of any indicator microorganism for assessing process control in RTE foods. Given the low frequency of L. monocytogenes or other pathogens, FSIS would benefit from identifying appropriate alternative microbial indicators that could be used to assess applicable process controls in RTE foods. Although generic E. coli might not be an indicator of fecal contamination in RTE products, it is still a valuable indicator of general sanitation, recontamination problems, and temperature abuse.
Finding 14: Microbes currently used as process control indicators are only rarely found in some commodities and are therefore of limited usefulness (e.g., Salmonella in ground beef). It is anticipated that in the future, Salmonella will be even less frequent and therefore less valuable as an indicator. Furthermore, in the proposed algorithm, there are no identified process control indicators for RTE foods.
RECOMMENDATION 16: FSIS should investigate the potential utility of industry data on generic E. coli as an indicator of process control. The committee recognizes the challenges of this approach, but encourages FSIS to act promptly to complete the analysis of the data it has already acquired, collect additional data as necessary, and analyze them for their predictive ability as potential indicators of process control.
Use of the HACCP System
A HACCP plan is developed by identifying steps in a specific meat and poultry process that are critical to ensuring food safety and is meant
OCR for page 55
Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report
to include the application of corrective actions when those critical control points are not met. Critical control points in HACCP plans were regarded as points in a process in need of specific interventions that, if failed, might result in an end product with risks to public health. If a commodity that was produced under a process deviation leaves the plant without corrective action, this constitutes a loss of control. By regulation, the control point limits are to be validated and verified by the establishments (Hazard Analysis and Critical Control Point [HACCP] Systems. Validation, Verification, Reassessment. 2008. 9 CFR § 417.4). Therefore, it may be appropriate for FSIS to study the feasibility of a system in which deviations from control point limits are incorporated as NRs and used to categorize plants according to the inspection level required. In fact, for most HACCP plans, critical control points and limits should be similar in nature for all facilities processing the same commodity. The committee acknowledges that for the processing of raw product, defining the control points is challenging; in these cases, more weight could be allocated to pathogen contamination as a control indicator.
RECOMMENDATION 17: The committee recommends that FSIS consider using specific critical control point deviations as indicators of process control. Process deviations should be integrated into an algorithm to categorize plants according to the level of inspection needed. Because of inherent problems in the use of NRs described above, the committee recommends redefining public health-related NRs and creating new ones where appropriate so that they reflect the current view of HACCP as a food safety control approach. This approach should identify true science-based indicators of process control. This concept should be included in inspection training programs. USDA should conduct a pilot study in a few plants to determine if the new NRs based on HACCP critical control point adherence are valid and useful parameters to be considered as predictors of loss of process control. This should be followed by longitudinal studies designed to validate the new NRs.
Value of Real-Time In-Plant Data Acquisition
The committee supports FSIS’s efforts to explore options for rapid collection and reporting of real-time data that indicate potential failures of process control. The real-time data should focus on objective measures of control (e.g., critical control points) for the process and take advantage of electronic data-capturing tools.
OCR for page 56
Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report
The committee agrees that FSIS should use in-plant inspection personnel to collect real-time data. They would provide immediate input into the algorithm indicating a potential failure of process control. To be effective, objective process performance measures should be defined. For example, real-time tracking of repetitive instances of noncompliance that are related to food safety and that affect process control would be reported and used as indicators of process control. Also, whenever feasible, performance measures should allow action to be taken before the process fails. The committee supports FSIS’s current activities to develop such a system and urges that it do so concurrently with carefully designed training of its inspection and supervisory personnel.
CONCLUSION
The committee recognizes the magnitude of the task of designing a risk-based system to rank meat and poultry slaughtering and processing establishments based on their impact on public health. The committee notes that at the request of FSIS, only the data on and analysis of indicators of process control were reviewed. Other components of the algorithm (e.g., volume) vital to determining its applicability were not. FSIS should include as part of the proposed inspection system a specific plan for when and how it will evaluate the system. Scientific verification and validation are essential to evaluate the success or failure of the new program.
The committee agrees with the general concept of using process control indicators as part of an algorithm to rank establishments in different levels of inspection. The committee recommends that FSIS continue the collection and analysis of data and, in consultation with stakeholders and expert panels, continue to improve its proposed risk-based inspection system so that it more effectively allocates inspection resources according to risk. Prior to implementing this algorithm, the recommendations in this report should be followed. Specifically, the committee emphasizes the need to align the process control indicators of a risk-based inspection system with HACCP, a framework required throughout the meat and poultry slaughtering and processing industry that serves to minimize the risk of foodborne illness.
The committee also recommends that FSIS improve the clarity and transparency of the algorithm so that its intent, scientific basis, and implementation are clearly articulated and understood by all stakeholders. One option for FSIS to communicate effectively with stakeholders would
OCR for page 57
Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report
be to produce supplemental informative documents targeted to specific audiences (e.g., inspectors, plant managers), in addition to a technical report. Also, because this new algorithm would bring about changes in inspection procedures, a parallel training program for the inspection force would likewise be necessary.
The Committee on Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System thanks FSIS for the opportunity to review the technical report Public Health Risk-Based Inspection System for Processing and Slaughter and hopes that its findings and recommendations are useful. The committee will be available to FSIS for any clarifications regarding this letter.
Sanford Miller,
Chair
Committee on Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System
Attachments
Appendix A Meeting Agendas
Appendix B Levels of Inspection
Appendix C Acronyms
Appendix D Glossary
Appendix E Biographical Sketches of Committee Members
REFERENCES
CDC (Centers for Disease Control and Prevention). 2008. Preliminary FoodNet data on the incidence of infection with pathogens transmitted commonly through food—10 states, 2007. Morbidity and Mortality Weekly Report 57(14):366-370.
Dreyling, E. 2008. Public health risk ranking for processing and slaughter establishments. Presented at the November 6, 2008, meeting of the IOM Committee on Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System, Washington, DC.
FDA-USDA-EPA-CDC (U.S. Food and Drug Administration-U.S. Department of Agriculture-U.S. Environmental Protection Agency-Centers for Disease Control and Prevention). 1997. Food Safety from farm to table: A national food safety initiative. Report to the President. Available at http://www.foodsafety.gov/~dms/fsreport.html (accessed February 20, 2009).
FSIS (Food Safety and Inspection Service). 1996. 9 CFR Parts 304, 308, 310, 320, 327, 416, and 417. Pathogen Reduction; Hazard Analysis and Critical
OCR for page 58
Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report
Control Point (HACCP) Systems; Final Rule. Federal Register 61(144):38805-38989.
FSIS. 1999. 9 CFR Parts 301, 317, 318, 320, and 381. Performance standards for the production of certain meat and poultry products; Final Rule. Federal Register 64(3):732-749.
FSIS. 2008a. Improvements for poultry slaughter inspection. Technical report. Available at http://www.fsis.usda.gov/Regulations_&_Policies/NationalAdvisory_Committee_on_Meat_&_Poultry/index.asp#August (accessed January 26, 2009).
FSIS. 2008b. Public health risk-based inspection system for processing and slaughter. Technical report. Available at http://www.fsis.usda.gov/Regulations_&_Policies/National_Advisory_Committee_on_Meat_&_Poultry/index.asp#August (accessed January 7, 2009).
FSIS. 2008c. Progress report on Salmonella testing of raw meat and poultry products, 1998-2007. Available at http://www.fsis.usda.gov/Science/Progress_Report_Salmonella_Testing/index.asp (accessed February 23, 2009).
FSIS. 2008d. The FSIS Microbiological Testing Program for Ready-to-Eat (RTE) Meat and Poultry Products, 1990-2007. Available at http://www.fsis.usda.gov/Science/Micro_Testing_RTE/index.asp#trends (accessed February 23, 2009).
FSIS. 2008e. Current recalls and alerts. Available at http://www.fsis.usda.gov/Fsis_Recalls/Open_Federal_Cases/index.asp (accessed December 7, 2008).
GAO (U.S. General Accounting Office). 1992. Food safety and quality. Uniform, risk-based inspection system needed to ensure safe food supply. Report to the Chairman, Subcommittee on Oversight and Investigations, Committee on Energy and Commerce, U.S. House of Representatives. Resources, Community, and Economic Division, Report 92-152. Washington, DC.
Giovannacci, I., C. Ragimbeau, S. Queguiner, G. Salvat, J. L. Vendeuvre, V. Carlier, and G. Ermel. 1999. Listeria monocytogenes in pork slaughtering and cutting plants. Use of RAPD, PFGE and PCR-REA for tracing and molecular epidemiology. International Journal of Food Microbiology 53(2-3):127-140.
HHS (U.S. Department of Health and Human Services). 2000. Healthy people 2010, Volume 1. Washington, DC: U.S. Government Printing Office. Available at http://www.healthypeople.gov/Publications/ (accessed January 7, 2009).
ICMSF (International Commission on Microbiological Specifications for Foods). 2002. Microorganisms in foods 7. Microbiological testing in food safety management. New York: Klewer Academic/Plenum Publishers.
IOM (Institute of Medicine). 1990. Cattle inspection. Washington, DC: The National Academy Press.
OCR for page 59
Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report
IOM. 1998. Ensuring safe food: From production to consumption. Washington, DC: The National Academy Press.
IOM. 2003. Scientific criteria to ensure safe food. Washington, DC: The National Academies Press.
Keto-Timonen, R., R. Tolvanen, J. Lundén, and H. Korkeala. 2007. An 8-year surveillance of the diversity and persistence of Listeria monocytogenes in a chilled food processing plant analyzed by amplified fragment length polymorphism. Journal of Food Protection 70(8):1866-1873.
Lundén, J. M., T. J. Autio, A. M. Sjöberg, and H. J. Korkeala. 2003. Persistent and nonpersistent Listeria monocytogenes contamination in meat and poultry processing plants. Journal of Food Protection 66(11):2062-2069.
NRC (National Research Council). 1987. Poultry inspection: The basis for a risk-assessment approach. Washington, DC: National Academy Press.
Peccio, A., T. Autio, H. Korkeala, R. Rosmini, and M. Trevisani. 2003. Listeria monocytogenes occurrence and characterization in meat-producing plants. Letters in Applied Microbiology 37(3):234-238.
Thévenot, D., A. Dernburg, and C. J. Vernozy-Rozand. 2006. An updated review of Listeria monocytogenes in the pork meat industry and its products. Applied Microbiology 101(1):7-17.
WHO-FAO (World Health Organization-Food and Agriculture Organization of the United Nations). 2004. Risk assessment of Listeria monocytogenes in ready-to-eat food: Technical report. Microbiological Risk Assessment Series 5. Geneva, Switzerland: WHO.
OCR for page 60
Review of the Use of Process Control Indicators in the FSIS Public Health Risk-Based Inspection System: A Letter Report
This page intentionally left blank.
INSTITUTE OF MEDICINE OF THE NATIONAL ACADEMIES