Review of the Methodology Proposed by the Food Safety and Inspection Service for Followup Surveillance of In-Commerce Businesses: A Letter Report (2009)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

BOARD ON AGRICULTURE AND NATURAL RESOURCES 500 Fifth Street, NW Washington, DC 20001 Phone: 202 334 3062 Fax: 202 334 1978 E-mail: banr@nas.edu www.dels.nas.edu/banr September 30, 2009 Mr. Matthew Gonzales Program Analyst Program Evaluation and Improvement Staff USDA/FSIS/OPEER 1400 Independence Ave., SW Room 3833 South Building Washington, DC 20250-3700 Dear Mr. Gonzales: At the request of the Food Safety and Inspection Service (FSIS), the National Academies’ Division on Earth and Life Studies established the ad hoc Committee for the Review of the Methodology Proposed by the Food Safety and Inspection Service for Followup Surveillance of In-Commerce Businesses. The committee’s charge was to review and comment on the assumptions, risk factors, and methodology FSIS proposes to use to prioritize followup surveillance at in-commerce business with prior surveillance history. The committee held one in-person meeting and two conference calls. During the first meeting, FSIS staff presented their proposed approach and answered questions raised by committee members. There were also presentations by representatives of the Association of Food and Drug Officials, the Food and Drug Administration, and the Food Marketing Institute. The remainder of the committee’s time was spent preparing the report and responding to comments of external reviewers. This letter report contains the committee’s findings and recommendations. The committee commends FSIS for continuing its efforts to develop in-commerce surveillance activities, based on sound scientific principles, for the protection of public health. Sincerely, John N. Sofos, Chair Committee for the Review of the Methodology Proposed by the Food Safety and Inspection Service for Followup Surveillance of In-Commerce Businesses ix 



SUMMARY The National Academies issued a report on initial surveillance of in-commerce businesses by the U.S. Department of Agriculture Food Safety and Inspection Service (FSIS). FSIS requested feedback on its proposed process for priority-setting for followup surveillance in cases in which initial surveillance did not lead to an investigation or enforcement action, and this report is a sequel to the first report. To provide context for the current report, the Committee for the Review of the Methodology Proposed by the Food Safety and Inspection Service (FSIS) for Followup Surveillance of In-Commerce Businesses summarized and commented on the response by FSIS to the earlier report. FSIS agreed that whether there is oversight by other authorities should be the primary risk consideration and that product volume, inherent hazard, consumer susceptibility, and food-defense vulnerability should be used as secondary risk considerations. FSIS, however, seems to have given equal weight to all secondary risk considerations instead of assigning relative importance to them. As illustrated in the appendices to this report, the committee’s proposed stepwise decision process for priority-setting for surveillance involved setting two priority levels: high and low (or Tiers 1 and 2). With the exception of retail, institutions, restaurants, and custom exempt, in its draft document, FSIS seems not to have set priorities among business types primarily according to oversight by other regulatory authorities inasmuch as it did not change the business types that it had originally placed in Tier 1. When final decisions are made, FSIS should clearly state which of the initial committee’s recommendations will be adopted and the rationale for the decisions. FSIS Directive 8010.1, Revision 2, outlines procedures for conducting both initial and followup surveillance of in-commerce businesses. However, the directive contains subjective material, and several elements are unclear. The committee recommends that FSIS modify the directive to increase clarity and definition, to the extent possible, and to include more specific guidance about the differences between initial and followup surveillance, types of findings that should result in followup surveillance, and guidelines for determining the period for completion of followup. A key component of FSIS surveillance of in-commerce businesses is the development and implementation of a computer database known as the In-Commerce System (ICS), which the committee finds is a useful tool. FSIS should consider carefully the types of data that are recorded in the ICS because these data will be important in deciding whether followup surveillance is needed by FSIS or by state or local authorities. To the extent possible, quantitative, rather than qualitative, information should be recorded. FSIS should also consider developing more objective surveillance forms, such as those used by the Food and Drug Administration. The committee believes that additional data collection is an essential component of both initial and followup surveillance for eventual development of the desired risk-based surveillance system. FSIS is proposing a framework for priority-setting for followup surveillance. The process—which leads to high, medium, or low rankings for findings in each of two tiers for each initial surveillance finding—has not been clearly stated, and at this point there are no written criteria for defining or distinguishing among the rankings. In addition, it is unclear how the rankings were assigned in each of the two tiers and how different priority levels were assigned to a given finding in each tier. As presented to the committee, the proposed framework is not reproducible. 1 

2 REVIEW OF IN-COMMERCE SURVEILLANCE Among the committee’s recommendations is a re-evaluation of organizational procedures and structures within FSIS, with one objective being improvement of use of staff time. Currently, approximately 10% of the time of 120 employees is assigned to in-commerce surveillance. One example is to consider whether it would be more efficient and effective to use 100% of the time of 12 employees. Such a change, if feasible, could enhance training and facilitate collaboration with other jurisdictions. Finally, it is unclear how FSIS plans to determine the relative proportions of investigators’ time that would be spent on initial surveillance and followup surveillance and how priorities would be set for the two activities on a regular basis. That is a key element of in-commerce surveillance, but it was not part of the task assigned to the committee. As indicated in the In-Commerce I report, this committee re-emphasizes the need to avoid duplicative and redundant inspection efforts, making sure, however, there is adequate surveillance in all situations. 

INTRODUCTION The Food Safety and Inspection Service (FSIS) of the U.S. Department of Agriculture is the federal agency responsible for enforcing the regulatory statutes applicable to meat, poultry, and egg products. The goal of the regulations is to ensure that the products are wholesome and safe for human consumption. To enforce its regulatory mandate outside slaughter facilities and processing plants (which are inspected continuously or daily, respectively), FSIS conducts surveillance of businesses that are engaged in the transport, storage, distribution, and sale of meat, poultry, and egg products. The agency refers to those as in-commerce businesses and uses the term surveillance to describe its oversight of such businesses. Surveillance of in-commerce businesses is the responsibility of the Compliance and Investigations Division (CID) of the FSIS Office of Program Evaluation, Enforcement, and Review. However, the CID has limited resources to carry out that task. There are approximately 120 personnel with only 10% of their time available for surveillance of products in commerce, and more than 700,000 facilities may be considered for surveillance visits. The business types include restaurants, retailers, institutions, warehouses, transporters, distributors, animal-food suppliers, food banks, renderers, retail salvage companies, custom slaughter companies, exempt poultry companies, and companies that handle dead, dying, disabled, or diseased (4-D) animals. About 90% of CID investigators’ time is allocated to emergency responses, outbreak investigations, product recalls, and withdrawal activities; these activities merit higher priority because of their immediate and critical public-health importance. FSIS provides instructions to its compliance officers on how to conduct in-commerce surveillance activities.1 The activities are carried out at in-commerce locations to verify that firms whose businesses involve FSIS-regulated products prepare, store, transport, sell, or offer for sale or transportation such products in compliance with FSIS statutory and regulatory requirements. An initial surveillance visit that identifies no apparent violation of FSIS requirements leads to no scheduled followup. However, initial surveillance may result in CID personnel’s conducting an investigation, taking product-control action,2 or scheduling followup surveillance. A goal of FSIS is to manage all in-commerce surveillance activities (both initial and followup) with a computer database, the In-Commerce System (ICS). The ICS is being designed to record findings from initial and followup surveillance, including the characteristics of each business examined. An overview of the ICS is provided later in this report. A previous National Academies committee, referred to as the In-Commerce I (I-C I) committee, reviewed and commented on FSIS proposals for new methods to organize its initial surveillance of in-commerce businesses. Its report was delivered to FSIS on March 13, 2009.3 1 FSIS Directive 8010.1, Rev. 2, Methodology for Conducting In-Commerce Surveillance Activities. http://www.fsis.usda.gov/OPPDE/rdad/FSISDirectives/8010.1Rev2.pdf. Accessed July 31, 2009. 2 The process for initiating a product-control action is outlined in FSIS Directive 8410.1, Detention and Seizure. http://www.fsis.usda.gov/OPPDE/rdad/FSISDirectives/8410.1Rev5.pdf. Accessed July 31, 2009. Most product control-actions result in voluntary action by the product owners, such as voluntary disposal of the product. If a detained product is not be disposed of within 20 days, FSIS may request an order to seize it. 3 National Research Council, 2009, Review of the Methodology Proposed by the Food Safety and Inspection Service for Risk-Based Surveillance of In-Commerce Activities: A Letter Report. http://www.nap.edu/catalog.php?record_id=12634. 3 

4 REVIEW OF IN-COMMERCE SURVEILLANCE The committee provided FSIS with suggestions for improving its proposed changes for initial in- commerce surveillance activities. To provide context for the present report, by the In-Commerce II (I-C II) committee, those suggestions and FSIS’s responses are discussed in the next section. FSIS asked that the second committee (I-C II) review and comment on new proposals for procedures to organize followup surveillance activities of in-commerce businesses when necessary after an initial surveillance visit. The specific request is presented in Box 1, and the official statement of task to the I-C II committee is presented in Appendix A. The agenda for the open session in which the task was presented to the committee is given as Appendix B, and copies of the PowerPoint presentations by FSIS staff members to the I-C II committee, showing their proposed approach for followup activities, are in Appendix C. Box 1 Specific Request from FSIS FSIS is requesting that the National Academies convene a committee to provide feedback on FSIS’ proposed guidance to its Investigators concerning the prioritization of followup surveillance reviews in cases where initial surveillance did not rise to the level of an investigation or enforcement action. (From material provided to the committee by FSIS on June 29, 2009.) BACKGROUND Before commenting on the plan proposed by FSIS for followup surveillance, the present (I-C II) committee considers it appropriate to review the recommendations made by the I-C I committee about initial risk-based surveillance of in-commerce activities and the response by FSIS to them. Any strategy to revisit facilities for followup surveillance will be influenced by FSIS’s strategy for conducting initial in-commerce surveillance and by the findings of that activity. Summary of the Report of the In-Commerce I Committee Because it is impossible to carry out routine surveillance activities on all in-commerce business types and facilities with the existing resources, FSIS created a risk-based tier system for businesses considered critical, very important, and important to set priorities for visits to 13 business types. The National Academies was charged to examine the methods proposed by FSIS to inspect and regulate in-commerce activities using the proposed qualitative risk-based approach. To that end, the I-C I committee was formed to review and comment on the data, assumptions, risk factors, and methods that FSIS used to rank the relative public-health risks posed by the various types of in-commerce businesses that handle meat, poultry, and egg products and to review and comment on the agency's preliminary risk rankings of various business types. In its letter report, the I-C I committee presented general comments and findings and made general and specific recommendations to help in the setting of priorities for surveillance 

LETTER REPORT 5 activities by the CID. Briefly, the I-C I committee agreed that a risk-based approach for surveillance of in-commerce establishments is appropriate but suggested that some of the 13 business types identified by FSIS could be separated into more precise categories because of the great diversity in them. The I-C I committee had difficulty in understanding some of the five risk factors or considerations, particularly inherent risk, used by FSIS in categorizing the business types into tiers for surveillance priority-setting and recommended that they all be weighted to reflect a risk-based approach better. In particular, the I-C I committee ranked whether there is surveillance by other authorities as the primary risk consideration and recommended that FSIS partner with state and local regulatory agencies to increase its knowledge of existing meat, poultry, and egg product surveillance so that it could improve the CID’s priority-setting. The I-C I committee recommended a stepwise decision process (Appendix D) for setting priorities for surveillance of individual establishments, as opposed to business types, instead of the FSIS-proposed three-tier system. According to the I-C I committee’s proposal, surveillance by other authorities would constitute the main selection step of the five FSIS risk considerations for surveillance priority-setting. Facilities that were receiving vigorous inspection by other federal agencies or by state or local agencies would be considered as having low priority for FSIS surveillance; typical examples are restaurants, institutions, and retailers. For establishments that warranted FSIS surveillance, product volume would be the most important criterion in setting priorities because businesses that handle large amounts of product can have a higher impact on the public than businesses that handle small amounts. Examples of high-volume businesses are warehouses, transporters, and distributors. Next, with less weight but still important, the investigator would consider inherent hazard (restaurants, retailers, and institutions may rank high) and consumer susceptibility (certain institutions rank high), followed by food- defense vulnerability issues (transporters, institutions, retailers, restaurants, food banks, and rendering plants) as criteria in selecting establishments that warrant surveillance. In this scenario, most facilities (e.g., the approximately 500,000 restaurants, 120,000 retailers, and 55,000 institutions) would not be inspected by the CID except “for cause.” Thus, it was suggested that FSIS create a single category of high-risk businesses over which it has sole jurisdiction and that most (potentially 90%) of the CID’s available resources be devoted to surveillance of such businesses—including renderers, 4-D businesses, and animal-food businesses—to provide assurance that their products do not enter the human food supply. If other jurisdictions do not inspect custom-exempt and exempt-poultry businesses, they could be included in CID activities. The I-C I committee also recommended that the CID use an initial period of activity of at least 1 year to collect data, develop collaborations with other jurisdictions, and benchmark where surveillance activities should take place before establishing a specific allocation of time for these activities so that risk ranking may be modified appropriately as specific knowledge and data became available. Food Safety and Inspection Service Response to the Report of the In-Commerce I Committee As illustrated in its draft revision of the I-C I committee’s stepwise decision process diagram (Appendix E), FSIS accepted the recommendation that oversight by other regulatory authorities should be used as the primary risk consideration in determining the allocation of the 10% of CID investigators’ time that they have available for surveillance activities. Other regulatory authorities include the Food and Drug Administration (FDA); the FSIS Office of Field 

6 REVIEW OF IN-COMMERCE SURVEILLANCE Operations; and state, county, and municipal agencies. FSIS also accepted the recommendation that product volume, inherent hazard, consumer susceptibility, and food-defense vulnerability be used as secondary risk considerations for the allocation of time. FSIS, however, seems to have given equal weight to each of the four secondary risk considerations instead of assigning relative importance to them as recommended by the I-C I committee. The I-C I committee’s proposed stepwise decision process involved setting two priority levels for FSIS surveillance: high and low. FSIS renamed them Tier 1 and Tier 2 to be consistent with its previous terminology. The Tier 1 business types (as defined by FSIS)—transporters, distributors, and warehouses—would occupy 80–90% of investigators’ surveillance time. The Tier 2 business types—food banks, 4-D, retail salvage, renderers, exempt poultry, and animal food—would occupy the remaining 10–20% of the investigators’ surveillance time. FSIS removed four business types—institutions, retailers, restaurants, and custom exempt—from the 10% time available for surveillance because the agency believes that they receive significant oversight by other regulatory authorities. The I-C II committee does not believe that the revised FSIS decision process adequately uses the “oversight by other regulatory authorities” risk consideration to rank businesses. FSIS may have used product volume and perhaps other secondary risk considerations in selecting its Tier 1 business types. Or perhaps it inadvertently carried over the Tier 1 business types from earlier documents; they are identical with those presented to the I-C I committee as Tier I. The I-C I committee had determined in its report that 4-D facilities, renderers, and animal-food business types should be given the highest priority for CID surveillance because they receive little or no surveillance by any other regulatory authority. It seems unlikely that in FSIS’s revised decision process those business types will receive any CID surveillance, because so little time is available for surveillance of business types in the proposed Tier 2. FSIS also informed the I-C II committee that it would continue to conduct surveillance activities on a “for cause” basis. FSIS stated that it would include restaurants, retailers, custom- exempt establishments, and institutions in its for-cause surveillance category and that these businesses would not be part of risk-based initial surveillance. Those businesses would receive for-cause visits for activities such as sampling programs that are already carried out by FSIS to assess compliance, product safety, and other regulatory determinations. Examples include FSIS programs for testing for melamine and Escherichia coli O157:H7 in in-commerce products. A CID investigator who finds products that are out of compliance may initiate an investigation, conduct followup activities as outlined in the appropriate surveillance program (Directive 5500.24 and Directive 8080.15), and/or share information with other federal, state, and local public-health officials. The present committee was informed that FSIS does not automatically conduct initial surveillance activities during for-cause visits unless evidence suggests that they are needed. For-cause surveillance and associated followup are outside this committee’s task. Although the PowerPoint presentation by FSIS staff to the committee indicated that for-cause surveillance may be part of the approximately 10% of CID investigators’ available for in- commerce surveillance (Appendix E), FSIS personnel explained during a telephone conference call with a National Academies staff member on July 8, 2009, that for-cause surveillance is part of their regular duties (90% time). 4 FSIS Directive 5500.2, Rev. 3, Significant Incident Response. http://www.fsis.usda.gov/OPPDE/rdad/FSISDirectives/5500.2Rev3.pdf. Accessed July 31, 2009. 5 FSIS Directive 8080.1, Rev. 5, Recall of Meat and Poultry Products. http://www.fsis.usda.gov/OPPDE/rdad/FSISDirectives/8080.1Rev5.pdf. Accessed July 31, 2009. 

LETTER REPORT 7 For all other surveillance activities, a CID investigator will determine whether a business is in compliance on the basis of a set of factors outlined by FSIS Directive 8010.1, Revision 2. If initial surveillance identifies an alleged or apparent violation of FSIS statutory and regulatory requirements, CID investigators will conduct an investigation. If a CID investigator’s findings do not result in an investigation but are still of concern, the investigator may conduct a product- control action or schedule followup surveillance at the business in question. CID investigators will use FSIS Directive 8010.1, Revision 2, to determine whether followup surveillance is necessary and how quickly to schedule it. Such followup surveillance is the subject of the present committee’s report. OVERVIEW OF FOOD SAFETY AND INSPECTION SERVICE METHODOLOGY FOR IN-COMMERCE SURVEILLANCE (BOTH INITIAL AND FOLLOWUP) FSIS Directive 8010.1, Revision 2, outlines (but does not specifically define) how investigators should set priorities for their activities in order of public-health importance and accounting for public-health tiers and other information present in the ICS. It also states that an investigator may contact other investigators at FSIS or other federal, state, or local agencies with knowledge of a business in question and may offer representatives of such agencies an opportunity to participate in the surveillance activity. In-commerce surveillance involves, but is not limited to, activities and observations related to 1. Food safety–CID investigators should verify that meat, poultry, and egg products (hereinafter referred to as products) are wholesome and not adulterated; sanitary conditions are such as to prevent contamination of products; hazard controls are adequate to prevent products from becoming adulterated; products not intended for human consumption are denatured or made inedible; and all appropriate records are kept and maintained. FSIS provides a series of general questions (see Appendix F) that can be used by an investigator to determine whether product has been adulterated, held under insanitary conditions, or violates hazard controls. 2. Food defense–CID investigators should verify that FSIS-regulated products are secure from threats and intentional contamination. 3. Non–food-safety consumer protection–CID investigators should verify that products are not misbranded, economically adulterated, or otherwise unacceptable for reasons that do not raise a food-safety concern. Misbranding can also be a public-health concern or a food-safety issue.6 FSIS provides a set of questions to help investigators to determine whether meat, poultry, or egg products are correctly marked, labeled, and packaged without being misbranded (Appendix F). 4. Order verification–CID investigators should verify whether persons or firms are in compliance with the terms and conditions of any applicable administrative orders, court orders, settlements, or other binding case-disposition terms (e.g., administrative 6 It should be noted that some instances of misbranding can be of public-health concern (e.g., presence of an allergen not noted on the label) and thus be considered food-safety issues. Such issues would require an investigation and are outside the scope of this review. 

8 REVIEW OF IN-COMMERCE SURVEILLANCE consent decisions, consent decrees, injunctions, and plea agreements). Order verification not related to food safety is not mentioned. 5. Imported products–CID investigators must ensure that imported products meet the same standards as domestic products. Questions in Appendix F are also used to help investigators to determine potential problems related to imported products. 6. Public-health response–CID investigators conduct activities associated with recalls (Directive 8080.1) related to consumer complaints and reports of foodborne illness (Directive 8080.37). 7. Emergency response–CID investigators conduct activities associated with nonroutine emergencies resulting from intentional and nonintentional contamination affecting meat, poultry, and egg products (Directive 5500.2). According to Directive 8010.1, Revision 2, the activities briefly described above are conducted as a whole and are not independent or exclusive of one another. FSIS investigators have the authority to examine facilities (inner and outer perimeters), check products, review records, and take samples of the inventory. The directive also states that followup surveillance activities should be conducted to verify whether meat, poultry, and egg products prepared, stored, transported, sold, or offered for sale or transportation are safe, wholesome, and correctly labeled and packaged and whether persons and firms are in compliance with FSIS statutory and regulatory requirements, applicable administrative orders, and other regulatory requirements. However, the directive does not define clearly the criteria and categories used to determine when followup surveillance may be needed. CID investigators must specify in the ICS whether followup is necessary and, if so, the period within which followup should be conducted. Broad guidelines to determine the period for followup (see Appendix G) are based on the type of establishment and the tier in the ICS; whether the business in question is operating under compliance; surveillance review findings; apparent violations of the Federal Meat Inspection Act, the Poultry Products Inspection Act, or the Egg Products Inspection Act; whether an investigator initiated a product-control action; and the history of the business’s compliance. According to FSIS Directive 8010.1 Revision 2, “Generally, Investigators conduct followup surveillance reviews, when necessary, within a period of 3-, 6- or 12-months.” Therefore, according to the directive (8010.1 Revision 2), “investigators generally are to complete the followup surveillance review within a period of 3-months from the date of the reminder (i.e., 3-6 months, 6-9 months, or 12-15 months).”. The committee notes that reminders should be issued in a timely manner and with a specific due date that would meet the timing in the directive. For example, a high priority reminder could be issued upon receipt of the report, with timing adjusted to achieve a visit within 3 months. OVERVIEW OF THE IN-COMMERCE COMPUTER SYSTEM In presentations to the committee, FSIS personnel emphasized that a key component of its target risk-based surveillance of in-commerce businesses is the development and implementation of a computer database, the ICS. When completed, the ICS will contain 7 FSIS Directive 8080.3, Foodborne Illness Investigations. http://www.fsis.usda.gov/OPPDE/rdad/FSISDirectives/8080.3.pdf. Accessed July 31, 2009. 

LETTER REPORT 9 information about all in-commerce businesses subject to FSIS surveillance. Information about product-control actions, investigations, and enforcement actions is also entered into the ICS. During their initial visit to an in-commerce business, CID personnel review and update information about the business. At the conclusion of a visit, information about whether a followup is needed and, if so, the date by when it should take place is entered. As indicated, a key feature of this important and useful ICS is that it will generate reminders to investigators to conduct followup surveillance and that investigators are instructed to complete the activity within 3 months of the reminder. FOOD SAFETY AND INSPECTION SERVICE PROPOSED FRAMEWORK FOR PRIORITY-SETTING FOR FOLLOWUP SURVEILLANCE OF IN-COMMERCE BUSINESSES In a PowerPoint presentation to the committee, FSIS staff identified five potential findings determined during initial surveillance of in-commerce businesses—no findings, food- defense–related deficiencies, non–food-safety consumer-protection issues, food-safety problems, and product-control action needed. As noted previously, CID investigators can also initiate investigations in response to particular findings. FSIS assigned high, medium, or low priority to the potential findings to guide CID investigators in followup surveillance activities, as noted in Table 1 (adapted from the presentation by FSIS). TABLE 1 Initial Surveillance Findings and Priorities for Followup Surveillance Tier 1 2 Business Type Transporters, Food Banks, 4-D, Retail Distributors, and Salvage, Renderers, Warehouses Exempt Poultry, and Animal Feed Initial Surveillance Findinga Followup Surveillance Priority No findings Low Low Food-defense finding Medium Low Non–food-safety consumer- Medium Low protection finding Food-safety finding High Medium Product-control action High Medium Investigation initiated As outlined in FSIS As outlined in FSIS Directive 8010.28 Directive 8010.2 a Derived from Directive 8010.1, Revision 2. 8 FSIS Directive 8010.2, Rev. 2, Investigative Methodology. http://www.fsis.usda.gov/OPPDE/rdad/FSISDirectives/8010.2Rev2.pdf. Accessed July 31, 2009. 

10 REVIEW OF IN-COMMERCE SURVEILLANCE The followup surveillance priorities presented in the table are detailed below: 1. No findings.—A designation of low priority for followup has been assigned for both Tier 1 and Tier 2 businesses. 2. Food-defense finding—Food-defense activities have no FSIS regulatory requirements, but a relevant guidance has been issued to the industry for transportation, distribution, and warehousing. Some findings, such as deliberate tampering, could trigger an investigation and/or a significant-incident response and would not be part of the followup surveillance priority-setting framework. 3. Non–food-safety consumer-protection finding—Non–food-safety consumer-protection activities are not generally related to food safety and involve products that are misbranded, economically adulterated, or otherwise unacceptable. Some misbrandings (such as the presence of an allergen not listed on the label) can cause food-safety concerns and thus be raised to food-safety findings, and others (such as violation of the Federal Meat Inspection Act or applicable court orders) can trigger investigations and so would not be part of the followup surveillance priority-setting framework. 4. Food-safety finding—Food-safety findings include product adulteration, diversion of inedible products to human food, insanitary conditions, inadequate hazard controls, and inadequate recordkeeping. As for non–food-safety findings, certain findings in this category would trigger investigations and/or recalls and therefore would not be part of the followup surveillance priority-setting framework. 5. Product-control action—Product-recall actions are outlined in FSIS Directive 8410.1, Detention and Seizure. Most product-control actions result in voluntary action by the product owners or custodians, such as voluntary disposal of the product. If a detained product is not disposed of within 20 days, FSIS may request an order of seizure. COMMITTEE RESPONSE TO METHODOLOGY PROPOSED BY THE FOOD SAFETY AND INSPECTION SERVICE FOR FOLLOWUP SURVEILLANCE The proposal that FSIS gave the committee in PowerPoint presentations on July 6, 2009 (Appendix C) seemed incomplete and appeared to lack important information about priority- setting for followup surveillance, and this limited the committee’s capacity to comment on its merits. The committee determined that any strategy to revisit facilities is influenced by FSIS’s strategy for conducting initial in-commerce surveillance and by the findings of such activity. It appeared that FSIS had not addressed the crucial question of how initial surveillance and followup surveillance would be related to each other, although the two would clearly be interlinked in reasons, goals, and objectives. The basis on which to set priorities for initial and followup surveillance (i.e., how much of CID investigators’ time should be devoted to each) is important, but it was not part of the task given to the committee. FSIS has not directly labeled followup surveillance activities as risk-based. The committee believes that this is appropriate and suggests that FSIS refrain from calling them risk- based because in their present form they do not fully meet the criteria for such a designation, although that should be its future goal. 

LETTER REPORT 11 FSIS has provided, in Directive 8010.1, Revision 2, instructions to field staff in the form of questions that they should answer when visiting facilities and instructions for followup visits. The committee raised a number of questions about the general lack of definitions in the FSIS proposal. More sharply defined instructions and definitions for assessing establishments are likely to result in data that will facilitate more consistent enforcement actions, help to inform strategies for future inspections, and help in the evaluation of the impact of surveillance in the long run. Providing specific triggers or a decision tree could result in a clearer determination of the types of findings that would warrant followup. Followup surveillance should then be defined on the basis of public-health considerations. If the latter considerations are taken into account, a followup period of 3–15 months seems too long. The committee identified four possible outcomes of an initial visit: 1) no followup, 2) an investigation (which would be due primarily to significant food-safety problems), 3) a routine followup, and 4) a request or suggestion to state or local authorities that they follow up. Following upon the recommendation of the IC-I committee for closer collaboration and cooperation between FSIS and local jurisdictions, this committee suggests that in some cases it might be appropriate for FSIS to work with state and local agencies and, if feasible, to request that they conduct followup surveillance. In such cases, the findings in followup surveillance would need to be transmitted back to FSIS in a standard format so that they could be entered into the ICS. Such cooperative efforts would be beneficial to both sides, and contribute to better in- commerce surveillance. The committee reminds FSIS that not all state and local regulatory agencies have the same resources; the type and extent of surveillance often varies within and between states. State and local resources for food-safety inspections and compliance activities are often subject to budgetary cuts,9 which affect their ability to conduct food safety activities. Furthermore, some state and local jurisdictions have been unable to perform required food inspections.10 Consequently, it is important for FSIS to develop increased communication and data-sharing with state and local regulatory agencies to improve its understanding of their surveillance activities and to ensure that data collected at the state and local levels can be used. State and local surveillance or inspection capacities should be assessed periodically. As FDA has a program for assisting in the development of the capacities of state and local jurisdictions,11 it may be valuable for FSIS to look into that effort as it considers its own assessment of local programs. These suggestions agree with recommendation made by the I-C I committee. The present committee believes that additional data collection is an essential component of both initial and followup surveillance for the desired risk-based system to be developed. Additional data can help FSIS to make such decisions as which facilities should receive initial visits, which facilities should receive followup surveillance, and when FSIS should suggest initial or followup visits by state or local authorities. To assist it in making the first two of those, FSIS needs information about the public-health significance of potential problems. It is important to know the amounts of specific products handled at a business site and whether there are any chemical, physical, and/or biological hazards associated with the products that might result in violations. To determine whether a followup visit is warranted, FSIS would need to make a judgment about the likelihood that a business will correct a problem without further 9 http://www.tribune-democrat.com/local/local_story_025232154.html. Accessed January 29, 2009. 10 http://www.oppaga.state.fl.us/reports/pdf/0867rpt.pdf. Accessed January 29, 2009. 11 http://www.fda.gov/downloads/RegulatoryInformation/Guidances/UCM125448.pdf; http://www.fda.gov/Food/FoodSafety/RetailFoodProtection/ProgramStandards/ucm124968.htm. 

12 REVIEW OF IN-COMMERCE SURVEILLANCE inspections. FSIS may want to examine specific characteristics of a business and its management. Initially, it may rely on expert judgment of current inspectors and state and local food regulatory agencies for this type of information by using expert elicitation; ultimately, more sophisticated primary data may be available to fill this data gap. FSIS should train its personnel to collect specific, consistent and valid data that appropriately target the risk factors of in-commerce establishments. It can consider more objective surveillance forms, such as forms used by FDA12 and other inspection agencies that were developed on the basis of risk-factor studies. The ICS should be used to generate data that can be analyzed for trends over time and the data would help to inform decisions for routine or followup visits and help to validate the ICS in the long run. The data generated should include the probability of finding a violation on the first visit and the severity of resulting hazards. A low likelihood of finding a severe violation (and hazard) might indicate that only infrequent followup is needed; if the likelihood of finding violations on a first visit is greater (e.g., one of three facilities instead of one of 100), FSIS should focus its resources on those facilities. Findings 1. The committee determined that FSIS agreed with the In-Commerce I committee’s recommendation that oversight by other regulatory authorities should be the primary risk consideration in determining surveillance activities. 2. In maintaining the business types that it had originally placed in Tier 1, the draft framework for followup surveillance by FSIS seems not to use oversight by other regulatory authorities (except for retail, institutions, restaurants, and custom exempt) as the primary risk consideration, as recommended by the In-Commerce I committee. 3. FSIS Directive 8010.1, Revision 2, is to be used to conduct both initial and followup surveillance. However, the directive contains large amounts of subjective material, and several elements are unclear: a. The directive does not clearly delineate the differences between initial and followup surveillance. b. The directive does not clearly define what would constitute a relevant public-health finding (e.g., pass–fail or compliance–noncompliance with any of the questions) and how identification of a problem would affect the surveillance outcome (e.g., triggering an investigation, an enforcement action, a product-control action, or a followup activity). c. Food-safety and non–food-safety consumer-protection–related questions that are provided for guidance to CID investigators in determining compliance or non- compliance seem broad and answers are left too much to investigators’ judgment. Questions that are more objective and precise are likely to result in data that will facilitate more consistent enforcement actions and are likely to yield information that will be helpful in validating the proposed system and inform strategies for future surveillance. 12 FDA 2005 Food Code, Model Forms, Guides, and Other Aids. http://www.fda.gov/downloads/Food/FoodSafety/RetailFoodProtection/FoodCode/FoodCode2005/ucm124043.pdf. Accessed August, 13, 2009. 

LETTER REPORT 13 d. The directive does not provide clear, objective guidelines on how to determine whether followup surveillance is necessary or how soon it should be scheduled. For example, it would be useful to provide investigators with a decision tree or an otherwise structured and reproducible approach to determine the need for followup surveillance. It seems to the committee that a period of 3–15 months for followup is subjective and may be too long in some instances. The committee believes that the public-health considerations that the agency applies to establish followup timeframes should be stated in the directive. 4. Although details of the types of information entered into the ICS were not shared during presentations by FSIS, the committee believes that the development of this computer database is an important accomplishment and commends FSIS for the effort. As both the quantity and quality of the information in the ICS increase, FSIS will be able to make more reasoned and risk-based judgments about how to conduct followup surveillance and will be able to validate and evaluate its system. 5. The initial-surveillance finding category of “no findings” has not been clearly defined, but it was given a ranking of “low priority”, which is the same designation as that for a food- defense finding or a non–food-safety consumer-protection finding in a Tier 2 business. It is difficult to understand why an absence of a finding during an initial surveillance would warrant the same priority for followup as a finding during an initial surveillance. 6. FSIS has developed draft priority rankings of high, medium, and low for surveillance findings in Tiers 1 and 2.The criteria that define or distinguish the rankings have not been clearly explained. It is unclear how the rankings were assigned or why different priority levels were assigned for the same finding between tiers. For example, a food-safety finding in Tier 1 is given a “high” priority, whereas a food-safety finding in Tier 2 is given a “medium” priority. Clarification of how the rankings were determined would be helpful. 7. Based on the material provided to the committee, it is unclear what situation would create a need for a product-control action and what the result of such an action would be in terms of followup surveillance. 8. No followup of initial surveillance conducted during for-cause visits is identified; the committee assumes that priority-setting for such followup would follow the same scheme as in Table 1. 9. It is unclear how FSIS determines the relative proportions of CID investigators’ time to be spent on initial and followup surveillance and how the two proportions are related to one another. That is a key element of in-commerce surveillance, but it was not part of the task assigned to the committee. Recommendations The committee offers the following recommendations to FSIS: 1. Continue to explore cooperative arrangements with other regulatory agencies to provide assistance in both initial and followup surveillance. If followup surveillance is needed, it may be more efficient and effective for CID investigators to contact federal, state, or local agencies to take over the activity with or without CID involvement. Other agencies should have vigorous regulatory programs, and relevant data must be shared between the agencies and FSIS and must follow FSIS standards to allow data to be 

14 REVIEW OF IN-COMMERCE SURVEILLANCE entered into the ICS. As indicated in the In-Commerce I report, this committee re- emphasizes the need to avoid duplicative and redundant inspection efforts, making sure, however, there is adequate surveillance in all situations. 2. When FSIS establishes its final framework it should revise the business types in the proposed Tier 1 to be in accord with the recommendations of the In-Commerce I committee or, alternatively, describe in detail the rationale for its decisions.. 3. In order to reduce subjectivity in CID surveillance activities and make results of surveillance as objective and consistent as possible, FSIS should provide greater clarity and definition in Directive 8010.1, Revision 2. This may be accomplished, to the extent possible, by providing better-defined criteria and potentially by using better-designed inspection/surveillance forms. Furthermore, FSIS should clarify and define the differences between initial surveillance and followup surveillance, the types of findings of initial surveillance that should result in followup surveillance, and the guidelines for determining the period for followup. 4. Risk-based initial and followup surveillance should be based on data that need to be collected for the ICS. Based on surveillance objectives, type of business, and the risk considerations used by FSIS in determining surveillance priorities, consider carefully the types of data that need to be captured during surveillance and recorded in the ICS because these data will be important in deciding whether to follow up with a second visit or more and will be used to validate and evaluate the impact of surveillance activities in the long run. To the extent possible, quantitative, rather than qualitative, information should be recorded. Consider developing more objective inspection/surveillance forms that target the relevant risk factors for in-commerce establishments, such as those used by FDA or other agencies. In addition, having high quality data and applying trend analysis may reveal gaps in policies or regulations for certain segments of the in-commerce industries. Closing the gaps could lead to improved consumer protection through more complete regulatory policies and enforcement along the total food chain. 5. Collect data to enable the agency to make more informed initial and followup surveillance decisions. Priority-setting should be data-driven and evidence-based so that plans can be risk-based. Initially, FSIS might use expert elicitation to capture knowledge of current CID personnel about, for example, the likelihood of compliance. FSIS may also be able to obtain data from other organizations about past inspections, at least of facilities in areas of the country that have rigorous local inspection programs. 6. Identify more clearly the response when “no findings” are reported during an initial surveillance. For example, the response might be “no followup activity” or a designation for less frequent surveillance. 7. Clearly define the ranking categories of high, medium, and low priority and how they are distinguished from one another. That will make the framework more reproducible and more systematic. Reconsider whether a food-safety finding should differ between Tier 1 and Tier 2 businesses. For setting priorities for followup surveillance, explain how the characteristics of the business (e.g., the complexity of processes at the establishment and the population served) are accounted for in conjunction with the findings of initial surveillance. 8. Consider re-evaluation of organizational procedures and structures within FSIS (surveillance resource deployment) to make better use of CID staff time. One 

LETTER REPORT 15 consideration may be the allocation of time for in-commerce surveillance activities by CID personnel. For example, if feasible, it may be more efficient and effective to use 100% of the time of 12 employees for in-commerce surveillance activities rather than the proposed 10% of the time of 120 employees. Employees dedicated full-time to surveillance activities could be trained to understand the high-priority businesses where they are most needed. That could facilitate collaboration with other jurisdictions. FSIS should state more clearly how it uses risk-based priorities when determining how to use the time set aside for in-commerce surveillance, both initial and followup. 9. Establish a plan to evaluate the impact of followup surveillance to determine whether establishments are improving as a result of followup surveillance (e.g., as shown by a reduction in violations or in noncompliance) on the basis of data captured in the ICS. FSIS should develop a strategy to evaluate and validate the ICS system (for both initial and followup surveillance). 

APPENDIX A Statement of Task An ad hoc committee will examine methods proposed by Food Safety Inspection Service (FSIS) to schedule followup surveillance of in-commerce establishments using a risk-based approach. The committee will review and comment on the assumptions, risk factors, and methodology FSIS proposes to use to prioritize followup surveillance at in-commerce business with prior surveillance history. The committee will also review and comment on the agency's proposal for determinants that would indicate the need for followup surveillance and review and comment on the proposed frequency of followup surveillance activities. A letter report will be issued. 16 

APPENDIX B Meeting Agenda Committee on Review of the Methodology Proposed by the Food Safety and Inspection Service for Followup Surveillance of In-Commerce Businesses Board on Agriculture and Natural Resources National Research Council The Keck Center of the National Academies (Room 204) 500 Fifth Street, NW Washington, DC 20001 Monday, July 6 Open Session – 2:00 p.m. to 5:30 p.m. 2:00 p.m. FSIS Presentations Donald Anderson and Matthew Gonzales (Office of Program Evaluation, Enforcement and Review, Food Safety Inspection Service, USDA) 3:00 p.m. Industry Surveillance, Regulation, and Perspective on In-Commerce Activities Larry Kohl (Senior Director of Food Safety Programs, Food Marketing Institute) 3:30 p.m. Association of Food and Drug Officials (AFDO) Surveillance, Regulation, and Perspectives on In-Commerce Activities Joseph Corby (Executive Director, AFDO) 4:00 p.m. FDA Surveillance, Regulation, and Perspectives on In-Commerce Activities Kara Morgan (Senior Advisor for Risk Analysis, Office of Regulatory Affairs, FDA) 4:30 p.m. Public Comment Period 5:30 p.m. Adjourn Open Session 17 

APPENDIX C Presentations by Donald Anderson and Matthew Gonzales of FSIS Public Health Risk-Based In-Commerce “Initial” and Follow-up Surveillance Presented to the National Academy of Sciences July 6, 2009 Today’s Topics • Review the “initial surveillance” methodology we are implementing based-- in-part-- on the prior committee’s recommendations • Ask NAS to comment on preliminary ideas for prioritizing follow-up surveillance activities 2 18 

LETTER REPORT 19 In-Commerce System Facility Types - Distributors -Transporters -Warehouses -Retailers OPEER Investigators -Others . ICS Surveillance Tier Initial Surveillance Tier 1. Critical Priority Tier 2. Very Important Priority Tier 3. Important Priority Risk Considerations - Inherent Risk -Consumer Susceptibility -Volume -Food Defense Vulnerability Reports -Jurisdiction Business Types Transporters, Distributors, 8,262 16,754 Other, 4,275 Warehouses, 23,625 Institutions, 54,621 Retailers, 120,249 Restaurants, 500,000 4 

20 REVIEW OF IN-COMMERCE SURVEILLANCE “Other” Expanded Exempt Poultry, 100 4-D, 64 Retail Salvage, 162 Custom Slaughter, 170 Renderer, 383 Food Banks, 642 Animal Food, 2,754 5 Tier Results for 13 Business Types Tier 1 (Critical) Tier 2 (Very Important) Tier 3 (Important) y od ge s rs ks s ts rs ns m rs D tr se or 4- an to te re le an va Fo tio ul ut ou or ai us de ur Po B al itu ib et al sp eh C ta en lS od tr m R st pt es an ar is R ai ni Fo In em D W R et Tr A R Ex 6 

LETTER REPORT 21 Previous NAS Review • November 2008 public meeting • March 23 NAS report 7 Risk Considerations • Volume of meat, poultry and egg products handled • Susceptibility to Foodborne Illness of the populations served • Inherent hazards of the products handled and processes engaged in • Food Defense Vulnerability of the operations & • Extent of Surveillance by other Federal, State, or Local authorities 8