Click for next page ( 54


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 53
4 Public Release of Food Safety and Inspection Service Establishment-Specific Data The release of establishment-specific Food Safety and Inspective Service (FSIS) data would provide public access to detailed sampling and testing data and inspection and enforcement data. As discussed in Chapter 2, those data, with few exceptions, have been available to the public only in aggregated form without establishment-level detail. Under the “right to know” principles of the Freedom of Information Act (FOIA), the government is obliged to provide data solicited by the public except in particular cases (such as personal or medical information and trade secrets; see Chapter 2, Box 2-1). Thus, many of the establishment-specific data that FSIS might release are available through FOIA requests. However, the availability of the data through such requests is limited by the request process and a requester’s exclusive use of the data unless the requester chooses to share them. Thus, public release of data by posting on the Internet would result in a fundamentally changed information environment, including more information and potentially more users and uses. The experience of other federal agencies that have posted detailed data (reviewed in Chapter 3) suggests that there may be benefits, as well as some potentially adverse unintended consequences, of posting establishment-specific data that FSIS collects as part of its regulatory mission. The conclusions of Chapter 3 also suggest that the benefits can grow and that the concerns stemming from adverse consequences can be mitigated through careful design of data release. That implies the need for a strategic plan designed to guide the agency in its choice of data to release, how to release them, and the means by which to ensure that data are continuously updated and improved. The purpose of this chapter is three-fold: to discuss the potential benefits and adverse consequences of releasing establishment-specific FSIS data; to present issues related to data release that FSIS may want to consider during the development of a strategic data-release plan, including approaches to measuring the public-health and other relevant effects of data release; and to present the committee’s findings and conclusions regarding the public release of establishment-specific FSIS data. 53

OCR for page 53
POSSIBLE BENEFITS, COSTS, AND UNINTENDED ADVERSE CONSEQUENCES The committee identified a number of favorable outcomes that might be anticipated as a consequence of the public release of FSIS establishment-specific data. At the most basic level, such release would directly serve the first broad purpose of transparency in supporting the public’s right to know. It would also serve the second broad purpose of transparency in helping to achieve specific public-policy goals. In the latter role of “targeted transparency”, the major effects of expanded data access would include the potential for better decision-making based on improved information and stronger incentives for both the agency and food companies to improve their performance. Although it has not been definitively documented, one of the expected advantages of providing public access to establishment-specific FSIS data is the improvement of public health. That appears to have been the case for the Environmental Protection Agency’s release of Toxics Release Inventory data and the publication of restaurant-inspection data (see Chapter 3). Releasing establishment-specific FSIS data could potentially motivate individual companies and sectors of the food industry to improve their overall food safety efforts. For example, data release could provide incentives to protect brand reputation in food safety and to protect or enhance customer base and profitability; allow downstream purchasers and consumers or public- interest organizations to identify companies whose performance records were consistently above or below the industry average and potentially create economic pressure to improve food safety; provide better insights into strengths and weaknesses of different processing practices, which could lead to industrywide improvements in food safety practices; enhance performance benchmarking by individual companies, sectors, and the industry as a whole, including efforts by individual companies that are seeking to avoid being identified as “below average”; and improve the consistency of inspector performance. Even if individual firms do not change their behavior in response to data posting, overall food safety could improve if information about performance leads consumers to favor high- performing establishments and hence causes a shift in the composition of the market. In addition to providing incentives for the private sector, release of establishment-specific data could help to identify needs for improvement in regulatory practices, and this might result in activities that lead to improved public-health outcomes. For example, industry representatives raised concerns about variation in enforcement practices among inspectors and districts. Analysis of enforcement data could help to identify variability in enforcement outcomes (if present) of comparable facilities. It could also help to identify effective practices in regulated facilities that could be more broadly adopted. As in other systems that provide establishment-level enforcement data, a coding system could be developed to protect the identities of individual inspectors and still achieve the above outcomes. Public release of establishment-specific FSIS data, by themselves or in combination with other privately or publicly available data, could yield valuable insights that go well beyond the regulatory uses for which the data were collected. For example, establishment-specific FSIS microbial testing data might be combined with region-specific climate data in an effort to develop better predictive risk models of pathogen load as a function of environmental conditions. Making establishment-specific data publicly available might provide information that would be useful for training the next generation of researchers, regulators, and industry food-safety 54

OCR for page 53
experts. The ability to analyze establishment-specific data would probably create a network of third-party analysts who, because of their familiarity with the data and their structure, could help FSIS to mine its own data and help individual companies or industry sectors to use the data to improve their practices. By publicly releasing establishment-level data, FSIS would be sharing with stakeholders, particularly those in the academic and industrial sectors, the opportunity to perform data analysis. Those groups may be able to use the data in conjunction with other sources of information to yield new insights or conclusions that could have significance for food safety and public health. Public release of FSIS establishment-specific data could also lead to improved public understanding of the considerable efforts made by FSIS and the industry to ensure food safety. For example, if data release could be linked to specific improvements in food safety, it might promote public perceptions of and confidence in the safety and integrity of the food supply and in the companies and regulatory agencies that are responsible for ensuring them. The benefits of releasing establishment-specific FSIS data must be balanced against the potential unintended adverse consequences. Several of those were noted by industry representatives who spoke to the committee in the open session of its first meeting. For example, there was concern about the potential for misinterpretation of data. FSIS data are complex, and appropriate analysis of them would require considerable training and skill in statistical analysis. Making sense of the data also requires knowledge and experience to put them into an appropriate context. Without such knowledge and experience, users could misinterpret the data, reach unwarranted conclusions, or take the data out of context in an effort to support predetermined positions. For example, FSIS publishes the results of microbiological sampling of ready-to-eat (RTE) meat and poultry products. 40 In discussing those data, FSIS (USDA/FSIS, 2011) stated that the agency “does not view the results of regulatory testing as estimates of national product prevalence”. However, the data are often misused by the industry, mass media, and other organizations as the basis for calculating pathogen prevalence in products. Adverse effects on brand reputation could also occur as a consequence of public release of establishment-specific data. It is possible that those effects will be experienced differently and as a function of organization size. Larger organizations with more sophisticated corporate communication functions or hired public-relations agencies will probably be able to establish clear systems to explain violations. But smaller organizations that do not have the resources to support such communication efforts might experience more lasting damage to brand reputation. In short, it may be that the smaller companies are not unwilling to talk about food safety but just do not know how to do it effectively. To minimize the potential for adverse consequences of the public release of establishment-specific data on small and very small establishments, FSIS could provide adequate documentation and explanation of both a noted deficiency and the possible outcomes of such a deficiency. Industry representatives were also concerned that releasing establishment-specific pathogen-testing data could affect international trade. For example, foreign countries might use publicly available FSIS testing data to bar entry of products from specific establishments on the grounds of presumed risks to public health. 41 In the absence of a similar requirement to release comparable data from their own countries’ companies, determining the true food-safety benefit of barring import of products from select US companies would be difficult. The World Trade 40 See “The FSIS Microbiological Testing Program for Ready-to-Eat (RTE) Meat and Poultry Products, 1990–2010” at http://www.fsis.usda.gov/science/Micro_Testing_RTE/index.asp#results05 (accessed August 17, 2011). 41 For example, foreign countries could conceivably use establishment-specific data to delist US establishments and effective eliminate some international markets for select establishments and commodities. 55

OCR for page 53
Organization would eventually decide such disputes, but the short-term economic consequences for individual firms could be substantial. The committee notes, however, that just as data on US- based establishments would be released, FSIS data collected from foreign-plant inspections would be released, as would data collected as part of FSIS imported-product testing and approval or refusal. The effects of the release of establishment-specific data on imported products and on US exports are unknown. However, if the release of data leads to improved food safety of both domestic supplies and exports, the benefits would be realized not only by US consumers but by foreign consumers. Another concern is related to the unintentional release of proprietary or confidential information. For example, Food Safety Assessments (FSAs) often correspond to specific components of an establishment’s Hazard Analysis and Critical Control Points (HACCP) plan. Hence, they may reveal details that are considered proprietary by the establishment or include sensitive food-defense information. Clearly, the unintentional release of such information would need to be guarded against, for example, by redaction of sensitive information before the release of establishment-specific data. Similarly, the agency would need to take precautions to avoid the possibility that portions of the data could be linked in ways that would allow users to deduce confidential information about particular establishments, such as FSIS inspection patterns, regulatory assignments, or sampling regimes. For example, FSIS may choose to release data on enforcement actions only after they have been completed, but real-time release of establishment-specific microbial-testing data could reveal that additional testing had been ordered for a particular establishment, which would indicate a new enforcement action. In that respect, the facility’s response to the corrective action and its resolution would also need to be released. Experience (see Chapter 3) suggests that public data release can affect inspector behavior both favorably and unfavorably. For example, the public release of information could put more FSIS pressure on inspectors to ensure the quality and consistency of their work. That could have the beneficial effect of reducing variation in enforcement procedures that does not further the agency’s mission. Or, if increased transparency of enforcement increases the stakes of their outcomes, inspectors may face more pressure from firms regarding their outcomes (as has been documented in, for example, restaurant hygiene). Finally, public release of establishment-specific FSIS data in whole or in part does not ensure that they will be useful or used. To make them so, FSIS will need to define a timetable for data release and commit the resources necessary to ensure the accessibility, quality, timeliness, and usefulness of the data. The costs could theoretically be offset, at least in part, by reduction in resources now dedicated to responding to FOIA requests. Each year, FSIS spends about 20,000 hours and over $500,000 in complying with about 500 FOIA requests.42 That cost is supposed to be compensated by external parties (requesters) 43 , but the committee learned that there are exemptions to FOIA compensation , and it appears that current compensation is well below the actual cost of providing such data (J. Reed, USDA-FSIS, Washington, DC, personal communication, July 7, 2011). Public data release might save the agency some of the time and money spent in operating the current FOIA system. However, there is a risk that an open system will trigger more in-depth FOIA requests once reporters or other interested parties begin to 42 The committee derived these estimates on the basis of the listing of FOIA requests (http://www.fsis.usda.gov/PDF/FOIA_Requests.pdf) and the information on making a FOIA request at the FSIS Web site (http://www.fsis.usda.gov/FOIA/FOIA_Request/index.asp) (accessed July 8, 2011). 43 Fees collected by FSIS for FOIA requests are sent to the US Department of the Treasury. 56

OCR for page 53
peruse the information. Another challenge in making establishment-specific FSIS data publicly available is that although FOIA requests may decrease, there may be a much larger need for public-affairs staff to handle news-media requests that might be associated with the new data and an increased need for an agency spokesperson to help the news media or the public interpret information. The above discussion shows that both benefits and unintended adverse consequences could result from providing the public with access to data on individual FSIS-regulated establishments. As discussed in Chapter 3, guidance from a carefully designed data-disclosure strategic plan, as discussed in the next section, could maximize the effectiveness and minimize the potential adverse consequences of sharing establishment-specific data. Because effective data release requires cooperation among FSIS, industry, and the stakeholders most likely to use the data, the development of the strategic plan would benefit from their input. And because data themselves evolve, as do their uses, FSIS may also want to consider the need for continuous improvements based on industry and user feedback and agency response to that feedback. Only with such communication can FSIS maximize the value of public release of establishment-level data. STRATEGIC PLANNING OF DATA RELEASE As noted above, the committee believes in the development of a strategic plan for public release of establishment-specific data. The plan would be part of a larger comprehensive strategic plan for data collection, management, and analysis. FSIS has a start on such an overall plan in its Strategic Data Analysis Plan for Domestic Inspection (USDA/FSIS, 2010), and inclusion of a public data-release component would be appropriate. Below are some key issues to be considered in developing a data-disclosure strategic plan. Identifying Potential Users Multiple parties potentially have an interest in FSIS establishment-specific data. They include consumers, the mass media, consumer groups, parties along the entire food-supply chain (such as suppliers, producers, processors, distributors, retailers, and food-service operators), third-party inspectors, researchers, and other government agencies. The parties differ in how they would use FSIS data and in how information will be embedded in their decisions (if at all). For example, it is doubtful that individual consumers would have the ability to (or even want to) sift through FSIS data to trace connections between establishment-specific inspections and choices made at the supermarket. Some third-party groups will probably have greater desire or ability to translate information into a more useful form for ultimate consumer use. Consumers in particular are inundated with information, including that having to do with food safety. Although the public’s right to know is paramount, the reality is that the vast majority of the public does not access government data on the Internet (Smith, 2010). A more likely scenario is that the data will be used by the mass media to create news stories that will then be passed on to the consumer either in traditional print format or by newer social-media channels. For food processors, retailers, and food-service operators, the data could be valuable in making sourcing decisions and managing risks associated with their supply chains. This group 57

OCR for page 53
would probably be more capable of analyzing detailed data provided by FSIS and integrating them into monitoring, sourcing, and other core business decisions. Industry groups and trade associations might play a similar role, serving as collective agents in analyzing information for members and potentially using data for education or even for self-regulation. Researchers in multiple disciplines have broad aims—shaped by disciplinary interests, academic activity, and public-policy evaluation—for gaining access to more detailed FSIS data. The research community would analyze the reliability and correlation of FSIS data to other food- safety data. Other US government agencies and international entities may also analyze and draw lessons from the released data. The criteria for choosing which datasets to make public are directly related to the potential users. The many parties that may use the data will use them in different and creative ways that agency planners themselves might not foresee. Although the committee believes that it will be difficult for FSIS to predict the full array of users and uses of the data, it also recognizes the importance of determining the utility of data for different users. The committee believes that this situation presents a strong argument for pursuing the broadest possible data release at the most disaggregated level. Users can always aggregate data for their analytic needs, but they cannot access disaggregate detail from aggregated data. Databases, Linking, and Facilitating Analysis FSIS establishment-specific data are held in a number of data tables in the Public Health Information System (PHIS) (described in Chapter 2) and in older and diverse legacy data systems (for example, the Performance Based Information System and the Automated Import Information System). The data format and categories in the PHIS and various legacy systems are not necessarily compatible. Unless that problem is addressed, users of the publicly released data might find them difficult to analyze. The desire to analyze multiple databases can result in linking problems. For example, a linking problem could occur if a user wanted to know whether FSIS administrative actions (see Tables 1, 2, 3, and 4 in Appendix C) have any relationship to FSIS food-safety adjudicatory actions (see Table 5 in Appendix C). The only way to perform that analysis now would be to extract the data from the tables manually, parse the establishment number from each field, and then manually join them in a query. Creation of a relational database format that allows linking of different datasets is likely to be the most effective means of facilitating that kind of analysis. Such a relational database would provide linking up front, saving time and reducing errors. In addition, release of the data in formats amenable to statistical analysis (such as, .xls and .csv), rather than in .pdf or text formats, would allow broader user audience. Of course, the publicly accessible database would also need to be highly secure and protected from modification or hacking. FSIS will need to address the extent to which it will provide bridges between legacy data systems and data held in the PHIS, which represent different eras and different versions of related programs. The agency will also need to provide guidance on how these datasets can be combined in a way that is both valid and useful and that does not introduce systematic errors. However, building bridges between different data systems need not necessarily be a prerequisite for data release. Different users can develop different ways to standardize or adjust data as part of their own uses of the released data. 58

OCR for page 53
Mechanics of Data Release The Internet provides a low-cost and flexible form of disclosure, and, as detailed earlier in this report, a variety of federal agencies have already developed sophisticated sites for data disclosure. The experience of other federal agencies has shown the importance of providing data to the extent possible in machine-readable formats (rather than in static forms, such as PDFs, that do not lend themselves easily to analysis) to enhance their utility to users. Posting on the Internet can also facilitate the timely release of data, but must be approached cautiously and with careful planning. On the one hand, it could be argued that there is no reason not to release data as soon as they are available. On the other hand, premature release of a dataset might mean that valuable data are missing; this would make it difficult for a user to complete a comprehensive analysis or to place the data in their proper context. FSIS will need to balance timely release with the need for completeness. The best data-sharing sites provide not only technical details but context for the variables in the datasets. Describing the methods of data collection, sources of variability, and changes in procedures that affect data consistency can be helpful for those analyzing and interpreting the data. In addition, various federal agencies and departments, such as the Department of Labor (DOL), are reaching out to potential users to assist them in developing sites and “apps” that will improve the utility of data released to the public. 44 Providing Context for Interpretation Adequate context of how data were collected and their limitations is important for the use and interpretation of released data. For example, it is well known that Escherichia coli O157:H7 prevalence fluctuates by region and season. An analyst who compares data from two establishments from which the data were collected at different times of the year or in different regions, without an appreciation of temporal and geographic influences on pathogen prevalence, might conclude that one establishment had a better ability to control the pathogen when the opposite might be true. The agency is well aware of the need for such guidance, as evidenced by statements on the page linked to its quarterly enforcement report. 45 Statements like the following are provided: “It is important to recognize that this [report] is only one aspect of the Agency's mission to protect public health through food safety.” “FSIS does not view the results of regulatory testing as estimates of national product prevalence.” “This report is a snapshot in time of a dynamic process. . . . Matters shown as under appeal may be resolved. . . . Other actions could be appealed or closed.” 44 See, for example, http://challenge.gov/Labor/201-dol-informaction-app-challenge (accessed August 7, 2011). 45 See http://www.fsis.usda.gov/regulations_&_policies/QER_Q1_FY2011/index.asp (accessed August 5, 2011). 59

OCR for page 53
While an extensive discussion of risk communication is beyond the scope of this report, and has been covered in a previous report by the National Research Council, 46 it is a vital component of the implementation and sustainability of the data-disclosure program. In particular, in the context of releasing establishment-specific data, it should be acknowledged that much of the public has a poor understanding of microbiology, microbial risks, food processing hygiene, and foodborne diseases (Hallman, 2008). Moreover, many people who are likely to be interested in understanding its meaning may have difficulty interpreting numerical data. Studies suggest that many people have difficulty grasping the magnitudes of very large and very small numbers (e.g. parts per million), and have a hard time interpreting the meanings of fractions, proportions, and probabilities (Paulos, 1988). As such, communications of risk involving mathematical operations or statistical descriptions may not be easily understood by non-expert audiences. Indeed, a variety of stakeholders will be interested in food safety, as identified in Chapter 3, and would probably value assistance in data interpretation if it were made available. To mitigate the risk of misinterpretation of data and records, it may be useful for FSIS to plan the rollout of the release of establishment-specific data in a graduated manner to help key audiences to know what to expect of the data-release program and to prepare them to interpret the data accurately. For example, the agency might wish to develop a series of recorded webinars and other formal materials that would help those visiting the Web site to understand what is being presented. The risk of misinterpretation may also be mitigated by third parties in scientific societies, academe, or independent auditing agencies. Therefore, it may be useful to identify independent third parties who are able to interpret FSIS data appropriately ahead of the rollout of an open system and to make them publicly known. E-mail alerts could also be useful in positioning information. These could be set up so that individual stakeholders could opt in to receive content or audience-specific alerts or could be sent to targeted audiences, such as key mass-media outlets, academics, industry, advocacy organizations, trade associations, and scientific societies. CONSIDERATIONS FOR RELEASE OF FOOD SAFETY AND INSPECTION SERVICE ESTABLISHMENT-SPECIFIC DATA Sampling and Testing Data FSIS routinely collects sampling and testing data on the foodborne pathogens E. coli O157:H7, Salmonella, and Listeria monocytogenes 47 and on the presence and concentration of chemicals and residues. Microbial sampling and testing data can be divided into two broad categories: those used for regulatory purposes and those used for baseline studies. As mentioned in Chapter 2, FSIS is not considering the public release of establishment-level baseline data on pathogen prevalence. Therefore, the ensuing discussion will focus on data produced for 46 The Commission on Risk Perception and Communication, Commission on Behavioral and Social Sciences and Education, Commission on Physical Sciences, Mathematics, and Resources, and National Research Council. 1989. Improving Risk Communication. Washington, DC: National Academy Press. 47 See http://www.fsis.usda.gov/Science/Microbiology/index.asp (accessed August 5, 2011). 60

OCR for page 53
regulatory purposes. A brief description of various regulatory sampling and testing databases can be found in Chapter 2, Box 2-2. FSIS now posts the data from its E. coli O157:H7 ground-beef testing program in a summarized or aggregated form. 48 The reports do not disclose the establishment unless there is a recall of finished product. Public release of establishment-level data might include the disposition of recalled and otherwise embargoed product. For example, if a product that was not in commerce tested positive for E. coli O157:H7, the report could indicate whether the product was diverted to a processing facility that fully cooked the meat or whether the raw product was destroyed, in accordance with FSIS regulations. FSIS is publishing results of completed sample sets from its Salmonella verification program for young chicken (broiler) and turkey slaughter establishments in performance Category 3; 49 this category consists of establishments whose Salmonella prevalence exceeds the performance standard. The posted data include Product Class (for example, broilers), Establishment Number, Company Name, City and State, Date of Sample Set Analysis Completion, Most Current FSIS Set Result (but only repeating the category definition, not providing exact numbers), and Previous FSIS Set Result (following the same format). To be complete, public release of those data would also include establishments in Category 1 (whose Salmonella prevalence is at or below half the performance standard) and Category 2 (whose Salmonella prevalence is above half but not over the performance standard) and the level of detail discussed above for E. coli O157:H7 testing, including sampling dates. It is also important that released data include the regulatory thresholds on which data categorizations are based. For example, effective July 1, 2011, FSIS increased the stringency of the Salmonella performance standards, highlighting the need for data disclosure to state explicitly the performance standards in effect for a particular test set. FSIS publishes highly aggregated summaries of L. monocytogenes sampling and testing of products, product-contact surfaces, and environmental surfaces. For example, an aggregated report 50 shows Number of Samples, Number of Positives, and Percent Positive for the three L. monocytogenes testing programs (ALLRTE, RTE001, and RLm). A more detailed but still aggregated report is the Percent Positive Listeria monocytogenes Tests for RTE Meat and Poultry by Product Category. 51 Again, as discussed for the two other pathogen-testing programs, the committee concluded that for establishment-specific test results for L. monocytogenes to have the greatest benefit to users outside the agency, the report would include detailed information on the testing regime and results. As part of its regulatory activities, FSIS tests meat, poultry, and processed egg products destined for human consumption for the presence of antibiotics, sulfonamides, various other drugs, pesticides, and environmental chemicals in. It also tests for the presence of such contaminants as dioxin. More information on those programs can be found at the FSIS Web site. 52 With the exception of the Residue Repeat Violator Lists (which include production facility name and address, animal type, tissue sampled, residue type, level of residue, and tolerance level that was exceeded), FSIS usually provides public access to chemical and residue testing results as reports of aggregated data, sometimes with detailed statistical analyses, 48 See http://www.fsis.usda.gov/Science/Ecoli_O157_Summary_Tables/index.asp (accessed August 5, 2011). 49 See http://www.fsis.usda.gov/PDF/Category_3_Broilers.pdf (accessed August 5, 2011). 50 See http://www.fsis.usda.gov/Science/Table24_RTE_Listeria_2009/index.asp and http://www.fsis.usda.gov/Science/micro_testing_rte/index.asp (accessed August 5, 2011). 51 See http://www.fsis.usda.gov/Science/Table22_Micro_Testing_RTE_2008/index.asp (accessed August 5, 2011). 52 See http://www.fsis.usda.gov/Science/Chemistry/index.asp (accessed August 5, 2011). 61

OCR for page 53
but with little or no access to establishment-specific or product-specific information. If FSIS decides to post establishment-specific data on chemical residues, it might want to consider posting the names of facilities without violations with the information listed above. Inspection and Enforcement Data Inspection and enforcement data include noncompliance records (NRs) and administrative actions (i.e., notices of intended enforcement or NOIEs). FSIS publicly releases these data in aggregated or summarized form in its quarterly enforcement reports. Release of these data in a more disaggregated and publicly available form might resemble that shown, for example, in Tables 5a and b; Table 7; and Table 8 of that report. 53 Since these serve as useful examples for discussion of the data, they are reproduced as Tables 6, 7, 1, and 2 in Appendix C. Briefly, Appendix C Tables 6 and 7 provide the number of detentions and the pounds of product involved in these actions for meat, poultry, and egg products; Appendix C Table 6 provides information on detentions made by the Office of Program Evaluation, Enforcement, and Review (OPEER), while Appendix C Table 7 provides the detention information for the Office of International Affairs (OIA). For establishment-specific data to have maximum benefit to users, inclusion of company names and addresses as well as the reason(s) for the product detention, would be important. Appendix C Table 1 provides quarterly totals of the number of establishments with administrative actions, while a more detailed summary by establishment, sorted by size is presented in Appendix C Table 2 (note that this table refers to large establishments, but similar tables corresponding to small and very small establishments are available in the original report). These include regulatory control actions, withholding actions, and suspensions. The current reporting system indicates the number of assessments performed and the number of actions taken; the publicly posted quarterly enforcement system reports identify the establishment, the action taken, and the basis for action. While considerable detail is provided in these tables, including the establishment name and the date and type of event, as well as its basis, some specific yet important information is missing. For example, an exact description of the basis for action (e.g., the specific nature of the Standard Sanitary Operating Procedure or HACCP failure) is not provided. It must be noted that industry representatives who spoke during the open session of the first committee meeting expressed strong reservations about the public release of both types of data. For example, NRs are free-form, text-based descriptions of deficiencies written by inspectors. Some industry stakeholders believe that they are subjective and that their frequency and quality vary by inspector and by district. There was also concern that the data could be taken out of context, particularly if a user were not privy to relevant background information regarding establishment operations, history, or events that occurred before the NR was issued. Industry representatives also cited considerable variation in FSAs. They expressed concerns that many FSAs relate to and reveal specific components of an establishment’s HACCP system that are considered proprietary. For example, if an establishment is performing validation or testing new equipment or procedures, the data collected are considered proprietary and are not normally subject to a FOIA request or to other forms of public release. In those cases, such sensitive information would need to be redacted from FSA data before public release. 53 See http://www.fsis.usda.gov/regulations_&_policies/QER_Q1_FY2011/index.asp (accessed August 5, 2011). 62

OCR for page 53
Committee deliberations revealed additional concerns about public release of NR and FSA data. Specifically, there was concern that if appropriate care were not taken with inspector coding issues, the data could be used to produce inspection comparisons within and between establishments even after inspector names were redacted. Public access to NRs and FSAs may also place front-line inspectors under increased scrutiny not only by the industry and FSIS supervisory staff but by the public. Despite the subjective nature of FSIS enforcement data, the committee noted that inspector-based data are generated in many regulatory arenas and have previously been released to the public. For example, the data released by DOL (for example, the Mine Safety and Health Administration [MSHA]; see Chapter 3) reflect investigators’ assessments of mine operators’ compliance with specific federal regulations, which often require some subjective judgment. In the short term, it may be necessary for FSIS to differentiate between NRs written by different inspection personnel or regions in initial public release efforts. The committee notes that the PHIS could make the NR and FSA reporting processes less subjective; this would ultimately result in greater consistency between inspections and inspectors. Product Recalls FSIS publicly posts recall information on its website. 54 Recall notification reports are issued for Class III recalls (i.e., recalls done for food which when consumed will not cause adverse health consequences). Recall releases, which are also sent to the media, are issued for Class II recalls (i.e., recalls done for food which when consumed may pose a remote probability of adverse health consequences) and Class I recalls (i.e., recalls done for food which when consumed poses a reasonable probability of health problems or death). Unlike most other data collected and posted by FSIS, recall information is used directly by consumers. Recalls are undertaken when there is a reasonable likelihood of injury to the public, so it is in the interest of public health to include as much detail as possible. A recall-notification report contains the name of the establishment, the establishment location, the type and quantity of the product, and the reason for the recall. When multiple product types are involved, name and product size, package establishment number, and general information about where the products were sold are provided. The recall notifications also include contact information for both the product manufacturer and FSIS. Recall-notification reports give some general background and guidance to consumers on the reasons for the recalls. They are updated periodically, in some cases daily. In addition, FSIS maintains a recall archive, 55 which lists all the recalls by year, beginning in 1994. Except for the earlier years (1994–1995), the archive links a recall to the recall-notification report that includes all the previously mentioned details. The data provided are useful for consumers who are seeking information about an individual product. The major issue relative to enhanced release of recall data is the need for the data to be in machine-readable format so that they can be linked to the other types of establishment-specific datasets. 54 See http://www.fsis.usda.gov/Fsis_Recalls/index.asp (accessed August 5, 2011). 55 See http://www.fsis.usda.gov/Fsis_Recalls/Recall_Case_Archive/index.asp (accessed August 5, 2011). 63

OCR for page 53
MEASURING THE POTENTIAL EFFECTS OF THE RELEASE OF ESTABLISHMENT-SPECIFIC FOOD SAFETY AND INSPECTION SERVICE DATA The first focus of government food-safety programs is on the protection of and improvement in public health. Food-safety regulatory programs have other effects as well, such as effects on domestic and international food markets, consumer and public perceptions of food safety, and individual and institutional trust in the integrity of the food-supply system (IOM and NRC, 2010; Ragona et al., 2011; Ruzante et al., 2010). The systematic release and analysis of FSIS data at the establishment level may have effects in all those arenas, and such effects are difficult to measure, but metrics for determining effects would be an important component of a data-disclosure strategic plan. From a public-health perspective, it remains difficult to establish a direct link between a single regulatory action or food-safety intervention and specific public-health outcomes. However, there are instances in which implementation of food-safety policies has been followed by measurable improvements in public health, albeit true causality has not been established. For example, the implementation of restaurant grade cards appears to be associated with a decrease in foodborne-illness hospitalizations (Jin and Leslie, 2003; Simon et al., 2005). Changes in the processing of poultry (specifically, the requirement for freezing) in New Zealand have been temporally associated with declines in human campylobacteriosis (Sears et al., 2011). In the United States, a decrease in the incidence of foodborne illness was observed in the years after the implementation of the Pathogen Reduction/HACCP Rule in 1997 (CDC, 2004; White et al., 2007). However, in the case of E. coli O157:H7, the declines in incidence could have been associated with multiple factors and not just with the adoption of HACCP (CDC, 2011). The link between those observations and the specific changes in processing practices has yet to be proved. The committee recognizes that the United States does not have the data or intervention analysis systems in place that could directly measure the potential public-health (or other) effects of specific activities in the FSIS food-safety programs (Batz et al., 2011; IOM and NRC, 2010). Thus, it is not now possible to measure directly the value of a public data-release program for improvements in food safety and public health. Nonetheless, that challenge is of great interest to all stakeholders. The committee understands that FSIS, the Food and Drug Administration, and the Centers for Disease Control and Prevention have embarked on a collaborative effort to develop food-safety metrics (FDA, 2010) so that public-health effects of food-safety activities can be measured better, and it encourages FSIS and other federal public- health agencies to continue and expand on these efforts. Although it is difficult to link the release of data with public-health outcomes directly, there are metrics that could potentially provide a means of approximating the value of public data release. For example, such tangible measures as incidence of positive pathogen-testing results or indicators of process integrity could be used as intermediate food-safety metrics. Metrics on the use of publicly released data could also be collected. These might logically include the number of Web downloads, reported and peer-reviewed reports generated, policy changes, and changes in industry practices. Qualitative measures are also necessary, including assessment of how data are interpreted and used by stakeholders. Determining how data are being used also could fall on the agency public-affairs staff, who are best equipped to interpret news-media stories or e-mail inquiries from the public. In all cases, such metrics would serve as a way to measure the value associated with public release of FSIS establishment-specific data. 64

OCR for page 53
MAJOR FINDINGS AND CONCLUSIONS  Public release of regulatory data is motivated by two broad purposes. The first addresses the public’s “right to know” about the actions of government. The second, “targeted transparency”, seeks to use information disclosure as a means of achieving specific public-policy objectives. The committee concluded that both purposes are relevant to the desire of FSIS to release establishment-specific data and that an effective disclosure policy would contribute to increased transparency to stakeholders. Releasing establishment-specific data might also affect public health favorably; this could be assessed, contingent on the development of measures specifically designed to evaluate such effects.  The committee identified several examples of links between release of detailed data by federal, state, or local agencies and the performance of individual facilities or firms or their products. In many cases, those data originate in regulatory (compliance and enforcement) activities. Three relevant examples are efforts supported by DOL (for example, in MSHA) the Environmental Protection Agency (for example, Enforcement and Compliance History Online [ECHO]), and several state and local public-health departments (for example, with respect to restaurant hygiene and inspection grading). The committee concluded that FSIS would benefit from consultation with those agencies and could build on their effective practices when designing a public data release program.  There is a substantial body of literature documenting the effects of disclosing establishment-specific regulatory information similar to that collected by FSIS. The literature suggests that release of those sorts of data can have substantial benefits. On the basis of a review of literature on the experience of other public agencies, the committee identified a number of potential benefits of public release of establishment-specific FSIS data, including providing incentives to protect brand reputation in food safety and to protect or enhance customer base and profitability; allowing downstream users to identify companies with performance records below and above the industry average and to create economic pressure to improve food safety; providing better insights into strengths and weaknesses of different processing practices, which could lead to industrywide improvements in food-safety practices; enhancing performance benchmarking; and improving the consistency of inspector performance. The committee concluded that public release of FSIS establishment-specific data, by themselves or in combination with other privately or publicly available data, could yield valuable insights that go beyond the regulatory uses for which the data were collected.  The committee concluded that the available evidence of adverse effects of public release of establishment-specific data by other government agencies is insufficient to predict specific problems that would be inherent in the release of establishment- specific data by FSIS. In the absence of information specific to FSIS, the committee identified a number of possible costs or unintended consequences of public release of establishment-specific data, including the financial commitment associated with designing and maintaining a useful data-disclosure system; the drawing of inappropriate conclusions as a result of misinterpretation of the data, particularly if appropriate context 65

OCR for page 53
is not provided to users; adverse effects on international trade; the risk that proprietary or confidential information could be deduced from the data; and adverse effects on inspector performance. Those unintended consequences might affect some stakeholder groups, but other groups may not consider them adverse. For example, although the literature suggests that disclosure of information about the performance of a specific facility has the potential to affect the facility’s profitability, it is precisely this possibility that creates an incentive for improved performance, which would constitute a benefit from the perspective of the public.  On the basis of its review of information and its deliberations, the committee concluded that strong arguments support public release of establishment-specific FSIS data, especially data that are now subject to release through FOIA, unless there is compelling evidence that such release is not in the public interest.  The committee concluded that to maximize its effectiveness and minimize its potential adverse unintended consequences, data disclosure needs to be guided by a carefully designed information-disclosure strategy. The committee also concluded that effective disclosure systems should be designed to allow continuous improvement as users gain a better understanding of how the data might be used and FSIS responds to stakeholder input. The disclosure strategy would consider the utility of the data to be released, how to release them (for example, their presentation), and how to ensure that the data are continuously updated and improved. The committee identified some key features of an effective information-disclosure plan, including ensuring the integrity of the data (requiring the development of protocols to ensure that they are accurate, timely, and likely to be useful before posting), providing precise and appropriate definitions of what is being quantified and adequate documentation of context (to mitigate the potential for misinterpretation of data), providing support for the analysis of the data by users (at a minimum providing them in machine-readable form to facilitate third-party analysis), and providing precautionary measures to prevent the linking of portions of the data in ways that would allow users to deduce confidential information about particular establishments. For all data types, it will be important to seek periodic input from stakeholders (industrial, academic, and consumer) to understand their needs and concerns. Focus groups targeted to key stakeholders may be an effective means of accomplishing that.  As part of its charge, the committee examined the issues specific to the public release of two types of FSIS establishment-specific data: sampling and testing data (derived from standard laboratory tests) and inspection and enforcement data (derived from text written by inspectors). In their deliberations, committee members expressed different views about the implications of releasing inspection and enforcement data, which are subjective in nature. A minority noted that minimizing the potential adverse consequences of releasing this type of data on an establishment-specific basis would be especially challenging, citing concerns about inspector variability, the potential for misinterpretation of the data, and confidentiality issues. The majority, however, believed strongly that public access to this type of data could help to identify 66

OCR for page 53
variability in inspector performance and enforcement outcomes and ultimately facilitate more uniform inspection.  In keeping with the purpose of attaining targeted transparency, public release of establishment-specific data is expected to result in improvement in food-safety efforts on the part of industry and government and ultimately to result in beneficial public-health outcomes. Although it is not possible to make a direct causal link between public data access and specific food-safety improvements, the committee concluded that measures of other outcomes of public release of establishment-specific data are available and that documenting those outcomes could provide insights into the relationship between data release and food safety. For example, public release of establishment-specific data could result in increased compliance with regulatory requirements, and FSIS could measure this. There are also ways of measuring the extent to which released data are used (such as number of Web downloads, peer- reviewed reports generated, and policy changes). REFERENCES Batz, M.B., Hoffman, S., and J.G. Morris, Jr. 2011. Ranking the Risks: The 10 Pathogen-Food Combinations with the Greatest Burden on Public Health. University of Florida, Emerging Pathogens Institute. Available at http://www.rwjf.org/files/research/72267report.pdf. Accessed June 1, 2011. CDC (Centers for Disease Control and Prevention). 2004. Preliminary FoodNet Data on the Incidence of Infection with Pathogens Transmitted Commonly Through Food --- Selected Sites, United States, 2003. Morbidity and Mortality Weekly Report (MMWR). Available at http://www.cdc.gov/mmwr/preview/mmwrhtml/mm5316a2.htm. Accessed June 8, 2011. CDC. 2011. Vital Signs: Incidence and Trends of Infection with Pathogens Transmitted Commonly through Food—Foodborne Diseases Active Surveillance Network, 10 U.S. Sites, 2006-2010. Morbidity and Mortality Weekly Report (MMWR). Available at http://www.cdc.gov/mmwr/preview/mmwrhtml/mm6022a5.htm?s_cid=mm6022a5_w. Accessed June 8, 2011. FDA (Food and Drug Administration). 2010. Public Workshop on Measuring Progress in Food Safety-Current Status and Future Directions (Transcript). Available at http://www.fda.gov/Food/NewsEvents/WorkshopsMeetingsConferences/ucm216259.htm . Accessed August 2, 2011. Hallman, W. K. 2008. Communicating about microbial risks in foods Pp. 205-262 In D. W. Schaffner (Ed.). Microbial Risk Analysis of Foods.Washington D. C.: American Society for Microbiology (ASM) Press. IOM/NRC (Institute of Medicine and National Research Council). 2010. Enhancing Food Safety: The Role of the Food and Drug Administration, R.B. Wallace, R.B. and M. Oria, eds. Washington, DC: The National Academies Press. Available at http://www.nap.edu/catalog.php?record_id=12892. Accessed June 10, 2011. 67

OCR for page 53
Jin, G.Z., and P. Leslie. 2003. The effect of information on product quality: Evidence from restaurant hygiene grade cards. Quarterly Journal of Economics 118(2):409-451. NRC. 2010. An Evaluation of the Food Safety Requirements of the Federal Purchase Ground Beef Program. Washington, DC: The National Academies Press. Paulos, J. A. 1988. Innumeracy: Mathematical Illiteracy and Its Consequences. New York: Hill and Wang. Ragona, M., Mazzocchi, M., Zanoli, A., Alldrick, A.J., Solfrizzo, M., and H. P. van Egmond 2011. Testing a toolbox for impact assessment of food safety regulations: Maximum levels for T-2 and HT-2 toxins in the European Union. Quality Assurance and Safety of Crops and Foods (3):12-23. Ruzante, J.M., Davidson, V.J., Caswell, J.A., Fazil, A., Cranfield, J.A.L., Henson, S.J., Anders, S.M., Schmidt, C., and J.M. Farber. 2010. A multifactorial risk prioritization framework for foodborne pathogens. Risk Analysis 30(5):724-742. Sears, A., Baker, M.G., Wilson, N., Marshall, J., Muellner, P., Campbell, D.M., Lake, R.J., and N.P. French. 2011. Marked campylobacteriosis decline after interventions aimed at poultry, New Zealand. Emerging Infectious Diseases 17(6):1007-1015. Available at: http://www.cdc.gov/eid/content/17/6/pdfs/EID_Vol17No6.pdf. Accessed August 2, 2011. Simon, P.A., Leslie, P., Run, G., Jin, G.Z., Reporter, R., Aguirre, A., and J.E. Fielding. 2005. Impact of restaurant hygiene grade cards on Foodborne-Disease hospitalizations in Los Angeles County. Available at http://www.stanford.edu/~pleslie/hospitalizations.pdf. Accessed June 10, 2011. Smith, A. 2010. Government Online: The internet gives citizens new paths to government services and information. Washington, D.C.: Pew Internet & American Life Project. Available at http://www.pewinternet.org/~/media//Files/Reports/2010/PIP_Government_Online_2010 _with_topline.pdf. Accessed October 27, 2010. USDA/FSIS (United States Department of Agriculture/Food Safety and Inspection Service). 2010. FSIS Strategic Data Analysis Plan for Domestic Inspection. www.fsis.usda.gov/oppde/nacmpi/.../2010_strategic_data_analysis_plan.pdf. Accessed June 10, 2011. USDA/FSIS. 2011. The FSIS Microbiological Testing Program for Ready-to-Eat (RTE) Meat and Poultry Products, 1990–2010”. Available at www.fsis.usda.gov/science/Micro_Testing_RTE/index.asp#results05. Accessed August 17, 2011. White, P.L., Naugle, A.L., Jackson, C.R., Fedorka-Cray, P.J., Rose, B.E., Pritchard K.M., Levine P., Saini P.K., Schroeder C.M., Dreyfuss M.S., Tan R., Holt K.G., Harman J, and S. Buchanan. 2007. Salmonella Enteritidis in meat, poultry, and pasteurized egg products regulated by the U.S. Food Safety and Inspection Service, 1998 through 2003. Journal of Food Protection 70(3):582-591. 68

OCR for page 53
Appendixes

OCR for page 53