Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 53
4
Public Release of Food Safety and Inspection Service
Establishment-Specific Data
The release of establishment-specific Food Safety and Inspective Service (FSIS) data
would provide public access to detailed sampling and testing data and inspection and
enforcement data. As discussed in Chapter 2, those data, with few exceptions, have been
available to the public only in aggregated form without establishment-level detail. Under the
“right to know” principles of the Freedom of Information Act (FOIA), the government is obliged
to provide data solicited by the public except in particular cases (such as personal or medical
information and trade secrets; see Chapter 2, Box 2-1). Thus, many of the establishment-specific
data that FSIS might release are available through FOIA requests. However, the availability of
the data through such requests is limited by the request process and a requester’s exclusive use of
the data unless the requester chooses to share them. Thus, public release of data by posting on
the Internet would result in a fundamentally changed information environment, including more
information and potentially more users and uses.
The experience of other federal agencies that have posted detailed data (reviewed in
Chapter 3) suggests that there may be benefits, as well as some potentially adverse unintended
consequences, of posting establishment-specific data that FSIS collects as part of its regulatory
mission. The conclusions of Chapter 3 also suggest that the benefits can grow and that the
concerns stemming from adverse consequences can be mitigated through careful design of data
release. That implies the need for a strategic plan designed to guide the agency in its choice of
data to release, how to release them, and the means by which to ensure that data are continuously
updated and improved.
The purpose of this chapter is three-fold: to discuss the potential benefits and adverse
consequences of releasing establishment-specific FSIS data; to present issues related to data
release that FSIS may want to consider during the development of a strategic data-release plan,
including approaches to measuring the public-health and other relevant effects of data release;
and to present the committee’s findings and conclusions regarding the public release of
establishment-specific FSIS data.
53
OCR for page 54
POSSIBLE BENEFITS, COSTS, AND UNINTENDED ADVERSE CONSEQUENCES
The committee identified a number of favorable outcomes that might be anticipated as a
consequence of the public release of FSIS establishment-specific data. At the most basic level,
such release would directly serve the first broad purpose of transparency in supporting the
public’s right to know. It would also serve the second broad purpose of transparency in helping
to achieve specific public-policy goals. In the latter role of “targeted transparency”, the major
effects of expanded data access would include the potential for better decision-making based on
improved information and stronger incentives for both the agency and food companies to
improve their performance.
Although it has not been definitively documented, one of the expected advantages of
providing public access to establishment-specific FSIS data is the improvement of public health.
That appears to have been the case for the Environmental Protection Agency’s release of Toxics
Release Inventory data and the publication of restaurant-inspection data (see Chapter 3).
Releasing establishment-specific FSIS data could potentially motivate individual companies and
sectors of the food industry to improve their overall food safety efforts. For example, data
release could provide incentives to protect brand reputation in food safety and to protect or
enhance customer base and profitability; allow downstream purchasers and consumers or public-
interest organizations to identify companies whose performance records were consistently above
or below the industry average and potentially create economic pressure to improve food safety;
provide better insights into strengths and weaknesses of different processing practices, which
could lead to industrywide improvements in food safety practices; enhance performance
benchmarking by individual companies, sectors, and the industry as a whole, including efforts
by individual companies that are seeking to avoid being identified as “below average”; and
improve the consistency of inspector performance.
Even if individual firms do not change their behavior in response to data posting, overall
food safety could improve if information about performance leads consumers to favor high-
performing establishments and hence causes a shift in the composition of the market. In addition
to providing incentives for the private sector, release of establishment-specific data could help to
identify needs for improvement in regulatory practices, and this might result in activities that
lead to improved public-health outcomes. For example, industry representatives raised concerns
about variation in enforcement practices among inspectors and districts. Analysis of
enforcement data could help to identify variability in enforcement outcomes (if present) of
comparable facilities. It could also help to identify effective practices in regulated facilities that
could be more broadly adopted. As in other systems that provide establishment-level
enforcement data, a coding system could be developed to protect the identities of individual
inspectors and still achieve the above outcomes.
Public release of establishment-specific FSIS data, by themselves or in combination with
other privately or publicly available data, could yield valuable insights that go well beyond the
regulatory uses for which the data were collected. For example, establishment-specific FSIS
microbial testing data might be combined with region-specific climate data in an effort to
develop better predictive risk models of pathogen load as a function of environmental conditions.
Making establishment-specific data publicly available might provide information that would be
useful for training the next generation of researchers, regulators, and industry food-safety
54
OCR for page 55
experts. The ability to analyze establishment-specific data would probably create a network of
third-party analysts who, because of their familiarity with the data and their structure, could help
FSIS to mine its own data and help individual companies or industry sectors to use the data to
improve their practices. By publicly releasing establishment-level data, FSIS would be sharing
with stakeholders, particularly those in the academic and industrial sectors, the opportunity to
perform data analysis. Those groups may be able to use the data in conjunction with other
sources of information to yield new insights or conclusions that could have significance for food
safety and public health. Public release of FSIS establishment-specific data could also lead to
improved public understanding of the considerable efforts made by FSIS and the industry to
ensure food safety. For example, if data release could be linked to specific improvements in food
safety, it might promote public perceptions of and confidence in the safety and integrity of the
food supply and in the companies and regulatory agencies that are responsible for ensuring them.
The benefits of releasing establishment-specific FSIS data must be balanced against the
potential unintended adverse consequences. Several of those were noted by industry
representatives who spoke to the committee in the open session of its first meeting. For example,
there was concern about the potential for misinterpretation of data. FSIS data are complex, and
appropriate analysis of them would require considerable training and skill in statistical analysis.
Making sense of the data also requires knowledge and experience to put them into an appropriate
context. Without such knowledge and experience, users could misinterpret the data, reach
unwarranted conclusions, or take the data out of context in an effort to support predetermined
positions. For example, FSIS publishes the results of microbiological sampling of ready-to-eat
(RTE) meat and poultry products. 40 In discussing those data, FSIS (USDA/FSIS, 2011) stated
that the agency “does not view the results of regulatory testing as estimates of national product
prevalence”. However, the data are often misused by the industry, mass media, and other
organizations as the basis for calculating pathogen prevalence in products.
Adverse effects on brand reputation could also occur as a consequence of public release
of establishment-specific data. It is possible that those effects will be experienced differently and
as a function of organization size. Larger organizations with more sophisticated corporate
communication functions or hired public-relations agencies will probably be able to establish
clear systems to explain violations. But smaller organizations that do not have the resources to
support such communication efforts might experience more lasting damage to brand reputation.
In short, it may be that the smaller companies are not unwilling to talk about food safety but just
do not know how to do it effectively. To minimize the potential for adverse consequences of the
public release of establishment-specific data on small and very small establishments, FSIS could
provide adequate documentation and explanation of both a noted deficiency and the possible
outcomes of such a deficiency.
Industry representatives were also concerned that releasing establishment-specific
pathogen-testing data could affect international trade. For example, foreign countries might use
publicly available FSIS testing data to bar entry of products from specific establishments on the
grounds of presumed risks to public health. 41 In the absence of a similar requirement to release
comparable data from their own countries’ companies, determining the true food-safety benefit
of barring import of products from select US companies would be difficult. The World Trade
40
See “The FSIS Microbiological Testing Program for Ready-to-Eat (RTE) Meat and Poultry Products, 1990–2010”
at http://www.fsis.usda.gov/science/Micro_Testing_RTE/index.asp#results05 (accessed August 17, 2011).
41
For example, foreign countries could conceivably use establishment-specific data to delist US establishments and
effective eliminate some international markets for select establishments and commodities.
55
OCR for page 56
Organization would eventually decide such disputes, but the short-term economic consequences
for individual firms could be substantial. The committee notes, however, that just as data on US-
based establishments would be released, FSIS data collected from foreign-plant inspections
would be released, as would data collected as part of FSIS imported-product testing and approval
or refusal. The effects of the release of establishment-specific data on imported products and on
US exports are unknown. However, if the release of data leads to improved food safety of both
domestic supplies and exports, the benefits would be realized not only by US consumers but by
foreign consumers.
Another concern is related to the unintentional release of proprietary or confidential
information. For example, Food Safety Assessments (FSAs) often correspond to specific
components of an establishment’s Hazard Analysis and Critical Control Points (HACCP) plan.
Hence, they may reveal details that are considered proprietary by the establishment or include
sensitive food-defense information. Clearly, the unintentional release of such information would
need to be guarded against, for example, by redaction of sensitive information before the release
of establishment-specific data.
Similarly, the agency would need to take precautions to avoid the possibility that portions
of the data could be linked in ways that would allow users to deduce confidential information
about particular establishments, such as FSIS inspection patterns, regulatory assignments, or
sampling regimes. For example, FSIS may choose to release data on enforcement actions only
after they have been completed, but real-time release of establishment-specific microbial-testing
data could reveal that additional testing had been ordered for a particular establishment, which
would indicate a new enforcement action. In that respect, the facility’s response to the corrective
action and its resolution would also need to be released.
Experience (see Chapter 3) suggests that public data release can affect inspector behavior
both favorably and unfavorably. For example, the public release of information could put more
FSIS pressure on inspectors to ensure the quality and consistency of their work. That could have
the beneficial effect of reducing variation in enforcement procedures that does not further the
agency’s mission. Or, if increased transparency of enforcement increases the stakes of their
outcomes, inspectors may face more pressure from firms regarding their outcomes (as has been
documented in, for example, restaurant hygiene).
Finally, public release of establishment-specific FSIS data in whole or in part does not
ensure that they will be useful or used. To make them so, FSIS will need to define a timetable for
data release and commit the resources necessary to ensure the accessibility, quality, timeliness,
and usefulness of the data. The costs could theoretically be offset, at least in part, by reduction in
resources now dedicated to responding to FOIA requests. Each year, FSIS spends about 20,000
hours and over $500,000 in complying with about 500 FOIA requests.42 That cost is supposed to
be compensated by external parties (requesters) 43 , but the committee learned that there are
exemptions to FOIA compensation , and it appears that current compensation is well below the
actual cost of providing such data (J. Reed, USDA-FSIS, Washington, DC, personal
communication, July 7, 2011). Public data release might save the agency some of the time and
money spent in operating the current FOIA system. However, there is a risk that an open system
will trigger more in-depth FOIA requests once reporters or other interested parties begin to
42
The committee derived these estimates on the basis of the listing of FOIA requests
(http://www.fsis.usda.gov/PDF/FOIA_Requests.pdf) and the information on making a FOIA request at the FSIS
Web site (http://www.fsis.usda.gov/FOIA/FOIA_Request/index.asp) (accessed July 8, 2011).
43
Fees collected by FSIS for FOIA requests are sent to the US Department of the Treasury.
56
OCR for page 57
peruse the information. Another challenge in making establishment-specific FSIS data publicly
available is that although FOIA requests may decrease, there may be a much larger need for
public-affairs staff to handle news-media requests that might be associated with the new data and
an increased need for an agency spokesperson to help the news media or the public interpret
information.
The above discussion shows that both benefits and unintended adverse consequences
could result from providing the public with access to data on individual FSIS-regulated
establishments. As discussed in Chapter 3, guidance from a carefully designed data-disclosure
strategic plan, as discussed in the next section, could maximize the effectiveness and minimize
the potential adverse consequences of sharing establishment-specific data. Because effective data
release requires cooperation among FSIS, industry, and the stakeholders most likely to use the
data, the development of the strategic plan would benefit from their input. And because data
themselves evolve, as do their uses, FSIS may also want to consider the need for continuous
improvements based on industry and user feedback and agency response to that feedback. Only
with such communication can FSIS maximize the value of public release of establishment-level
data.
STRATEGIC PLANNING OF DATA RELEASE
As noted above, the committee believes in the development of a strategic plan for public
release of establishment-specific data. The plan would be part of a larger comprehensive
strategic plan for data collection, management, and analysis. FSIS has a start on such an overall
plan in its Strategic Data Analysis Plan for Domestic Inspection (USDA/FSIS, 2010), and
inclusion of a public data-release component would be appropriate. Below are some key issues to
be considered in developing a data-disclosure strategic plan.
Identifying Potential Users
Multiple parties potentially have an interest in FSIS establishment-specific data. They
include consumers, the mass media, consumer groups, parties along the entire food-supply chain
(such as suppliers, producers, processors, distributors, retailers, and food-service operators),
third-party inspectors, researchers, and other government agencies. The parties differ in how they
would use FSIS data and in how information will be embedded in their decisions (if at all). For
example, it is doubtful that individual consumers would have the ability to (or even want to) sift
through FSIS data to trace connections between establishment-specific inspections and choices
made at the supermarket. Some third-party groups will probably have greater desire or ability to
translate information into a more useful form for ultimate consumer use. Consumers in particular
are inundated with information, including that having to do with food safety. Although the
public’s right to know is paramount, the reality is that the vast majority of the public does not
access government data on the Internet (Smith, 2010). A more likely scenario is that the data will
be used by the mass media to create news stories that will then be passed on to the consumer
either in traditional print format or by newer social-media channels.
For food processors, retailers, and food-service operators, the data could be valuable in
making sourcing decisions and managing risks associated with their supply chains. This group
57
OCR for page 58
would probably be more capable of analyzing detailed data provided by FSIS and integrating
them into monitoring, sourcing, and other core business decisions. Industry groups and trade
associations might play a similar role, serving as collective agents in analyzing information for
members and potentially using data for education or even for self-regulation.
Researchers in multiple disciplines have broad aims—shaped by disciplinary interests,
academic activity, and public-policy evaluation—for gaining access to more detailed FSIS data.
The research community would analyze the reliability and correlation of FSIS data to other food-
safety data. Other US government agencies and international entities may also analyze and draw
lessons from the released data.
The criteria for choosing which datasets to make public are directly related to the
potential users. The many parties that may use the data will use them in different and creative
ways that agency planners themselves might not foresee. Although the committee believes that it
will be difficult for FSIS to predict the full array of users and uses of the data, it also recognizes
the importance of determining the utility of data for different users. The committee believes that
this situation presents a strong argument for pursuing the broadest possible data release at the
most disaggregated level. Users can always aggregate data for their analytic needs, but they
cannot access disaggregate detail from aggregated data.
Databases, Linking, and Facilitating Analysis
FSIS establishment-specific data are held in a number of data tables in the Public Health
Information System (PHIS) (described in Chapter 2) and in older and diverse legacy data
systems (for example, the Performance Based Information System and the Automated Import
Information System). The data format and categories in the PHIS and various legacy systems are
not necessarily compatible. Unless that problem is addressed, users of the publicly released data
might find them difficult to analyze.
The desire to analyze multiple databases can result in linking problems. For example, a
linking problem could occur if a user wanted to know whether FSIS administrative actions (see
Tables 1, 2, 3, and 4 in Appendix C) have any relationship to FSIS food-safety adjudicatory
actions (see Table 5 in Appendix C). The only way to perform that analysis now would be to
extract the data from the tables manually, parse the establishment number from each field, and
then manually join them in a query. Creation of a relational database format that allows linking
of different datasets is likely to be the most effective means of facilitating that kind of analysis.
Such a relational database would provide linking up front, saving time and reducing errors. In
addition, release of the data in formats amenable to statistical analysis (such as, .xls and .csv),
rather than in .pdf or text formats, would allow broader user audience. Of course, the publicly
accessible database would also need to be highly secure and protected from modification or
hacking.
FSIS will need to address the extent to which it will provide bridges between legacy data
systems and data held in the PHIS, which represent different eras and different versions of
related programs. The agency will also need to provide guidance on how these datasets can be
combined in a way that is both valid and useful and that does not introduce systematic errors.
However, building bridges between different data systems need not necessarily be a prerequisite
for data release. Different users can develop different ways to standardize or adjust data as part
of their own uses of the released data.
58
OCR for page 59
Mechanics of Data Release
The Internet provides a low-cost and flexible form of disclosure, and, as detailed earlier
in this report, a variety of federal agencies have already developed sophisticated sites for data
disclosure. The experience of other federal agencies has shown the importance of providing data
to the extent possible in machine-readable formats (rather than in static forms, such as PDFs, that
do not lend themselves easily to analysis) to enhance their utility to users. Posting on the Internet
can also facilitate the timely release of data, but must be approached cautiously and with careful
planning. On the one hand, it could be argued that there is no reason not to release data as soon
as they are available. On the other hand, premature release of a dataset might mean that valuable
data are missing; this would make it difficult for a user to complete a comprehensive analysis or
to place the data in their proper context. FSIS will need to balance timely release with the need
for completeness.
The best data-sharing sites provide not only technical details but context for the variables
in the datasets. Describing the methods of data collection, sources of variability, and changes in
procedures that affect data consistency can be helpful for those analyzing and interpreting the
data. In addition, various federal agencies and departments, such as the Department of Labor
(DOL), are reaching out to potential users to assist them in developing sites and “apps” that will
improve the utility of data released to the public. 44
Providing Context for Interpretation
Adequate context of how data were collected and their limitations is important for the use
and interpretation of released data. For example, it is well known that Escherichia coli O157:H7
prevalence fluctuates by region and season. An analyst who compares data from two
establishments from which the data were collected at different times of the year or in different
regions, without an appreciation of temporal and geographic influences on pathogen prevalence,
might conclude that one establishment had a better ability to control the pathogen when the
opposite might be true.
The agency is well aware of the need for such guidance, as evidenced by statements on
the page linked to its quarterly enforcement report. 45 Statements like the following are provided:
“It is important to recognize that this [report] is only one aspect of the Agency's mission
to protect public health through food safety.”
“FSIS does not view the results of regulatory testing as estimates of national product
prevalence.”
“This report is a snapshot in time of a dynamic process. . . . Matters shown as under
appeal may be resolved. . . . Other actions could be appealed or closed.”
44
See, for example, http://challenge.gov/Labor/201-dol-informaction-app-challenge (accessed August 7, 2011).
45
See http://www.fsis.usda.gov/regulations_&_policies/QER_Q1_FY2011/index.asp (accessed August 5, 2011).
59
OCR for page 60
While an extensive discussion of risk communication is beyond the scope of this report,
and has been covered in a previous report by the National Research Council, 46 it is a vital
component of the implementation and sustainability of the data-disclosure program. In particular,
in the context of releasing establishment-specific data, it should be acknowledged that much of
the public has a poor understanding of microbiology, microbial risks, food processing hygiene,
and foodborne diseases (Hallman, 2008). Moreover, many people who are likely to be interested
in understanding its meaning may have difficulty interpreting numerical data. Studies suggest
that many people have difficulty grasping the magnitudes of very large and very small numbers
(e.g. parts per million), and have a hard time interpreting the meanings of fractions, proportions,
and probabilities (Paulos, 1988). As such, communications of risk involving mathematical
operations or statistical descriptions may not be easily understood by non-expert audiences.
Indeed, a variety of stakeholders will be interested in food safety, as identified in Chapter
3, and would probably value assistance in data interpretation if it were made available. To
mitigate the risk of misinterpretation of data and records, it may be useful for FSIS to plan the
rollout of the release of establishment-specific data in a graduated manner to help key audiences
to know what to expect of the data-release program and to prepare them to interpret the data
accurately. For example, the agency might wish to develop a series of recorded webinars and
other formal materials that would help those visiting the Web site to understand what is being
presented. The risk of misinterpretation may also be mitigated by third parties in scientific
societies, academe, or independent auditing agencies. Therefore, it may be useful to identify
independent third parties who are able to interpret FSIS data appropriately ahead of the rollout of
an open system and to make them publicly known. E-mail alerts could also be useful in
positioning information. These could be set up so that individual stakeholders could opt in to
receive content or audience-specific alerts or could be sent to targeted audiences, such as key
mass-media outlets, academics, industry, advocacy organizations, trade associations, and
scientific societies.
CONSIDERATIONS FOR RELEASE OF FOOD SAFETY AND INSPECTION SERVICE
ESTABLISHMENT-SPECIFIC DATA
Sampling and Testing Data
FSIS routinely collects sampling and testing data on the foodborne pathogens E. coli
O157:H7, Salmonella, and Listeria monocytogenes 47 and on the presence and concentration of
chemicals and residues. Microbial sampling and testing data can be divided into two broad
categories: those used for regulatory purposes and those used for baseline studies. As mentioned
in Chapter 2, FSIS is not considering the public release of establishment-level baseline data on
pathogen prevalence. Therefore, the ensuing discussion will focus on data produced for
46
The Commission on Risk Perception and Communication, Commission on Behavioral and
Social Sciences and Education, Commission on Physical Sciences, Mathematics, and
Resources, and National Research Council. 1989. Improving Risk Communication. Washington,
DC: National Academy Press.
47
See http://www.fsis.usda.gov/Science/Microbiology/index.asp (accessed August 5, 2011).
60
OCR for page 61
regulatory purposes. A brief description of various regulatory sampling and testing databases can
be found in Chapter 2, Box 2-2.
FSIS now posts the data from its E. coli O157:H7 ground-beef testing program in a
summarized or aggregated form. 48 The reports do not disclose the establishment unless there is a
recall of finished product. Public release of establishment-level data might include the
disposition of recalled and otherwise embargoed product. For example, if a product that was not
in commerce tested positive for E. coli O157:H7, the report could indicate whether the product
was diverted to a processing facility that fully cooked the meat or whether the raw product was
destroyed, in accordance with FSIS regulations.
FSIS is publishing results of completed sample sets from its Salmonella verification
program for young chicken (broiler) and turkey slaughter establishments in performance
Category 3; 49 this category consists of establishments whose Salmonella prevalence exceeds the
performance standard. The posted data include Product Class (for example, broilers),
Establishment Number, Company Name, City and State, Date of Sample Set Analysis
Completion, Most Current FSIS Set Result (but only repeating the category definition, not
providing exact numbers), and Previous FSIS Set Result (following the same format). To be
complete, public release of those data would also include establishments in Category 1 (whose
Salmonella prevalence is at or below half the performance standard) and Category 2 (whose
Salmonella prevalence is above half but not over the performance standard) and the level of
detail discussed above for E. coli O157:H7 testing, including sampling dates. It is also important
that released data include the regulatory thresholds on which data categorizations are based. For
example, effective July 1, 2011, FSIS increased the stringency of the Salmonella performance
standards, highlighting the need for data disclosure to state explicitly the performance standards
in effect for a particular test set.
FSIS publishes highly aggregated summaries of L. monocytogenes sampling and testing
of products, product-contact surfaces, and environmental surfaces. For example, an aggregated
report 50 shows Number of Samples, Number of Positives, and Percent Positive for the three L.
monocytogenes testing programs (ALLRTE, RTE001, and RLm). A more detailed but still
aggregated report is the Percent Positive Listeria monocytogenes Tests for RTE Meat and
Poultry by Product Category. 51 Again, as discussed for the two other pathogen-testing programs,
the committee concluded that for establishment-specific test results for L. monocytogenes to have
the greatest benefit to users outside the agency, the report would include detailed information on
the testing regime and results. As part of its regulatory activities, FSIS tests meat, poultry, and
processed egg products destined for human consumption for the presence of antibiotics,
sulfonamides, various other drugs, pesticides, and environmental chemicals in. It also tests for
the presence of such contaminants as dioxin. More information on those programs can be found
at the FSIS Web site. 52 With the exception of the Residue Repeat Violator Lists (which include
production facility name and address, animal type, tissue sampled, residue type, level of residue,
and tolerance level that was exceeded), FSIS usually provides public access to chemical and
residue testing results as reports of aggregated data, sometimes with detailed statistical analyses,
48
See http://www.fsis.usda.gov/Science/Ecoli_O157_Summary_Tables/index.asp (accessed August 5, 2011).
49
See http://www.fsis.usda.gov/PDF/Category_3_Broilers.pdf (accessed August 5, 2011).
50
See http://www.fsis.usda.gov/Science/Table24_RTE_Listeria_2009/index.asp and
http://www.fsis.usda.gov/Science/micro_testing_rte/index.asp (accessed August 5, 2011).
51
See http://www.fsis.usda.gov/Science/Table22_Micro_Testing_RTE_2008/index.asp (accessed August 5, 2011).
52
See http://www.fsis.usda.gov/Science/Chemistry/index.asp (accessed August 5, 2011).
61
OCR for page 62
but with little or no access to establishment-specific or product-specific information. If FSIS
decides to post establishment-specific data on chemical residues, it might want to consider
posting the names of facilities without violations with the information listed above.
Inspection and Enforcement Data
Inspection and enforcement data include noncompliance records (NRs) and
administrative actions (i.e., notices of intended enforcement or NOIEs). FSIS publicly releases
these data in aggregated or summarized form in its quarterly enforcement reports. Release of
these data in a more disaggregated and publicly available form might resemble that shown, for
example, in Tables 5a and b; Table 7; and Table 8 of that report. 53 Since these serve as useful
examples for discussion of the data, they are reproduced as Tables 6, 7, 1, and 2 in Appendix C.
Briefly, Appendix C Tables 6 and 7 provide the number of detentions and the pounds of product
involved in these actions for meat, poultry, and egg products; Appendix C Table 6 provides
information on detentions made by the Office of Program Evaluation, Enforcement, and Review
(OPEER), while Appendix C Table 7 provides the detention information for the Office of
International Affairs (OIA). For establishment-specific data to have maximum benefit to users,
inclusion of company names and addresses as well as the reason(s) for the product detention,
would be important. Appendix C Table 1 provides quarterly totals of the number of
establishments with administrative actions, while a more detailed summary by establishment,
sorted by size is presented in Appendix C Table 2 (note that this table refers to large
establishments, but similar tables corresponding to small and very small establishments are
available in the original report). These include regulatory control actions, withholding actions,
and suspensions. The current reporting system indicates the number of assessments performed
and the number of actions taken; the publicly posted quarterly enforcement system reports
identify the establishment, the action taken, and the basis for action. While considerable detail is
provided in these tables, including the establishment name and the date and type of event, as well
as its basis, some specific yet important information is missing. For example, an exact
description of the basis for action (e.g., the specific nature of the Standard Sanitary Operating
Procedure or HACCP failure) is not provided.
It must be noted that industry representatives who spoke during the open session of the
first committee meeting expressed strong reservations about the public release of both types of
data. For example, NRs are free-form, text-based descriptions of deficiencies written by
inspectors. Some industry stakeholders believe that they are subjective and that their frequency
and quality vary by inspector and by district. There was also concern that the data could be taken
out of context, particularly if a user were not privy to relevant background information regarding
establishment operations, history, or events that occurred before the NR was issued. Industry
representatives also cited considerable variation in FSAs. They expressed concerns that many
FSAs relate to and reveal specific components of an establishment’s HACCP system that are
considered proprietary. For example, if an establishment is performing validation or testing new
equipment or procedures, the data collected are considered proprietary and are not normally
subject to a FOIA request or to other forms of public release. In those cases, such sensitive
information would need to be redacted from FSA data before public release.
53
See http://www.fsis.usda.gov/regulations_&_policies/QER_Q1_FY2011/index.asp (accessed August 5, 2011).
62
OCR for page 63
Committee deliberations revealed additional concerns about public release of NR and
FSA data. Specifically, there was concern that if appropriate care were not taken with inspector
coding issues, the data could be used to produce inspection comparisons within and between
establishments even after inspector names were redacted. Public access to NRs and FSAs may
also place front-line inspectors under increased scrutiny not only by the industry and FSIS
supervisory staff but by the public.
Despite the subjective nature of FSIS enforcement data, the committee noted that
inspector-based data are generated in many regulatory arenas and have previously been released
to the public. For example, the data released by DOL (for example, the Mine Safety and Health
Administration [MSHA]; see Chapter 3) reflect investigators’ assessments of mine operators’
compliance with specific federal regulations, which often require some subjective judgment. In
the short term, it may be necessary for FSIS to differentiate between NRs written by different
inspection personnel or regions in initial public release efforts. The committee notes that the
PHIS could make the NR and FSA reporting processes less subjective; this would ultimately
result in greater consistency between inspections and inspectors.
Product Recalls
FSIS publicly posts recall information on its website. 54 Recall notification reports are
issued for Class III recalls (i.e., recalls done for food which when consumed will not cause
adverse health consequences). Recall releases, which are also sent to the media, are issued for
Class II recalls (i.e., recalls done for food which when consumed may pose a remote probability
of adverse health consequences) and Class I recalls (i.e., recalls done for food which when
consumed poses a reasonable probability of health problems or death). Unlike most other data
collected and posted by FSIS, recall information is used directly by consumers. Recalls are
undertaken when there is a reasonable likelihood of injury to the public, so it is in the interest of
public health to include as much detail as possible. A recall-notification report contains the name
of the establishment, the establishment location, the type and quantity of the product, and the
reason for the recall. When multiple product types are involved, name and product size, package
establishment number, and general information about where the products were sold are provided.
The recall notifications also include contact information for both the product manufacturer and
FSIS.
Recall-notification reports give some general background and guidance to consumers on
the reasons for the recalls. They are updated periodically, in some cases daily. In addition, FSIS
maintains a recall archive, 55 which lists all the recalls by year, beginning in 1994. Except for the
earlier years (1994–1995), the archive links a recall to the recall-notification report that includes
all the previously mentioned details. The data provided are useful for consumers who are seeking
information about an individual product. The major issue relative to enhanced release of recall
data is the need for the data to be in machine-readable format so that they can be linked to the
other types of establishment-specific datasets.
54
See http://www.fsis.usda.gov/Fsis_Recalls/index.asp (accessed August 5, 2011).
55
See http://www.fsis.usda.gov/Fsis_Recalls/Recall_Case_Archive/index.asp (accessed August 5, 2011).
63
OCR for page 64
MEASURING THE POTENTIAL EFFECTS OF THE RELEASE OF
ESTABLISHMENT-SPECIFIC FOOD SAFETY AND INSPECTION SERVICE DATA
The first focus of government food-safety programs is on the protection of and
improvement in public health. Food-safety regulatory programs have other effects as well, such
as effects on domestic and international food markets, consumer and public perceptions of food
safety, and individual and institutional trust in the integrity of the food-supply system (IOM and
NRC, 2010; Ragona et al., 2011; Ruzante et al., 2010). The systematic release and analysis of
FSIS data at the establishment level may have effects in all those arenas, and such effects are
difficult to measure, but metrics for determining effects would be an important component of a
data-disclosure strategic plan.
From a public-health perspective, it remains difficult to establish a direct link between a
single regulatory action or food-safety intervention and specific public-health outcomes.
However, there are instances in which implementation of food-safety policies has been followed
by measurable improvements in public health, albeit true causality has not been established. For
example, the implementation of restaurant grade cards appears to be associated with a decrease
in foodborne-illness hospitalizations (Jin and Leslie, 2003; Simon et al., 2005). Changes in the
processing of poultry (specifically, the requirement for freezing) in New Zealand have been
temporally associated with declines in human campylobacteriosis (Sears et al., 2011). In the
United States, a decrease in the incidence of foodborne illness was observed in the years after the
implementation of the Pathogen Reduction/HACCP Rule in 1997 (CDC, 2004; White et al.,
2007). However, in the case of E. coli O157:H7, the declines in incidence could have been
associated with multiple factors and not just with the adoption of HACCP (CDC, 2011).
The link between those observations and the specific changes in processing practices has
yet to be proved. The committee recognizes that the United States does not have the data or
intervention analysis systems in place that could directly measure the potential public-health (or
other) effects of specific activities in the FSIS food-safety programs (Batz et al., 2011; IOM and
NRC, 2010). Thus, it is not now possible to measure directly the value of a public data-release
program for improvements in food safety and public health. Nonetheless, that challenge is of
great interest to all stakeholders. The committee understands that FSIS, the Food and Drug
Administration, and the Centers for Disease Control and Prevention have embarked on a
collaborative effort to develop food-safety metrics (FDA, 2010) so that public-health effects of
food-safety activities can be measured better, and it encourages FSIS and other federal public-
health agencies to continue and expand on these efforts.
Although it is difficult to link the release of data with public-health outcomes directly,
there are metrics that could potentially provide a means of approximating the value of public
data release. For example, such tangible measures as incidence of positive pathogen-testing
results or indicators of process integrity could be used as intermediate food-safety metrics.
Metrics on the use of publicly released data could also be collected. These might logically
include the number of Web downloads, reported and peer-reviewed reports generated, policy
changes, and changes in industry practices. Qualitative measures are also necessary, including
assessment of how data are interpreted and used by stakeholders. Determining how data are
being used also could fall on the agency public-affairs staff, who are best equipped to interpret
news-media stories or e-mail inquiries from the public. In all cases, such metrics would serve as
a way to measure the value associated with public release of FSIS establishment-specific data.
64
OCR for page 65
MAJOR FINDINGS AND CONCLUSIONS
Public release of regulatory data is motivated by two broad purposes. The first addresses
the public’s “right to know” about the actions of government. The second, “targeted
transparency”, seeks to use information disclosure as a means of achieving specific
public-policy objectives. The committee concluded that both purposes are relevant to
the desire of FSIS to release establishment-specific data and that an effective
disclosure policy would contribute to increased transparency to stakeholders.
Releasing establishment-specific data might also affect public health favorably; this
could be assessed, contingent on the development of measures specifically designed
to evaluate such effects.
The committee identified several examples of links between release of detailed data by
federal, state, or local agencies and the performance of individual facilities or firms or
their products. In many cases, those data originate in regulatory (compliance and
enforcement) activities. Three relevant examples are efforts supported by DOL (for
example, in MSHA) the Environmental Protection Agency (for example, Enforcement
and Compliance History Online [ECHO]), and several state and local public-health
departments (for example, with respect to restaurant hygiene and inspection grading).
The committee concluded that FSIS would benefit from consultation with those
agencies and could build on their effective practices when designing a public data
release program.
There is a substantial body of literature documenting the effects of disclosing
establishment-specific regulatory information similar to that collected by FSIS. The
literature suggests that release of those sorts of data can have substantial benefits. On the
basis of a review of literature on the experience of other public agencies, the committee
identified a number of potential benefits of public release of establishment-specific FSIS
data, including providing incentives to protect brand reputation in food safety and to
protect or enhance customer base and profitability; allowing downstream users to identify
companies with performance records below and above the industry average and to create
economic pressure to improve food safety; providing better insights into strengths and
weaknesses of different processing practices, which could lead to industrywide
improvements in food-safety practices; enhancing performance benchmarking; and
improving the consistency of inspector performance. The committee concluded that
public release of FSIS establishment-specific data, by themselves or in combination
with other privately or publicly available data, could yield valuable insights that go
beyond the regulatory uses for which the data were collected.
The committee concluded that the available evidence of adverse effects of public
release of establishment-specific data by other government agencies is insufficient to
predict specific problems that would be inherent in the release of establishment-
specific data by FSIS. In the absence of information specific to FSIS, the committee
identified a number of possible costs or unintended consequences of public release of
establishment-specific data, including the financial commitment associated with
designing and maintaining a useful data-disclosure system; the drawing of inappropriate
conclusions as a result of misinterpretation of the data, particularly if appropriate context
65
OCR for page 66
is not provided to users; adverse effects on international trade; the risk that proprietary or
confidential information could be deduced from the data; and adverse effects on inspector
performance. Those unintended consequences might affect some stakeholder groups, but
other groups may not consider them adverse. For example, although the literature
suggests that disclosure of information about the performance of a specific facility has
the potential to affect the facility’s profitability, it is precisely this possibility that creates
an incentive for improved performance, which would constitute a benefit from the
perspective of the public.
On the basis of its review of information and its deliberations, the committee
concluded that strong arguments support public release of establishment-specific
FSIS data, especially data that are now subject to release through FOIA, unless
there is compelling evidence that such release is not in the public interest.
The committee concluded that to maximize its effectiveness and minimize its
potential adverse unintended consequences, data disclosure needs to be guided by a
carefully designed information-disclosure strategy. The committee also concluded
that effective disclosure systems should be designed to allow continuous
improvement as users gain a better understanding of how the data might be used
and FSIS responds to stakeholder input. The disclosure strategy would consider the
utility of the data to be released, how to release them (for example, their presentation),
and how to ensure that the data are continuously updated and improved. The committee
identified some key features of an effective information-disclosure plan, including
ensuring the integrity of the data (requiring the development of protocols to ensure that
they are accurate, timely, and likely to be useful before posting), providing precise and
appropriate definitions of what is being quantified and adequate documentation of
context (to mitigate the potential for misinterpretation of data), providing support for the
analysis of the data by users (at a minimum providing them in machine-readable form to
facilitate third-party analysis), and providing precautionary measures to prevent the
linking of portions of the data in ways that would allow users to deduce confidential
information about particular establishments. For all data types, it will be important to
seek periodic input from stakeholders (industrial, academic, and consumer) to understand
their needs and concerns. Focus groups targeted to key stakeholders may be an effective
means of accomplishing that.
As part of its charge, the committee examined the issues specific to the public release of
two types of FSIS establishment-specific data: sampling and testing data (derived from
standard laboratory tests) and inspection and enforcement data (derived from text written
by inspectors). In their deliberations, committee members expressed different views
about the implications of releasing inspection and enforcement data, which are
subjective in nature. A minority noted that minimizing the potential adverse
consequences of releasing this type of data on an establishment-specific basis would
be especially challenging, citing concerns about inspector variability, the potential
for misinterpretation of the data, and confidentiality issues. The majority, however,
believed strongly that public access to this type of data could help to identify
66
OCR for page 67
variability in inspector performance and enforcement outcomes and ultimately
facilitate more uniform inspection.
In keeping with the purpose of attaining targeted transparency, public release of
establishment-specific data is expected to result in improvement in food-safety efforts on
the part of industry and government and ultimately to result in beneficial public-health
outcomes. Although it is not possible to make a direct causal link between public
data access and specific food-safety improvements, the committee concluded that
measures of other outcomes of public release of establishment-specific data are
available and that documenting those outcomes could provide insights into the
relationship between data release and food safety. For example, public release of
establishment-specific data could result in increased compliance with regulatory
requirements, and FSIS could measure this. There are also ways of measuring the
extent to which released data are used (such as number of Web downloads, peer-
reviewed reports generated, and policy changes).
REFERENCES
Batz, M.B., Hoffman, S., and J.G. Morris, Jr. 2011. Ranking the Risks: The 10 Pathogen-Food
Combinations with the Greatest Burden on Public Health. University of Florida,
Emerging Pathogens Institute. Available at
http://www.rwjf.org/files/research/72267report.pdf. Accessed June 1, 2011.
CDC (Centers for Disease Control and Prevention). 2004. Preliminary FoodNet Data on the
Incidence of Infection with Pathogens Transmitted Commonly Through Food --- Selected
Sites, United States, 2003. Morbidity and Mortality Weekly Report (MMWR). Available
at http://www.cdc.gov/mmwr/preview/mmwrhtml/mm5316a2.htm. Accessed June 8,
2011.
CDC. 2011. Vital Signs: Incidence and Trends of Infection with Pathogens Transmitted
Commonly through Food—Foodborne Diseases Active Surveillance Network, 10 U.S.
Sites, 2006-2010. Morbidity and Mortality Weekly Report (MMWR). Available at
http://www.cdc.gov/mmwr/preview/mmwrhtml/mm6022a5.htm?s_cid=mm6022a5_w.
Accessed June 8, 2011.
FDA (Food and Drug Administration). 2010. Public Workshop on Measuring Progress in Food
Safety-Current Status and Future Directions (Transcript). Available at
http://www.fda.gov/Food/NewsEvents/WorkshopsMeetingsConferences/ucm216259.htm
. Accessed August 2, 2011.
Hallman, W. K. 2008. Communicating about microbial risks in foods Pp. 205-262 In D. W.
Schaffner (Ed.). Microbial Risk Analysis of Foods.Washington D. C.: American Society
for Microbiology (ASM) Press.
IOM/NRC (Institute of Medicine and National Research Council). 2010. Enhancing Food Safety:
The Role of the Food and Drug Administration, R.B. Wallace, R.B. and M. Oria, eds.
Washington, DC: The National Academies Press. Available at
http://www.nap.edu/catalog.php?record_id=12892. Accessed June 10, 2011.
67
OCR for page 68
Jin, G.Z., and P. Leslie. 2003. The effect of information on product quality: Evidence from
restaurant hygiene grade cards. Quarterly Journal of Economics 118(2):409-451.
NRC. 2010. An Evaluation of the Food Safety Requirements of the Federal Purchase Ground
Beef Program. Washington, DC: The National Academies Press.
Paulos, J. A. 1988. Innumeracy: Mathematical Illiteracy and Its Consequences. New York: Hill
and Wang.
Ragona, M., Mazzocchi, M., Zanoli, A., Alldrick, A.J., Solfrizzo, M., and H. P. van Egmond
2011. Testing a toolbox for impact assessment of food safety regulations: Maximum
levels for T-2 and HT-2 toxins in the European Union. Quality Assurance and Safety of
Crops and Foods (3):12-23.
Ruzante, J.M., Davidson, V.J., Caswell, J.A., Fazil, A., Cranfield, J.A.L., Henson, S.J., Anders,
S.M., Schmidt, C., and J.M. Farber. 2010. A multifactorial risk prioritization framework
for foodborne pathogens. Risk Analysis 30(5):724-742.
Sears, A., Baker, M.G., Wilson, N., Marshall, J., Muellner, P., Campbell, D.M., Lake, R.J., and
N.P. French. 2011. Marked campylobacteriosis decline after interventions aimed at
poultry, New Zealand. Emerging Infectious Diseases 17(6):1007-1015. Available at:
http://www.cdc.gov/eid/content/17/6/pdfs/EID_Vol17No6.pdf. Accessed August 2, 2011.
Simon, P.A., Leslie, P., Run, G., Jin, G.Z., Reporter, R., Aguirre, A., and J.E. Fielding.
2005. Impact of restaurant hygiene grade cards on Foodborne-Disease hospitalizations in
Los Angeles County. Available at http://www.stanford.edu/~pleslie/hospitalizations.pdf.
Accessed June 10, 2011.
Smith, A. 2010. Government Online: The internet gives citizens new paths to government
services and information. Washington, D.C.: Pew Internet & American Life Project.
Available at
http://www.pewinternet.org/~/media//Files/Reports/2010/PIP_Government_Online_2010
_with_topline.pdf. Accessed October 27, 2010.
USDA/FSIS (United States Department of Agriculture/Food Safety and Inspection Service).
2010. FSIS Strategic Data Analysis Plan for Domestic Inspection.
www.fsis.usda.gov/oppde/nacmpi/.../2010_strategic_data_analysis_plan.pdf. Accessed
June 10, 2011.
USDA/FSIS. 2011. The FSIS Microbiological Testing Program for Ready-to-Eat (RTE) Meat
and Poultry Products, 1990–2010”. Available at
www.fsis.usda.gov/science/Micro_Testing_RTE/index.asp#results05. Accessed August
17, 2011.
White, P.L., Naugle, A.L., Jackson, C.R., Fedorka-Cray, P.J., Rose, B.E., Pritchard K.M., Levine
P., Saini P.K., Schroeder C.M., Dreyfuss M.S., Tan R., Holt K.G., Harman J, and S.
Buchanan. 2007. Salmonella Enteritidis in meat, poultry, and pasteurized egg products
regulated by the U.S. Food Safety and Inspection Service, 1998 through 2003. Journal of
Food Protection 70(3):582-591.
68
OCR for page 69
Appendixes
OCR for page 70
