TRB Special Report 309: Evaluating the Effectiveness of Offshore Safety and Environmental Management Systems (2012)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

6 Conclusions and Recommended Approach Because government oversight alone cannot reduce risks to the full extent possible, the Bureau of Safety and Environmental Enforcement (BSEE) will need to look beyond its predecessor agencies’ historical role of assuring compliance with prescriptive regulations and seize the cur- rent opportunity to design its role, at least partially, to encourage an atmosphere that helps the industry migrate from a compliance mentality to a culture of safety that includes compliance. To assist the agency in this endeavor, the Committee on the Effectiveness of Safety and Envi- ronmental Management Systems for Outer Continental Shelf Oil and Gas Operations (the committee) drew on the information obtained from presentations it received, site visits, published regulations, notices of proposed rulemaking, recommended practices, and previously pub- lished reports to develop the conclusions and recommended approach presented in this chapter. ConClusions Building safety Culture Through safety and Environmental Management systems Conclusion 1: if BsEE’s goal is, as it should be, to encourage a culture of safety so that individuals know the safety aspects of their actions and are motivated to think about safety, then the agency will need to evolve an evaluation system for safety and Environmental Management sys- tems (sEMs) that emphasizes the evaluation of attitudes and actions rather than documentation and paperwork. All of the elements of SEMS must be addressed, but it is much more important that those who are 89 

90 Evaluating the Effectiveness of Offshore Safety and Environmental Management Systems actually doing the work understand and implement SEMS than it is that SEMS documentation be verified with a checklist. Conclusion 2: A sEMs program that contains all the elements laid out in the sEMs regulation is necessary but not sufficient for creating a culture of safety. An organization’s safety culture will reduce risk; SEMS is but a means to that end. a. A culture of safety must be supported throughout the organization— from the top to the bottom—to be effective. b. A culture of safety only exists where the work occurs. If it does not actually drive the actions that people take, then it is only theoretical. Merely following a strict interpretation of a minimal SEMS program will not guarantee safe operations offshore. An effective SEMS program can- not rely on checklist compliance; the program must become ingrained in the operation’s management structure to be successful. The tenets of SEMS must be fully acknowledged and accepted by workers and be motivated from the top. Only then can an effective culture of safety be established and grow. Conclusion 3: The operator “owns” the sEMs program and is respon- sible for ensuring that it is operating effectively. The operator’s upper management is responsible and accountable for ensuring that a culture of safety exists. A safe operation is only possible when it is fully embraced by the organization. An organization cannot turn over the development and monitoring of its safety program to a third party and expect the pro- gram to be effective. Therefore, the ultimate responsibility for successful implementation of the SEMS program should reside with top manage- ment. If they do not take direct and complete ownership of the program, then safety will be relegated to a low status when difficult trade-offs need to be made. Conclusion 4: To be effective, safety and environmental management must be a dynamic process that evolves with time and is reflected in the regulator’s actions. Operations offshore are dynamic. Operating conditions, personnel, production requirements, and technologies are continually changing. Safety practices that were applicable during an 

Conclusions and Recommended Approach 91 earlier phase of operations may no longer be effective. Likewise, inspec- tion and audit criteria will become outdated as new technologies are employed and new environments are explored. To be effective, safety procedures and the audits that verify their effectiveness should also be dynamic and informed by risk. Conclusion 5: BsEE can encourage or hurt the development of a culture of safety by the way it measures and enforces sEMs. Forcing an opera- tion to satisfy checklists that require specific forms of documentation and penalizing those operations that do not is likely to encourage a culture of compliance and discourage the development of a culture of safety. Conclusion 6: A holistic combination of methods is necessary to evaluate the effectiveness and continuous improvement of an opera- tor’s sEMs program. Because of the diversity, complexity, and evolving nature of offshore oil and gas operations and the comprehensive nature of a fully implemented SEMS program, no single approach to inspec- tions and audits will be sufficient to ensure a successful SEMS program. Both occupational safety and process, or system, safety need to be veri- fied. A single one-dimensional snapshot of compliance will not provide the necessary insight upon which to construct a successful program. Conclusion 7: All parties involved in a safety management program like sEMs share the common goal of safe operations. It is not possible to regulate a culture of safety by inspections or audits; that culture needs to come from within an organization. Regulators are an integral part of an organization’s safety program, but they are limited as to what they can accomplish. The regulator’s role should be to develop an approach for the inspection and auditing of increased safety rather than toward a path of compliance. It is important that BSEE inspectors and auditors do not direct or dictate specific action, because doing so would move responsi- bility from the operator to the regulator. In addition, it is important that BSEE strive to nurture safety culture within its own organizational system. As observed by the Organisation for Economic Co-operation and Devel- opment, “The nature of the relationship between the regulator and the operator can influence the operator’s safety culture at a [facility] either positively or negatively. In promoting safety culture, a regulatory body should set a good example in its own performance” (OECD 1999, p.11). 

92 Evaluating the Effectiveness of Offshore Safety and Environmental Management Systems Effective inspections Conclusion 8: The routine presence of competent BsEE inspectors on an offshore operator’s facility is essential for verifying that the off- shore oil and gas industry is generally complying with all of BsEE’s regulations as well as providing a potential indication of compliance with sEMs. Conclusion 9: To be effective at identifying problems inherent in an operator’s safety culture, BsEE inspectors need to spend enough time on a facility to observe multiple activities. Effective Audits Conclusion 10: Audits, in and of themselves, are not sufficient to improve safety. For audit results to be effective, the operator needs to detect trends, identify deficiencies, take appropriate corrective action, and document the actions taken. Conclusion 11: As part of its holistic approach to measuring effective- ness, BsEE is responsible for ensuring that the implementation of sEMs is audited; however, the primary responsibility for auditing the sEMs program rests with the operator. a. A properly conducted, truly independent internal audit is potentially more effective than an independent third-party audit, as it reinforces ownership of the safety culture. b. Some operators are too small to have sufficient staff to perform a truly independent internal audit and will need to use an independent third party to conduct the audit. Inclusion of at least one person from the operator’s organization on the audit team will help cultivate manage- ment ownership of the audit. It would not be appropriate, however, to include a person directly involved in the day-to-day operation being audited. A properly motivated, active in-house safety program can be the best vehicle for discovering and correcting unsafe practices. Under most cir- cumstances, the personnel within an organization are the best equipped 

Conclusions and Recommended Approach 93 to identify both unsafe practices and feasible solutions. A third-party audit helps to ensure independence and can provide an outside perspec- tive on operations, but at a cost in operator ownership. Almost by defini- tion, a properly motivated and conducted internal safety audit requires buy-in from management; however, BSEE is responsible for ensuring that these internal audits are properly motivated and conducted. When resources do not permit an organization to conduct effective internal audits, third parties will need to be used. Conclusion 12: BsEE is responsible for verifying that quality audits are carried out and acted on appropriately. Because of the compre- hensive nature of the SEMS requirements, BSEE’s oversight of internal and third-party audits needs to include a range of techniques, each of which focuses on a different aspect of an operation’s safety system. These techniques are discussed below in the section on the committee’s recom- mended approach. Conclusion 13: Conducting a quality sEMs audit requires enough qualified personnel with sufficient time to spend on location. A quality SEMS audit requires a. A mechanism for qualifying auditors, b. An audit team with the skills required for the specific audit, and c. An understanding of how those performing the work perceive SEMS, in addition to a review of SEMS documentation. Conclusion 14: The skills and competencies required by inspectors are different from those required by auditors. To ensure that operators have established and are maintaining viable SEMS programs, BSEE needs to employ personnel skilled and well trained in two different areas: inspect- ing and auditing. Inspections that rely primarily on checklist compliance require inspectors with firsthand knowledge of equipment characteristics and procedures. In addition to this knowledge, the personnel charged with auditing a SEMS program need a higher-level knowledge of opera- tions, a detailed knowledge of SEMS, a thorough understanding of how operating procedures and safety performance are related, and training in performing audits. 

94 Evaluating the Effectiveness of Offshore Safety and Environmental Management Systems Conclusion 15: Because BsEE will have access to all sEMs programs, including audit and follow-up reports and its own inspection and audit reports, the agency is in the best position to gather and analyze this data to identify best practices and common trends (good and bad), and disseminate this information to the regulated community in a timely manner. BSEE auditing of SEMS programs will encompass a wide range of different operators that are using a variety of approaches to SEMS. Some of these programs may be more effective than others and will provide valuable lessons that can be used to improve SEMS for the entire industry. RECoMMEndEd AppRoACh BsEE should establish a holistic combination of methods necessary to ensure the effectiveness and continuous improvement of sEMs pro- grams. BsEE should establish a system that employs a combination of compliance inspections, audits, key performance indicators (Kpis), and a whistleblower program to ensure that sEMs programs are adequate, in place, and operating effectively and in a manner that promotes a culture of safety among the operators. The committee did not reject outright any of the approaches presented in Chapter 3 as being of no value, and BSEE could employ a combination of all the methods presented in Chapter 3. However, on the basis of the experience of representatives from comparable regulatory agencies (see Chapter 4) and of the committee members’ personal experience and expertise, the committee selected a combination of those approaches for which it believed some evidence of success was available. These approaches, the committee believed, would result in the most effective evaluation with the resources available. The technology and data necessary to use the SEMS monitoring sensors and calculation of risk methods described in Chapter 3 are not available at this time. The following sections of this chapter provide specific details of the recommended approach. Specific recommendations are given in boldface. The first section describes the inspection criteria and proce- dures that BSEE could use to verify compliance with specific regula- tions and to uncover obvious deficiencies in the implementation of 

Conclusions and Recommended Approach 95 the operator’s SEMS program. The second section discusses audits and describes a process by which BSEE could ensure that operator- initiated audits are complete, accurate, and effective (i.e., an audit of the audit process). The final sections summarize recommendations for a KPI program and a whistleblower program. Together, these methods constitute a holistic system that BSEE should employ to evaluate the effectiveness of SEMS. BsEE sEMs inspections BSEE should continue its current program of ensuring compliance with specific regulations. However, an inspection of sEMs (scheduled or otherwise) should not be focused solely on what is not in compliance; rather, the inspection should attempt to obtain a holistic view of the facility’s safety culture. Does the operating company empower its per- sonnel to take corrective action? Does it provide the resources necessary to do so? Are facility personnel only focused on the items they know an inspector typically will look at, and do they ignore the rest? A well- trained evaluator should be able to look beyond the black-and-white regulation and identify operators in marginal compliance, in order to guide them into a more complete state of compliance. BSEE inspectors have been trained to measure compliance with a standard or prescriptive checklist and without further training are not equipped to measure the effectiveness of individual SEMS programs. It may not be practical to expect the current BSEE inspection force to make subjective decisions as to whether a SEMS program is working correctly so that it meets the intent of the SEMS regulation and helps create a culture of safety. Current BSEE inspectors will have a tendency to issue incident of noncompliance (INC) notices for deviations of documenta- tion from a checklist, and such deviations may or may not be important in meeting the intent of SEMS. In turn, the issuing of INCs may focus operator attention on compliance in the way documentation is written rather than on establishing a culture that actually promotes safety. Most injuries and blowouts on offshore oil and gas facilities are not usually caused by mechanical failures identified by INCs (NRC 1990). Thus, to the extent possible, BsEE should train inspectors to employ other options in addition to using prescriptive checklists and issuing 

96 Evaluating the Effectiveness of Offshore Safety and Environmental Management Systems inCs. ideally, BsEE inspectors should look beyond the written regula- tion to identify operators in marginal compliance and guide them into a more complete state of compliance. Although it may be difficult for BSEE inspectors to identify operators in marginal compliance when it comes to assessing the adequacy of a SEMS program, it is not unreason- able to expect them to make overall observations, which, in turn, could help focus BSEE-initiated SEMS audits. During presentations and site visits1 the committee was told that, in some cases, BSEE inspectors visiting an offshore installation might be able to spot problems inherent in an operator’s safety culture by noticing obvious safety issues such as loose handrails, corroded walkways, or staff not wearing the appropriate personal protective gear. Other situations that indicate problems in the safety culture might be much harder to notice and require an in-depth investigation of safety-related approaches and practices, not only at the installation, but also in the operator’s overall operations, both offshore and onshore. BSEE inspectors need to spend enough time on a facility to observe multiple activities. To avoid the appearance of a conflict of interest, BSEE inspectors are not generally permitted to travel on operator-furnished helicopters, eat food provided by the operator while on the offshore facility, or stay overnight in operator-furnished quarters.2 Every other off- shore regulatory regime the committee talked to uses operator-furnished transportation, catering, and accommodations. The California State Lands Commission made a point of saying that being able to talk to the crew when its personnel travel to an installation in operator-furnished transportation is extremely beneficial in determining what is really going on at the installation. All regulatory bodies consulted by the committee agreed that time spent offshore in operator-furnished accommodations is essential to understanding the culture of safety on the facility. BSEE inspectors currently spend an extensive amount of time traveling to and from offshore installations. A more efficient use of available manpower would be to use the extensive amount of travel time to and from offshore P. Schroeder. Pacific OCS Regional Office, BSEE. Presentation to a subgroup of the committee, 1 Camarillo, California, March 22, 2010. Inspectors do stay overnight on some facilities that are very far from land. 2 

Conclusions and Recommended Approach 97 installations for informal discussions with operator personnel before a formal audit or visit to an offshore installation. Furthermore, BSEE has a finite budget, and that budget should be maximized so that the inspectors’ work can be fully effective. A large portion of BSEE’s budget is allocated to offshore transportation costs, and the rules BSEE inspectors must adhere to became more stringent following the Macondo well accident. It would be beneficial to identify a way to minimize these costs and reallocate some of these resources to the hiring and retaining of highly capable staff. BSEE should aspire to having its inspectors and engineers be recognized as being among the more highly qualified people in the offshore industry. Therefore, to the maximum extent practicable consistent with its increasing safety, environmental, and auditing responsibilities, BsEE, with industry input, should analyze the benefits and risks of using operator-furnished transportation and accommodations when performing inspections and audits. The agency should plan its bud- get recognizing that per dollar spent, the safety value of ensuring that high-quality inspectors and auditors are recruited and appropriately compensated and that critical data are collected, stored, and analyzed is greater than maintaining a completely independent transportation capability. Allowing overnight stays would increase the time BSEE staff would be able to spend interacting with the operating crew. More time on an installation would enable inspectors to better judge the degree to which a safety culture exists there. All other offshore regulators that the committee heard from believed that these measures would lead to bet- ter inspections and a higher degree of operating safety and, with proper management, would not lead to conflicts of interest. in the analysis, consideration should also be given to increasing the fees charged for inspections and to changing the structure by which inspection (and audit) costs are passed on to industry. Finally, as noted in Conclusion 4, operating conditions, personnel, production requirements, and technologies are continually changing. Therefore, BsEE should design and implement its inspection program on the basis of risk. The use of a risk-informed framework that focuses attention where new or potential problems are likely to occur will aid in the evolution of practices and audit procedures. 

98 Evaluating the Effectiveness of Offshore Safety and Environmental Management Systems operator and BsEE Audits Besides the inspections described above—which are, in essence, checks of compliance with specific regulations—and spot checks to determine whether individual elements of SEMS are being used effectively, a system for evaluating the effectiveness of an operator’s SEMS program requires routine periodic as well as incident- and event-driven audits. For these audits to evaluate the effectiveness of a SEMS program successfully, auditors will need to understand how the organization’s safety culture is reflected in its implementation of its SEMS program. Developing this understanding will require auditors to interact with operating crews and ask questions pertinent to how well crew members understand the SEMS program and how well used the program’s elements are in day-to- day practice. As noted in Chapter 3, techniques used in inspections (e.g., interviews and witnessing) should also be used when conducting audits. BSEE is responsible for ensuring that audits are conducted in a timely fashion, are thorough, and accomplish the goals set out above. The audit scheme adopted by BsEE should have the following characteristics: operator ownership, audit team independence, training and accredita- tion of auditors, access to top levels of management and audit reports, a definition of required audit frequency, and a scheme for quality assurance of audits. BSEE will also need to audit operator audit reports and have personnel capable of carrying out these tasks effectively. Except in the case of highly deficient systems, the goal of the audit should not be to pass or to fail. Rather, an audit should be designed to help senior management by presenting them with an independent view of the state of their efforts to establish a safety management system and, by extension, a culture of safety. The audit should identify areas for improvement and measure progress toward improvements rec- ommended in previous audits. In its program, BSEE should take into account that safety management is a dynamic process that evolves with time and that should not be judged solely on a pass–fail system. Operator Ownership BsEE should ensure that operators are involved in the audit itself. Several members of the committee have participated in financial, qual- ity, and safety audits and have observed that properly conducted inter- 

Conclusions and Recommended Approach 99 nal audits by personnel familiar with the operation are much better at uncovering problems than are external audits. In discussions with the committee, a representative of the Center for Offshore Safety (COS) stated that his company operated on this principle in its safety audits and that if BSEE mandated third-party audits, his company would merely add the third-party audit as a regulatory requirement to the internal audits it was already conducting. His company would not accept the third-party audit as a substitute for one of its internal audits. Large operators such as ExxonMobil and Shell have the ability to form independent audit teams within their organization. There are many smaller operations, however, that do not have a pool of skilled personnel from outside the operating organization being audited who are capable of performing an adequate audit of the organization. These operators should be able to use third-party auditors. if an independent third party must be used, at least one member of the audit team should be from the operator’s organization, and that person should not be directly involved in the day-to-day operation being audited. In some cases of very small operators with only a handful of employees, it may be neces- sary for the chief executive officer of the company to participate as a member of the audit team. Although operators are responsible for conducting audits, BSEE is responsible for verifying that quality audits are carried out and acted on appropriately. BsEE should perform complete or partial audits of sEMs programs when justified by reports from inspectors, reviews of operators’ audit reports, incidents, or events. BSEE’s oversight of inter- nal and third-party audits needs to include a range of techniques, each of which focuses on a different aspect of an operation’s safety system. BsEE can use reports from its compliance inspectors and its reviews of audit reports to identify the need for specific BsEE-conducted targeted or spot audits, or complete audits, to determine whether an operator’s sEMs program is improving safety. Direct spot inspections to verify that specific requirements are being met could perhaps be accomplished by relying on checklists. BsEE can also check to determine whether an organization’s sEMs program is improving its safety culture. Inter- views, demonstrations, and observations, rather than checklists, are nec- essary to make such a determination. For example, the question, “Do you 

100 Evaluating the Effectiveness of Offshore Safety and Environmental Management Systems have an MOC (management of change process)?” would shift to, “Show me how you know if your MOC is working properly.” Operators with an effective safety culture will be able to answer that question—and, in fact, describe the possible weaknesses in the MOC as implemented—even if it meets the letter of law. Audit Team Independence The operator’s audit of its SEMS program should be conducted by a truly independent, qualified team. It is critical that the audit team be made up of members who are divorced from the organization within the company that is responsible for the day-to-day operations of the installation and for meeting the financial, operational, safety, or envi- ronmental targets set by management. The audit team should report to the highest level practical given the size and complexity of the opera- tor’s organization. Members of the audit team could be permanently assigned or assigned on a rotating basis for a set number of years. The latter method helps disseminate information about and respect for the audit team throughout the organization. Participation in the audit team may also be considered as an interim assignment to higher level opera- tions assignments within the company. In no instance should the audit team include as a member someone who was recently assigned to the offshore facility being audited. Training and Accreditation of Auditors Audit team members should be trained to conduct audits and should be accredited by a method prescribed by BSEE.3 General qualifications for SEMS auditors are described in more detail in Chapter 5. BSEE must have independence (from industry) in how it trains its audi- tors. Nonetheless, the agency should consider certifying its auditors using a process similar to that used to certify industry auditors, and the certifica- tion should be of the same standard as outside accreditation institutions. BSEE, in consultation with the industry and, potentially, COS, should develop an approach to certify auditors, develop audit standards, and establish the process by which audits themselves are conducted. This recommendation is supported by the National Academy of Engineering–National Research 3 Council (NAE-NRC) report on the Macondo Well–Deepwater Horizon blowout (NAE-NRC 2011). 

Conclusions and Recommended Approach 101 Access to Top Levels of Management and Audit Reports The key to a successful audit system lies in discussions with top man- agement and in the steps top management takes toward continuous improvement. BsEE should ensure that the audit team has reviewed the audit report with top levels of management and obtained their sign-off on findings and areas for improvement. A copy of the audit report and a summary of its findings and conclusions should be sent to BsEE so that the agency can spot trends and disseminate informa- tion to the industry in a timely manner. As an alternative, BSEE could consider allowing COS to screen all reports. Doing so would bring in an element of the peer-review–peer- assist method for assessing effectiveness that is described in Chapter 3 and would further involve the industry as a whole in taking ownership of the development of a culture of safety. Such a charge from BSEE to COS would be consistent with the following elements of the COS operating basis as presented to the committee:4 • Compiling and analyzing key industry metrics, • Sponsoring functions to facilitate sharing and learning, • Identifying and promoting opportunities for the industry to continu- ously improve, • Interfacing with industry leaders to ensure leadership and system deficiencies are recognized and addressed promptly, and • Communicating with government and external stakeholders. Audit Frequency Under 30 CFR 250, Subpart S, as it currently stands, the timing for audits is very prescriptive. An operator must audit every element in its SEMS program every 3 years and include at least 15 percent of its installations. Installations in the Gulf of Mexico are very diverse. There are single- well unmanned installations, manned and unmanned installations with production equipment and no wells, manned and unmanned installations with both wells and production equipment, platforms with simultane- ous drilling and production operations, floating and bottom-supported J. Toellner. Center for Offshore Safety. Presentation to the committee, Houston, Texas, October 19, 4 2011. 

102 Evaluating the Effectiveness of Offshore Safety and Environmental Management Systems platforms, platforms producing 200 barrels of oil a day and platforms producing 200,000 barrels of oil a day, and all manner of drilling and workover rigs. Some installations produce high-pressure oil, which has the potential of flowing at high rates to the surface; others require pumps to lift oil to the surface. Some facilities produce natural gas with few impu- rities; others produce gas that contains levels of acid gases such as carbon dioxide and hydrogen sulfide. Some drilling operations are conducted in well-defined subsurface environments, while others are geologically uncertain. Some facilities have old equipment; others are new. Similarly, operators in the Gulf of Mexico are very diverse. Some are large, international, integrated companies; others are large, domestic energy and petroleum companies; and still others are very small indepen- dent operators with only minimal staff. Some operators are responsible for a large number of fields and installations, and some operate only one or two fields and installations. Some fields have multiple platforms tied together by pipelines or bridges, or both, and some have just one platform. Thus, it is difficult to establish a formula for audit frequency that does not become a paperwork burden and exercise for some operations while it is at the same time too lenient an audit frequency for others. Neither result is conducive to using SEMS to help establish an improved culture of safety in the industry. Because of the diversity of operations and operators, each opera- tor should be allowed to develop its own audit plan, subject to B sEE approval. Operator development of the audit plan would be a further step in establishing operator ownership in SEMS and its implementa- tion and would replace the current prescriptive frequency with a more appropriate risk-based audit frequency. Quality Assurance of Audits In any system involving audits, BSEE is responsible for monitoring the quality of the audits and for ensuring that what is learned from the audits is implemented. Under 30 CFR 250, Subpart S, as it currently stands, BSEE accomplishes this task by requiring that operators submit their audit plans before conducting audits, submit the qualifications of audit team members and third-party audit companies, and submit the results of the audits once they have been completed. The assumption is 

Conclusions and Recommended Approach 103 that BSEE will review and approve all submittals and will disseminate to the industry what is learned from the audits. in addition to requiring operators to submit an audit plan to BsEE, the agency should further require them to identify and report the follow-up actions taken as a result of the audit. BsEE should require that the frequency and scope of the audits specified in an operator’s audit plan be guided by risk rather than by a one-size-fits-all formula. Auditing Audits in addition to conducting spot inspections of sEMs compliance, BsEE should audit the quality of an operator’s audits. This task includes performing spot inspections of documentation and audits and, where appropriate, more complete BSEE reaudits of specific facilities. BSEE should also have a plan for carrying out these activities. in its audits, BsEE should use objective and subjective risk-based processes such as those employed by petroleum safety Authority (psA) norway, and these audits should be carried out by BsEE employees who are them- selves accredited. Personnel for Auditing BsEE should hire or train a sufficient number of auditors, including qualified audit team leaders and an adequate number of staff for analyz- ing audit reports effectively and auditing the accreditation system that the agency puts in place. Key performance indicators Specific KPIs associated with SEMS effectiveness are difficult to define and need further study and evaluation by both the industry and BSEE. Common safety and environmental metrics such as the number of inju- ries per year or the volume of spills per year provide only a part of the effectiveness picture. Other metrics need to be identified as lagging or leading indicators in relation to process safety. Once identified, such metrics can be used to monitor and direct the improvement of SEMS. BsEE can collect and evaluate data from operations within and across installations to identify specific problems and trends in opera- tions at a particular facility and across the industry. This information 

104 Evaluating the Effectiveness of Offshore Safety and Environmental Management Systems is needed to evaluate the SEMS audit approach and to identify opportu- nities for improvement. While the benefits from such a data exchange are obvious and important, implementation is far from trivial. An open data-collection and data-sharing protocol requires agreements across all parties to ensure that confidentiality and legal concerns are satisfied. BsEE should distribute information in a timely manner to the industry on trends and methods for improving the sEMs process and overall safety, as well as lessons learned, by means of publications, workshops, seminars, and other methods. Offshore safety organizations abroad that have programs similar to SEMS, such as PSA Norway and the United Kingdom (UK) Health and Safety Executive (HSE), have access to a considerable amount of data. Because many of the safety and environmental issues associated with offshore oil and gas operations are common worldwide, a data set com- piled from all of these organizations would be invaluable. BsEE should create a task force with the industry, psA norway, the uK hsE, and other similar regulatory bodies worldwide to identify Kpis. Creation of such a task force will help BSEE ensure that it is collecting the proper SEMS-relevant data and analyzing it appropriately to direct the agency’s effort to measure the effectiveness of SEMS. Whistleblower program PSA Norway, the Occupational Safety and Health Administration, and other organizations have found that programs that allow personnel to anonymously report possible violations directly to the regulator are helpful in identifying possible issues that may not be found by other means. The SEMS II notice of proposed rulemaking (BOEMRE 2011a) describes an approach that provides for anonymous reports of potential violations. BsEE should have a program for anonymous reporting and a process to follow up such reports and should use the information gained from them appropriately to modify BsEE inspections and audits. This program should also allow for the anonymous reporting of inappropriate behavior of BSEE personnel and potential improve- ments in BSEE policies and procedures, as well as potential violations by operators. Care should be taken in devising the program to make sure that it does not become a tool for disgruntled employees seeking 

Conclusions and Recommended Approach 105 to punish perceived wrongs. This recommendation supports Summary Recommendations 5.4 and 6.14 in the NAE-NRC report on the Macondo well–Deepwater Horizon blowout: Industry, BSEE, and other regulators should improve corporate and industrywide systems for reporting safety-related incidents. Reporting should be facilitated by enabling anonymous or “safety privileged” inputs. Corporations should investigate all such reports and disseminate their lessons-learned findings in a timely manner to all their operating and decision-making personnel and to the industry as a whole. A comprehen- sive lessons-learned repository should be maintained for industrywide use. This information can be used for training in accident prevention and con- tinually improving standards. (NAE-NRC 2011, pp. 107 and 123) REsouRCEs REquiREd BsEE should analyze its budget to ensure that it has sufficient finan- cial resources to implement these recommendations. Savings from any increased use of operator transportation and more efficient time offshore derived from operator-furnished accommodations could potentially be reprogrammed for the agency’s enhanced inspection and SEMS audit pro- grams and other related activities required by these recommendations.