Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 109
6
Conclusions and Recommended Approach
Because government oversight alone cannot reduce risks to the full
extent possible, the Bureau of Safety and Environmental Enforcement
(BSEE) will need to look beyond its predecessor agencies’ historical role
of assuring compliance with prescriptive regulations and seize the cur-
rent opportunity to design its role, at least partially, to encourage an
atmosphere that helps the industry migrate from a compliance mentality
to a culture of safety that includes compliance. To assist the agency
in this endeavor, the Committee on the Effectiveness of Safety and Envi-
ronmental Management Systems for Outer Continental Shelf Oil and
Gas Operations (the committee) drew on the information obtained
from presentations it received, site visits, published regulations, notices
of proposed rulemaking, recommended practices, and previously pub-
lished reports to develop the conclusions and recommended approach
presented in this chapter.
ConClusions
Building safety Culture Through safety
and Environmental Management systems
Conclusion 1: if BsEE’s goal is, as it should be, to encourage a culture
of safety so that individuals know the safety aspects of their actions and
are motivated to think about safety, then the agency will need to evolve
an evaluation system for safety and Environmental Management sys-
tems (sEMs) that emphasizes the evaluation of attitudes and actions
rather than documentation and paperwork. All of the elements of SEMS
must be addressed, but it is much more important that those who are
89
OCR for page 110
90 Evaluating the Effectiveness of Offshore Safety and Environmental Management Systems
actually doing the work understand and implement SEMS than it is that
SEMS documentation be verified with a checklist.
Conclusion 2: A sEMs program that contains all the elements laid out
in the sEMs regulation is necessary but not sufficient for creating a
culture of safety. An organization’s safety culture will reduce risk; SEMS
is but a means to that end.
a. A culture of safety must be supported throughout the organization—
from the top to the bottom—to be effective.
b. A culture of safety only exists where the work occurs. If it does not
actually drive the actions that people take, then it is only theoretical.
Merely following a strict interpretation of a minimal SEMS program will
not guarantee safe operations offshore. An effective SEMS program can-
not rely on checklist compliance; the program must become ingrained
in the operation’s management structure to be successful. The tenets
of SEMS must be fully acknowledged and accepted by workers and be
motivated from the top. Only then can an effective culture of safety be
established and grow.
Conclusion 3: The operator “owns” the sEMs program and is respon-
sible for ensuring that it is operating effectively. The operator’s upper
management is responsible and accountable for ensuring that a culture
of safety exists. A safe operation is only possible when it is fully embraced
by the organization. An organization cannot turn over the development
and monitoring of its safety program to a third party and expect the pro-
gram to be effective. Therefore, the ultimate responsibility for successful
implementation of the SEMS program should reside with top manage-
ment. If they do not take direct and complete ownership of the program,
then safety will be relegated to a low status when difficult trade-offs need
to be made.
Conclusion 4: To be effective, safety and environmental management
must be a dynamic process that evolves with time and is reflected in
the regulator’s actions. Operations offshore are dynamic. Operating
conditions, personnel, production requirements, and technologies are
continually changing. Safety practices that were applicable during an
OCR for page 111
Conclusions and Recommended Approach 91
earlier phase of operations may no longer be effective. Likewise, inspec-
tion and audit criteria will become outdated as new technologies are
employed and new environments are explored. To be effective, safety
procedures and the audits that verify their effectiveness should also be
dynamic and informed by risk.
Conclusion 5: BsEE can encourage or hurt the development of a culture
of safety by the way it measures and enforces sEMs. Forcing an opera-
tion to satisfy checklists that require specific forms of documentation and
penalizing those operations that do not is likely to encourage a culture of
compliance and discourage the development of a culture of safety.
Conclusion 6: A holistic combination of methods is necessary to
evaluate the effectiveness and continuous improvement of an opera-
tor’s sEMs program. Because of the diversity, complexity, and evolving
nature of offshore oil and gas operations and the comprehensive nature
of a fully implemented SEMS program, no single approach to inspec-
tions and audits will be sufficient to ensure a successful SEMS program.
Both occupational safety and process, or system, safety need to be veri-
fied. A single one-dimensional snapshot of compliance will not provide
the necessary insight upon which to construct a successful program.
Conclusion 7: All parties involved in a safety management program like
sEMs share the common goal of safe operations. It is not possible to
regulate a culture of safety by inspections or audits; that culture needs to
come from within an organization. Regulators are an integral part of an
organization’s safety program, but they are limited as to what they can
accomplish. The regulator’s role should be to develop an approach for
the inspection and auditing of increased safety rather than toward a path
of compliance. It is important that BSEE inspectors and auditors do not
direct or dictate specific action, because doing so would move responsi-
bility from the operator to the regulator. In addition, it is important that
BSEE strive to nurture safety culture within its own organizational system.
As observed by the Organisation for Economic Co-operation and Devel-
opment, “The nature of the relationship between the regulator and the
operator can influence the operator’s safety culture at a [facility] either
positively or negatively. In promoting safety culture, a regulatory body
should set a good example in its own performance” (OECD 1999, p.11).
OCR for page 112
92 Evaluating the Effectiveness of Offshore Safety and Environmental Management Systems
Effective inspections
Conclusion 8: The routine presence of competent BsEE inspectors on
an offshore operator’s facility is essential for verifying that the off-
shore oil and gas industry is generally complying with all of BsEE’s
regulations as well as providing a potential indication of compliance
with sEMs.
Conclusion 9: To be effective at identifying problems inherent in an
operator’s safety culture, BsEE inspectors need to spend enough time
on a facility to observe multiple activities.
Effective Audits
Conclusion 10: Audits, in and of themselves, are not sufficient to
improve safety. For audit results to be effective, the operator needs to
detect trends, identify deficiencies, take appropriate corrective action,
and document the actions taken.
Conclusion 11: As part of its holistic approach to measuring effective-
ness, BsEE is responsible for ensuring that the implementation of
sEMs is audited; however, the primary responsibility for auditing the
sEMs program rests with the operator.
a. A properly conducted, truly independent internal audit is potentially
more effective than an independent third-party audit, as it reinforces
ownership of the safety culture.
b. Some operators are too small to have sufficient staff to perform a truly
independent internal audit and will need to use an independent third
party to conduct the audit. Inclusion of at least one person from the
operator’s organization on the audit team will help cultivate manage-
ment ownership of the audit. It would not be appropriate, however,
to include a person directly involved in the day-to-day operation
being audited.
A properly motivated, active in-house safety program can be the best
vehicle for discovering and correcting unsafe practices. Under most cir-
cumstances, the personnel within an organization are the best equipped
OCR for page 113
Conclusions and Recommended Approach 93
to identify both unsafe practices and feasible solutions. A third-party
audit helps to ensure independence and can provide an outside perspec-
tive on operations, but at a cost in operator ownership. Almost by defini-
tion, a properly motivated and conducted internal safety audit requires
buy-in from management; however, BSEE is responsible for ensuring
that these internal audits are properly motivated and conducted. When
resources do not permit an organization to conduct effective internal
audits, third parties will need to be used.
Conclusion 12: BsEE is responsible for verifying that quality audits
are carried out and acted on appropriately. Because of the compre-
hensive nature of the SEMS requirements, BSEE’s oversight of internal
and third-party audits needs to include a range of techniques, each of
which focuses on a different aspect of an operation’s safety system. These
techniques are discussed below in the section on the committee’s recom-
mended approach.
Conclusion 13: Conducting a quality sEMs audit requires enough
qualified personnel with sufficient time to spend on location. A quality
SEMS audit requires
a. A mechanism for qualifying auditors,
b. An audit team with the skills required for the specific audit, and
c. An understanding of how those performing the work perceive SEMS,
in addition to a review of SEMS documentation.
Conclusion 14: The skills and competencies required by inspectors are
different from those required by auditors. To ensure that operators have
established and are maintaining viable SEMS programs, BSEE needs to
employ personnel skilled and well trained in two different areas: inspect-
ing and auditing. Inspections that rely primarily on checklist compliance
require inspectors with firsthand knowledge of equipment characteristics
and procedures. In addition to this knowledge, the personnel charged
with auditing a SEMS program need a higher-level knowledge of opera-
tions, a detailed knowledge of SEMS, a thorough understanding of how
operating procedures and safety performance are related, and training
in performing audits.
OCR for page 114
94 Evaluating the Effectiveness of Offshore Safety and Environmental Management Systems
Conclusion 15: Because BsEE will have access to all sEMs programs,
including audit and follow-up reports and its own inspection and
audit reports, the agency is in the best position to gather and analyze
this data to identify best practices and common trends (good and bad),
and disseminate this information to the regulated community in a
timely manner. BSEE auditing of SEMS programs will encompass a
wide range of different operators that are using a variety of approaches
to SEMS. Some of these programs may be more effective than others and
will provide valuable lessons that can be used to improve SEMS for the
entire industry.
RECoMMEndEd AppRoACh
BsEE should establish a holistic combination of methods necessary to
ensure the effectiveness and continuous improvement of sEMs pro-
grams. BsEE should establish a system that employs a combination of
compliance inspections, audits, key performance indicators (Kpis), and
a whistleblower program to ensure that sEMs programs are adequate,
in place, and operating effectively and in a manner that promotes a
culture of safety among the operators. The committee did not reject
outright any of the approaches presented in Chapter 3 as being of no
value, and BSEE could employ a combination of all the methods presented
in Chapter 3. However, on the basis of the experience of representatives
from comparable regulatory agencies (see Chapter 4) and of the committee
members’ personal experience and expertise, the committee selected a
combination of those approaches for which it believed some evidence of
success was available. These approaches, the committee believed, would
result in the most effective evaluation with the resources available. The
technology and data necessary to use the SEMS monitoring sensors and
calculation of risk methods described in Chapter 3 are not available at
this time.
The following sections of this chapter provide specific details of
the recommended approach. Specific recommendations are given in
boldface. The first section describes the inspection criteria and proce-
dures that BSEE could use to verify compliance with specific regula-
tions and to uncover obvious deficiencies in the implementation of
OCR for page 115
Conclusions and Recommended Approach 95
the operator’s SEMS program. The second section discusses audits
and describes a process by which BSEE could ensure that operator-
initiated audits are complete, accurate, and effective (i.e., an audit of
the audit process). The final sections summarize recommendations for
a KPI program and a whistleblower program. Together, these methods
constitute a holistic system that BSEE should employ to evaluate the
effectiveness of SEMS.
BsEE sEMs inspections
BSEE should continue its current program of ensuring compliance with
specific regulations. However, an inspection of sEMs (scheduled or
otherwise) should not be focused solely on what is not in compliance;
rather, the inspection should attempt to obtain a holistic view of the
facility’s safety culture. Does the operating company empower its per-
sonnel to take corrective action? Does it provide the resources necessary
to do so? Are facility personnel only focused on the items they know
an inspector typically will look at, and do they ignore the rest? A well-
trained evaluator should be able to look beyond the black-and-white
regulation and identify operators in marginal compliance, in order to
guide them into a more complete state of compliance.
BSEE inspectors have been trained to measure compliance with a
standard or prescriptive checklist and without further training are not
equipped to measure the effectiveness of individual SEMS programs. It
may not be practical to expect the current BSEE inspection force to make
subjective decisions as to whether a SEMS program is working correctly
so that it meets the intent of the SEMS regulation and helps create a
culture of safety. Current BSEE inspectors will have a tendency to issue
incident of noncompliance (INC) notices for deviations of documenta-
tion from a checklist, and such deviations may or may not be important
in meeting the intent of SEMS. In turn, the issuing of INCs may focus
operator attention on compliance in the way documentation is written
rather than on establishing a culture that actually promotes safety.
Most injuries and blowouts on offshore oil and gas facilities are not
usually caused by mechanical failures identified by INCs (NRC 1990).
Thus, to the extent possible, BsEE should train inspectors to employ
other options in addition to using prescriptive checklists and issuing
OCR for page 116
96 Evaluating the Effectiveness of Offshore Safety and Environmental Management Systems
inCs. ideally, BsEE inspectors should look beyond the written regula-
tion to identify operators in marginal compliance and guide them into
a more complete state of compliance. Although it may be difficult for
BSEE inspectors to identify operators in marginal compliance when it
comes to assessing the adequacy of a SEMS program, it is not unreason-
able to expect them to make overall observations, which, in turn, could
help focus BSEE-initiated SEMS audits.
During presentations and site visits1 the committee was told that, in
some cases, BSEE inspectors visiting an offshore installation might be
able to spot problems inherent in an operator’s safety culture by noticing
obvious safety issues such as loose handrails, corroded walkways, or staff
not wearing the appropriate personal protective gear. Other situations
that indicate problems in the safety culture might be much harder to
notice and require an in-depth investigation of safety-related approaches
and practices, not only at the installation, but also in the operator’s overall
operations, both offshore and onshore.
BSEE inspectors need to spend enough time on a facility to observe
multiple activities. To avoid the appearance of a conflict of interest, BSEE
inspectors are not generally permitted to travel on operator-furnished
helicopters, eat food provided by the operator while on the offshore
facility, or stay overnight in operator-furnished quarters.2 Every other off-
shore regulatory regime the committee talked to uses operator-furnished
transportation, catering, and accommodations. The California State
Lands Commission made a point of saying that being able to talk to the
crew when its personnel travel to an installation in operator-furnished
transportation is extremely beneficial in determining what is really going
on at the installation. All regulatory bodies consulted by the committee
agreed that time spent offshore in operator-furnished accommodations
is essential to understanding the culture of safety on the facility. BSEE
inspectors currently spend an extensive amount of time traveling to and
from offshore installations. A more efficient use of available manpower
would be to use the extensive amount of travel time to and from offshore
P. Schroeder. Pacific OCS Regional Office, BSEE. Presentation to a subgroup of the committee,
1
Camarillo, California, March 22, 2010.
Inspectors do stay overnight on some facilities that are very far from land.
2
OCR for page 117
Conclusions and Recommended Approach 97
installations for informal discussions with operator personnel before a
formal audit or visit to an offshore installation.
Furthermore, BSEE has a finite budget, and that budget should be
maximized so that the inspectors’ work can be fully effective. A large
portion of BSEE’s budget is allocated to offshore transportation costs,
and the rules BSEE inspectors must adhere to became more stringent
following the Macondo well accident. It would be beneficial to identify
a way to minimize these costs and reallocate some of these resources to
the hiring and retaining of highly capable staff. BSEE should aspire to
having its inspectors and engineers be recognized as being among the
more highly qualified people in the offshore industry.
Therefore, to the maximum extent practicable consistent with
its increasing safety, environmental, and auditing responsibilities,
BsEE, with industry input, should analyze the benefits and risks of
using operator-furnished transportation and accommodations when
performing inspections and audits. The agency should plan its bud-
get recognizing that per dollar spent, the safety value of ensuring that
high-quality inspectors and auditors are recruited and appropriately
compensated and that critical data are collected, stored, and analyzed
is greater than maintaining a completely independent transportation
capability. Allowing overnight stays would increase the time BSEE staff
would be able to spend interacting with the operating crew. More time
on an installation would enable inspectors to better judge the degree to
which a safety culture exists there. All other offshore regulators that the
committee heard from believed that these measures would lead to bet-
ter inspections and a higher degree of operating safety and, with proper
management, would not lead to conflicts of interest. in the analysis,
consideration should also be given to increasing the fees charged for
inspections and to changing the structure by which inspection (and
audit) costs are passed on to industry.
Finally, as noted in Conclusion 4, operating conditions, personnel,
production requirements, and technologies are continually changing.
Therefore, BsEE should design and implement its inspection program
on the basis of risk. The use of a risk-informed framework that focuses
attention where new or potential problems are likely to occur will aid in
the evolution of practices and audit procedures.
OCR for page 118
98 Evaluating the Effectiveness of Offshore Safety and Environmental Management Systems
operator and BsEE Audits
Besides the inspections described above—which are, in essence, checks
of compliance with specific regulations—and spot checks to determine
whether individual elements of SEMS are being used effectively, a system
for evaluating the effectiveness of an operator’s SEMS program requires
routine periodic as well as incident- and event-driven audits. For these
audits to evaluate the effectiveness of a SEMS program successfully,
auditors will need to understand how the organization’s safety culture
is reflected in its implementation of its SEMS program. Developing this
understanding will require auditors to interact with operating crews
and ask questions pertinent to how well crew members understand the
SEMS program and how well used the program’s elements are in day-to-
day practice. As noted in Chapter 3, techniques used in inspections (e.g.,
interviews and witnessing) should also be used when conducting audits.
BSEE is responsible for ensuring that audits are conducted in a timely
fashion, are thorough, and accomplish the goals set out above. The audit
scheme adopted by BsEE should have the following characteristics:
operator ownership, audit team independence, training and accredita-
tion of auditors, access to top levels of management and audit reports,
a definition of required audit frequency, and a scheme for quality
assurance of audits. BSEE will also need to audit operator audit reports
and have personnel capable of carrying out these tasks effectively. Except
in the case of highly deficient systems, the goal of the audit should
not be to pass or to fail. Rather, an audit should be designed to help
senior management by presenting them with an independent view
of the state of their efforts to establish a safety management system
and, by extension, a culture of safety. The audit should identify areas
for improvement and measure progress toward improvements rec-
ommended in previous audits. In its program, BSEE should take into
account that safety management is a dynamic process that evolves with
time and that should not be judged solely on a pass–fail system.
Operator Ownership
BsEE should ensure that operators are involved in the audit itself.
Several members of the committee have participated in financial, qual-
ity, and safety audits and have observed that properly conducted inter-
OCR for page 119
Conclusions and Recommended Approach 99
nal audits by personnel familiar with the operation are much better at
uncovering problems than are external audits. In discussions with the
committee, a representative of the Center for Offshore Safety (COS)
stated that his company operated on this principle in its safety audits and
that if BSEE mandated third-party audits, his company would merely
add the third-party audit as a regulatory requirement to the internal
audits it was already conducting. His company would not accept the
third-party audit as a substitute for one of its internal audits.
Large operators such as ExxonMobil and Shell have the ability to
form independent audit teams within their organization. There are many
smaller operations, however, that do not have a pool of skilled personnel
from outside the operating organization being audited who are capable
of performing an adequate audit of the organization. These operators
should be able to use third-party auditors. if an independent third party
must be used, at least one member of the audit team should be from
the operator’s organization, and that person should not be directly
involved in the day-to-day operation being audited. In some cases of
very small operators with only a handful of employees, it may be neces-
sary for the chief executive officer of the company to participate as a
member of the audit team.
Although operators are responsible for conducting audits, BSEE is
responsible for verifying that quality audits are carried out and acted
on appropriately. BsEE should perform complete or partial audits of
sEMs programs when justified by reports from inspectors, reviews of
operators’ audit reports, incidents, or events. BSEE’s oversight of inter-
nal and third-party audits needs to include a range of techniques, each of
which focuses on a different aspect of an operation’s safety system. BsEE
can use reports from its compliance inspectors and its reviews of audit
reports to identify the need for specific BsEE-conducted targeted or
spot audits, or complete audits, to determine whether an operator’s
sEMs program is improving safety. Direct spot inspections to verify
that specific requirements are being met could perhaps be accomplished
by relying on checklists. BsEE can also check to determine whether an
organization’s sEMs program is improving its safety culture. Inter-
views, demonstrations, and observations, rather than checklists, are nec-
essary to make such a determination. For example, the question, “Do you
OCR for page 120
100 Evaluating the Effectiveness of Offshore Safety and Environmental Management Systems
have an MOC (management of change process)?” would shift to, “Show
me how you know if your MOC is working properly.” Operators with an
effective safety culture will be able to answer that question—and, in fact,
describe the possible weaknesses in the MOC as implemented—even if
it meets the letter of law.
Audit Team Independence
The operator’s audit of its SEMS program should be conducted by a
truly independent, qualified team. It is critical that the audit team be
made up of members who are divorced from the organization within
the company that is responsible for the day-to-day operations of the
installation and for meeting the financial, operational, safety, or envi-
ronmental targets set by management. The audit team should report to
the highest level practical given the size and complexity of the opera-
tor’s organization. Members of the audit team could be permanently
assigned or assigned on a rotating basis for a set number of years. The
latter method helps disseminate information about and respect for the
audit team throughout the organization. Participation in the audit team
may also be considered as an interim assignment to higher level opera-
tions assignments within the company. In no instance should the audit
team include as a member someone who was recently assigned to the
offshore facility being audited.
Training and Accreditation of Auditors
Audit team members should be trained to conduct audits and should
be accredited by a method prescribed by BSEE.3 General qualifications
for SEMS auditors are described in more detail in Chapter 5.
BSEE must have independence (from industry) in how it trains its audi-
tors. Nonetheless, the agency should consider certifying its auditors using
a process similar to that used to certify industry auditors, and the certifica-
tion should be of the same standard as outside accreditation institutions.
BSEE, in consultation with the industry and, potentially, COS, should
develop an approach to certify auditors, develop audit standards, and
establish the process by which audits themselves are conducted.
This recommendation is supported by the National Academy of Engineering–National Research
3
Council (NAE-NRC) report on the Macondo Well–Deepwater Horizon blowout (NAE-NRC 2011).
OCR for page 121
Conclusions and Recommended Approach 101
Access to Top Levels of Management and Audit Reports
The key to a successful audit system lies in discussions with top man-
agement and in the steps top management takes toward continuous
improvement. BsEE should ensure that the audit team has reviewed
the audit report with top levels of management and obtained their
sign-off on findings and areas for improvement. A copy of the audit
report and a summary of its findings and conclusions should be sent
to BsEE so that the agency can spot trends and disseminate informa-
tion to the industry in a timely manner.
As an alternative, BSEE could consider allowing COS to screen all
reports. Doing so would bring in an element of the peer-review–peer-
assist method for assessing effectiveness that is described in Chapter 3
and would further involve the industry as a whole in taking ownership of
the development of a culture of safety. Such a charge from BSEE to COS
would be consistent with the following elements of the COS operating
basis as presented to the committee:4
• Compiling and analyzing key industry metrics,
• Sponsoring functions to facilitate sharing and learning,
• Identifying and promoting opportunities for the industry to continu-
ously improve,
• Interfacing with industry leaders to ensure leadership and system
deficiencies are recognized and addressed promptly, and
• Communicating with government and external stakeholders.
Audit Frequency
Under 30 CFR 250, Subpart S, as it currently stands, the timing for audits
is very prescriptive. An operator must audit every element in its SEMS
program every 3 years and include at least 15 percent of its installations.
Installations in the Gulf of Mexico are very diverse. There are single-
well unmanned installations, manned and unmanned installations with
production equipment and no wells, manned and unmanned installations
with both wells and production equipment, platforms with simultane-
ous drilling and production operations, floating and bottom-supported
J. Toellner. Center for Offshore Safety. Presentation to the committee, Houston, Texas, October 19,
4
2011.
OCR for page 122
102 Evaluating the Effectiveness of Offshore Safety and Environmental Management Systems
platforms, platforms producing 200 barrels of oil a day and platforms
producing 200,000 barrels of oil a day, and all manner of drilling and
workover rigs. Some installations produce high-pressure oil, which has
the potential of flowing at high rates to the surface; others require pumps
to lift oil to the surface. Some facilities produce natural gas with few impu-
rities; others produce gas that contains levels of acid gases such as carbon
dioxide and hydrogen sulfide. Some drilling operations are conducted
in well-defined subsurface environments, while others are geologically
uncertain. Some facilities have old equipment; others are new.
Similarly, operators in the Gulf of Mexico are very diverse. Some are
large, international, integrated companies; others are large, domestic
energy and petroleum companies; and still others are very small indepen-
dent operators with only minimal staff. Some operators are responsible
for a large number of fields and installations, and some operate only one
or two fields and installations. Some fields have multiple platforms tied
together by pipelines or bridges, or both, and some have just one platform.
Thus, it is difficult to establish a formula for audit frequency that does
not become a paperwork burden and exercise for some operations while
it is at the same time too lenient an audit frequency for others. Neither
result is conducive to using SEMS to help establish an improved culture
of safety in the industry.
Because of the diversity of operations and operators, each opera-
tor should be allowed to develop its own audit plan, subject to B sEE
approval. Operator development of the audit plan would be a further
step in establishing operator ownership in SEMS and its implementa-
tion and would replace the current prescriptive frequency with a more
appropriate risk-based audit frequency.
Quality Assurance of Audits
In any system involving audits, BSEE is responsible for monitoring
the quality of the audits and for ensuring that what is learned from the
audits is implemented. Under 30 CFR 250, Subpart S, as it currently
stands, BSEE accomplishes this task by requiring that operators submit
their audit plans before conducting audits, submit the qualifications of
audit team members and third-party audit companies, and submit the
results of the audits once they have been completed. The assumption is
OCR for page 123
Conclusions and Recommended Approach 103
that BSEE will review and approve all submittals and will disseminate to
the industry what is learned from the audits. in addition to requiring
operators to submit an audit plan to BsEE, the agency should further
require them to identify and report the follow-up actions taken as a
result of the audit. BsEE should require that the frequency and scope
of the audits specified in an operator’s audit plan be guided by risk
rather than by a one-size-fits-all formula.
Auditing Audits
in addition to conducting spot inspections of sEMs compliance, BsEE
should audit the quality of an operator’s audits. This task includes
performing spot inspections of documentation and audits and, where
appropriate, more complete BSEE reaudits of specific facilities. BSEE
should also have a plan for carrying out these activities. in its audits,
BsEE should use objective and subjective risk-based processes such
as those employed by petroleum safety Authority (psA) norway, and
these audits should be carried out by BsEE employees who are them-
selves accredited.
Personnel for Auditing
BsEE should hire or train a sufficient number of auditors, including
qualified audit team leaders and an adequate number of staff for analyz-
ing audit reports effectively and auditing the accreditation system that
the agency puts in place.
Key performance indicators
Specific KPIs associated with SEMS effectiveness are difficult to define
and need further study and evaluation by both the industry and BSEE.
Common safety and environmental metrics such as the number of inju-
ries per year or the volume of spills per year provide only a part of the
effectiveness picture. Other metrics need to be identified as lagging or
leading indicators in relation to process safety. Once identified, such
metrics can be used to monitor and direct the improvement of SEMS.
BsEE can collect and evaluate data from operations within and
across installations to identify specific problems and trends in opera-
tions at a particular facility and across the industry. This information
OCR for page 124
104 Evaluating the Effectiveness of Offshore Safety and Environmental Management Systems
is needed to evaluate the SEMS audit approach and to identify opportu-
nities for improvement. While the benefits from such a data exchange
are obvious and important, implementation is far from trivial. An open
data-collection and data-sharing protocol requires agreements across
all parties to ensure that confidentiality and legal concerns are satisfied.
BsEE should distribute information in a timely manner to the industry
on trends and methods for improving the sEMs process and overall
safety, as well as lessons learned, by means of publications, workshops,
seminars, and other methods.
Offshore safety organizations abroad that have programs similar to
SEMS, such as PSA Norway and the United Kingdom (UK) Health and
Safety Executive (HSE), have access to a considerable amount of data.
Because many of the safety and environmental issues associated with
offshore oil and gas operations are common worldwide, a data set com-
piled from all of these organizations would be invaluable. BsEE should
create a task force with the industry, psA norway, the uK hsE, and
other similar regulatory bodies worldwide to identify Kpis. Creation
of such a task force will help BSEE ensure that it is collecting the proper
SEMS-relevant data and analyzing it appropriately to direct the agency’s
effort to measure the effectiveness of SEMS.
Whistleblower program
PSA Norway, the Occupational Safety and Health Administration, and
other organizations have found that programs that allow personnel to
anonymously report possible violations directly to the regulator are
helpful in identifying possible issues that may not be found by other
means. The SEMS II notice of proposed rulemaking (BOEMRE 2011a)
describes an approach that provides for anonymous reports of potential
violations. BsEE should have a program for anonymous reporting and
a process to follow up such reports and should use the information
gained from them appropriately to modify BsEE inspections and
audits. This program should also allow for the anonymous reporting
of inappropriate behavior of BSEE personnel and potential improve-
ments in BSEE policies and procedures, as well as potential violations
by operators. Care should be taken in devising the program to make
sure that it does not become a tool for disgruntled employees seeking
OCR for page 125
Conclusions and Recommended Approach 105
to punish perceived wrongs. This recommendation supports Summary
Recommendations 5.4 and 6.14 in the NAE-NRC report on the Macondo
well–Deepwater Horizon blowout:
Industry, BSEE, and other regulators should improve corporate and
industrywide systems for reporting safety-related incidents. Reporting
should be facilitated by enabling anonymous or “safety privileged” inputs.
Corporations should investigate all such reports and disseminate their
lessons-learned findings in a timely manner to all their operating and
decision-making personnel and to the industry as a whole. A comprehen-
sive lessons-learned repository should be maintained for industrywide use.
This information can be used for training in accident prevention and con-
tinually improving standards. (NAE-NRC 2011, pp. 107 and 123)
REsouRCEs REquiREd
BsEE should analyze its budget to ensure that it has sufficient finan-
cial resources to implement these recommendations. Savings from any
increased use of operator transportation and more efficient time offshore
derived from operator-furnished accommodations could potentially be
reprogrammed for the agency’s enhanced inspection and SEMS audit pro-
grams and other related activities required by these recommendations.
