Lessons Learned from Airport Safety Management Systems Pilot Studies (2012)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

35 BACKGROUND FAA states in its NPRM, “Safety Assurance is a set of processes that monitor the organization’s performance in meeting its current safety standards and objectives as well as contribute to continuous safety improvement.” Safety assurance essentially serves as a checks-and-balances system to ensure that SMS processes and procedures and risk mitigation strategies are having their intended effect (FAA 2010). ICAO states, Assurance can simply be defined as “something that gives confidence.” The Safety Risk Management process in the SMS starts with the organization obtaining a good under- standing of its operational processes and the environments in which it operates; progresses through hazard identification, safety risk assessment and safety risk mitigation, and cul- minates in development and implementation of appropriate safety risk controls (ICAO 2009). As a further explanation, The primary task of Safety Assurance is control. This is achieved through safety performance monitoring and mea- surement, the process by which the safety performance of the organization is verified in comparison with the Safety Policy and approved safety objectives. Safety assurance con- trol is conducted by monitoring and measuring the outcomes of activities that operational personnel must engage in for the delivery of services by the organization. [Therefore,] a process of permanent examination, analysis and assessment of these controls must continue throughout the daily opera- tion of the system. The Safety Assurance process mirrors that of quality assurance, with requirements regarding analysis, documentation, auditing, and management reviews of the effectiveness of the safety risk controls (ICAO 2009). The ACRP Guidebook states, One of the core concepts addressed by SMS is continu- ous improvement. The elements grouped under this pillar [component] provide the tools to accomplish that. This includes ensuring that all measures put in place are adhered to, reviewing and evaluating the actions taken to ensure that they are producing the desired effects, and monitoring busi- ness activities and their impact on safety to help determine where your efforts should be directed. Safety Assurance differs from SRM because the target of Safety Assurance is to identify and evaluate deficiencies and improve the perfor- mance of the system, instead of looking at individual hazards and associated risks. The focus of Safety Assurance is the effectiveness of the SMS (Ayers et al. 2009). To focus on the principle of continuous improvement, SMS requires a strong and comprehensive Safety Assurance pro- gram that includes “ . . . self-auditing, external auditing, and safety oversight. Safety oversight can be achieved through auditing and surveillance practices. Safety Assurance aims to ensure that the activities, plans, and actions taken to improve safety are implemented and effective” (Ayers et al. 2009). CHAPTER CONTENTS This chapter presents findings relating to the safety assurance component, including plans for expanded inspections to the ramp area and the baggage makeup area, audit and evaluation program development and use, and safety policy objectives metrics and measurements. Within the required tasks documented in the FAA’s Par- ticipant Guide for Part 139 SMS Implementation Study, the FAA requested that airport participants “Conduct an internal audit/evaluation following the methods and procedures pre- scribed under the Safety Assurance component of the airport’s SMS manual or applicable documentation and report the findings of its internal audit/evaluation” (FAA 2011). The ACRP Guidebook defines internal audits as “an internal inspection or assessment of the activities, systems, and pro- cesses used by the organization related to safety and the SMS” and “periodic assessments, audits, and inspections . . . is very similar to the gap analysis prior to SMS implementation, with the difference that an assessment should check the overall per- formance of the system and identify areas for improvement” (Ayers et al. 2009). Owing to the short duration of the SMS implementation study, the ability of participating airports to conduct audits was restricted based on the limited amount of data that could be collected within the 13-month pilot study time frame. Thus many airports did not conduct formal SMS audits and, instead, performed SMS evaluations to assess the current state of SMS components. FINDINGS AND ANALYSIS All Part 139 airports conduct daily inspections. Within SMS, inspection data are used (now or in the future) by airports as inputs into safety assurance audits. Questions relating to the chapter five SAFETY MANAGEMENT SYSTEM AND SAFETY ASSURANCE

36 extension or expansion of the Part 139 program were asked to assess whether airports were considering or had modified the current inspection program to encompass a broader reach into the ramp areas and the terminal (baggage makeup areas specifically). As mentioned previously, airports interviewed were in the preliminary stages of SMS implementation and, though they were able to discuss current inspection programs, they were not yet in a position to thoroughly discuss audits because of the lack of available data. Inspection Programs In shown in Figure 18, implementation and nonimplemen- tation airports were asked whether, through the SMS, they intended to extend or expand the current inspection program to the ramp. Four airports reported that they had existing detailed ramp inspection programs currently in place. Fifteen airports reported that they had or planned to extend the inspection program to the ramps and seven responded “no” they had not extended inspections to the ramps. As outlined in the FAA’s NPRM for implementations, the FAA states that with regard to regulation of the nonmovement area, Under this proposal, an airport would implement its SMS throughout the airport environment, including the movement and non-movement areas (including runways, taxiways, run- up areas, ramps, apron areas, and on-airport fuel farms). The FAA acknowledges the proposal extends the scope of part 139 by including the non-movement areas, but the FAA has concluded that ensuring safety in air transportation requires that an SMS applies to any place that affects safety during aircraft operations. The FAA later states, “[it] does not intend to require air- ports to extend their SMS to the landside environment such as terminal areas. Nevertheless, an airport may voluntarily expand its SMS to all airside and landside environments” (FAA 2010). Airports were queried whether they planned to extend the inspection program to the baggage makeup area. The three airports that had participated in the proof-of-concept study were asked to investigate the inclusion of the baggage makeup area in the SMS program as a means to manage ground ser- vice provider activities away from the ramp. One of the three airports that participated in the follow-on/proof-of-concept study has begun to implement a focused baggage makeup area inspection program as part of the SMS to address safety concerns such as speeding and FOD originating in ground service provider, staff, and tenant baggage carts and tugs. Of the implementation and nonimplementation airports, the response results (Figure 19) include eight “yes,” 13 “no,” two “not sure,” and three indicated that their general aviation airport had no baggage makeup area and therefore the ques- tion was not applicable. Airport comments regarding the extension of the program to the terminal and landside included statements that planned airport-wide reporting and management will require consistent reporting throughout the airport organization and that restrict- ing the safety program to the airside was not consistent with the safety policy and objectives. Also one airport commented that training staff and tenants to be safe in a certain way on the airside and another way in the terminal and landside areas would lead to confusion and inconsistent reporting; safety reporting was planned to encompass all airport activities regardless of location. Audit and Evaluation Programs and Quality Assurance The remaining questions relating to the safety assurance component of SMS focused on the audit and evaluation por- tion of the SMS program. Airports were asked whether as FIGURE 18 Inspections extended to the ramp.

37 part of their SMS they developed an evaluation or audit pro- gram. One of the deliverables in the SMS implementation study group was to conduct an program evaluation or audit. Therefore, expectations were that all 14 SMS implementation study airports participating in the survey would reply “yes” to the question. However, as shown in Figure 20, nine airport respondents replied “yes” to having developed an SMS eval- uation or audit program, 13 responded “no,” and the remaining four commented that the program evaluation and audit was “being developed.” Assuming that some of the airports responding “yes” to developing an SMS evaluation or audit program would even- tually perform the evaluation or audit, airports were asked if they had completed the task. As shown in Figure 21, one of the respondent airports replied “yes” to having conducted an SMS evaluation or audit, five replied “no,” and 13 SMS implementation study respondents replied, “not yet.” Only one of the total group had completed an audit or evaluation; however, owing to the timing of the Synthesis interviews (August–September 2011) most of the SMS implementation study airports had not yet completed the audit and evaluation task and report and replied, “not yet.” Data Collection In an effort to link program audit and evaluation to the safety policy objectives, airports were asked to provide information on data elements that they intended to collect to measure SMS performance. For the majority of airports surveyed, they were in the early stages of compiling existing data and formalizing hazard reporting, including procure- ment or development of software or database systems to assist in reporting, collecting, or trend analysis. Figure 22 represents responses of nine airports stating “none,” mean- ing no data elements have been identified to measure perfor- mance, seven responding “not defined yet” or “not initiated,” and the remaining 11 replying “using a formal system” and “accidents/incidents/wildlife.” FIGURE 19 Inspections extended to baggage makeup. FIGURE 20 Evaluation or audit program developed.

38 As part of the implementation study, some airports pur- chased or developed data or software systems. Although individual airport efforts varied, these systems were typi- cally developed or purchased to track, monitor, and report on hazards, accidents, incidents, risk, mitigations, non-Part 139 inspections, and audits. A variety of industry systems and software products exist to support the SMS. Table 22 pres- ents airport respondents’ plans for collecting and trending data, including a list of software solutions. For some smaller airports, spreadsheets, MS Access databases, and paper- based tracking were reported. Other airports are considering expanding existing systems such as Maximo or their Airport Security and Operations Compliance System or have pur- chased products such as Eagle, E-risk, Intelex, OAG Incident Reporter, SMSPro, TRA, and so forth. FIGURE 21 Evaluation or audit conducted. LESSONS LEARNED Inspections Airports conduct daily inspections for Part 139 compliance; extending inspections to the ramp and to other non-airside locations is being considered by many airports for a vari- FIGURE 22 Data elements to measure performance. TABLE 22 SOFTWARE AND OTHER MEANS TO COLLECT DATA Class How Do You Plan on Collecting and Trending Data? Class I Airport Security & Operations Compliance System (ASOCS) Class I Current asset management system Class II Custom developed Class I Custom mobile reporting on SMS database for safety critical systems Class I Class I Eagle software Class I ESIS e-risk Class II In-house custom-built system Class I Class I Maximo Class I Class I OAG Incident Reporter Class IV OAG Incident Reporter Class II Paper-based reporting Class IV Paper-based reporting Class IV Paper-based reporting Class II SMS Pro Class I TBD could be in-house or purchased Class I TRA software Class I Using an in-house Access database Class IV Using Excel Custom online database Intelex software Multiple programs

39 ety of reasons, including consistent reporting and program management. Lesson Learned: Expansion of the SMS program to the ter- minal and landside operations might be considered in addi- tion to the movement and nonmovement areas as the SMS program is being designed and developed. Audits Audit results benefit from a solid baseline set of data and a few year’s worth of information to assess trends. Lesson Learned: Conducting a program audit in Year 1 can be challenging. Performing a program evaluation where incremental milestones have been achieved may be of greater use to management, especially with regard to assessing SMS policy objectives and goals. Data Collection Data collection and use as part of the safety assurance com- ponent of the airport SMS are being addressed by airports using fit-for-purpose solutions such as commercial off-the- shelf software, custom application development, and paper- based systems. Data collection at some airports is currently or is planned to include the landside and terminal areas for continuity of management and reporting. Lesson Learned: Data collection and trending are at the core of safety assurance and data collection solutions vary depending on airport size and SMS program scope.