Airport Safety Risk Management Panel Activities and Outcomes (2016)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

37 As described in earlier sections, several techniques and approaches have been used by airports to conduct SRAs successfully. Current practice by Part 139 airports is to have a multidisciplinary panel of stakeholders when evaluating complex safety issues affecting airside stakeholders. The panel is led by a facilitator during a brainstorming session when a cluster of techniques is used, including the preparation of material to support the assessment, the use of the 5M Model to describe the system, a structured or unstructured brainstorming to identify hazards, a risk matrix to assess risks, and additional brainstorming with the panel to evaluate feasible actions to mitigate risks further. Part 139 airport staffs are gradually becoming familiar with these techniques, but a few gaps were identified during the survey and interviews for this project. This chapter addresses some of the gaps related to techniques frequently used by airports to identify hazards and assess risks. The goal of the hazard identification step in the SRA process is to identify as many hazards as pos- sible that are applicable to the operation, within the scope of the risk assessment. Most of the SRAs with panels have used facilitated brainstorming techniques to identify hazards. This is the second step of the SRM process recommended by the FAA; however, the airport industry has used other techniques described in this section. HAZARD IDENTIFICATION CONCEPTS According to H. de Jong (NLR-CR-2004-094—EUROCONTROL/NLR 2004), “experience shows that hazard identification brainstorming sessions are a rich source of hazards, not only in quantity but also in quality: brainstorming sessions often yield hazards that would not easily be obtained by other means, such as the functional approach to hazard identification in functional hazard analysis (FHA). Such functionally unimaginable hazards could not have been obtained by logical thinking in terms of functions and failures, but their identification depends in an essential way on the creativity of operational experts.” It is not possible to state that any of the hazard identification techniques described in this report will lead to the identification of each and every hazard involved with an airport project, a proposed change, or even accidents and incidents. Therefore it is necessary that hazard identification be peri- odically reviewed, particularly during the implementation of a project or change. In addition, the aviation system involves many complex interactions between the technical and human components operated by the airlines, the airport, air traffic control, and ground support organizations. Each of these stakeholders must manage the risks that are under its control; however cooperation between those parties will help the coordination and management of interactions and interfaces. When performing a brainstorming session, the participation of representatives of functions affected by the safety issue is important. Adapted from the work by de Jong (2004), Figure 16a illustrates a brainstorming session with representatives of only one airport function (e.g., operations). The white circle represents the entire system defined in the SRA; the small hashed circles represent hazards identified; and the dashed circle represents the function assessed for hazards. It is likely that the hazards identified are only those related to the specific airport function conducting the SRA and that many hazards will be missed. chapter five HAZARD IDENTIFICATION AND RISK ASSESSMENT TECHNIQUES

38 FIGURE 16 (a) Hazard identification with one function present (e.g., engineering); (b) hazard identification with additional function (e.g., operations and engineering); (c) structured hazard identification; (d) hazard identification with poor definition of the system (adapted from NLR-CR-2004-094 2004). (a) (b) (c) (d) In Figure 16b, another airport function is included in the brainstorming. In this situation, new hazards are identified, not only those associated with the second function but additional hazards derived from the interactions between the two functions in the brainstorming process. When more functions are included in the hazard identification, the scenario is changed and it will cover a much larger spectrum of hazards. Moreover, when the brainstorming process is structured to cover project/change implementation phases and activities, a broader region of potential hazards is covered and the process becomes more systematic and comprehensive, as illustrated in Figure 16c. One effective way to improve current SRA techniques used by Part 139 airports is to structure the hazard identification by outlining the different parts or sub-tasks of the change being examined. Breakdown may be by function, chronological order, operational phases, or other factors. This breakdown will help with the systematic identification of risks. An example of structuring a risk assessment is presented here. The airport was having problems with the aircraft towing service provided by airport staff. A number of incidents resulting in minor damage to aircraft had occurred, and the airport wanted to take measures to reduce risk. To help with the identification of hazards, the aircraft towing process was divided into different tasks: • Prepare work order, • Assign tow crew, • Select tow equipment, • Position tug and hook up, • Tow aircraft, • Maneuver aircraft in/out of hangar, and • Park and secure aircraft. By dividing the process into tasks, it became easier to identify hazards in each task and the pos- sibility of missing important hazards was reduced. Another important conclusion is that the system must be well defined, otherwise the panel will not be able to see and discuss hazards outside the limits of the defined system, as shown in Figure 16d.

39 COMMON HAZARD IDENTIFICATION TECHNIQUES Based on ACRP Report 131 (2015), the most common hazard identification techniques used by airports are: • Brainstorming, • Checklists, • Safety performance indicators (SPI), • Hazard, incident and inspection reports, • Structured What-if (SWIFT), and • Safety audits. It is important to note that these techniques are applied in all SRM processes used by airports; however, some of those techniques are not used commonly by SRM panels. Reports, SPIs, and audits are considered data-driven methodologies, and information normally is recorded through systematic processes that allow tracking and further analysis. The other methodologies, classified as qualitative, can be generated in a formal process such as an SRA, or an informal process based on discussions and interviews; these methodologies are considered heuristic processes based on judgment by expe- rienced people or experts. Hazard identification should be as comprehensive as possible and take into consideration design, organizational, work environment factors, procedures, and operating practices (ICAO SMM, 3rd ed. 2013). The FAA-EUROCONTROL Toolbox (2007) recommends that identification of hazards be done by an individual or group-based assessors. The main challenge of both approaches is to identify hazards that exist but are difficult to recognize. Other hazard identification techniques have been developed to help overcoming this type of deficiency; however, the use of those methods may require specific expertise not commonly available at airports. In the individual approach, one or two assessors trained in SRM take the responsibility of identi- fying routine hazards that normally represent the majority of hazards at an airport. In most cases, the SMS coordinator and assistants are responsible for this role. In addition, this method is helpful for a preliminary and high-level identification of hazards. The main question to be asked in the individual approach is, “What can go wrong?” The group-based approach involves stakeholders and experts for the hazard identification exer- cise. This is the basic PHA methodology used during SRAs supported by panels and managed by a facilitator. At smaller airports, the groups will be small but each department should be represented. A participant may represent two or more airport functions. Some hazards are difficult to identify (EUROCONTROL 2003) regardless of which approach is used, particularly those hazards associated with situational awareness issues. In certain situations a relevant function for the safety of the operations is implicit, or the description of the system is not complete and some “unimaginable hazards” may be present. According to Bircham (2015), one of the most productive means of encouraging cooperation between management and the rest of the airport workforce is to share the information resulting from the assess- ment with all who might be exposed to the risk. This provides management an opportunity to explain the precautionary risk control measures being considered, and to involve airport staff in their development. Wherever possible, a team exercise involving different levels of personnel and stakeholders will help strengthen the feeling of ownership and help develop a positive safety culture in the airport organization. Unless the airport manager responsible for dealing with eventual outcomes has a good understanding of the processes involved, the manager will have to implement recommendations under blind conditions. When an individual is responsible for the risk assessment, the work is done faster. However, that individual needs to have substantial experience. The most common problem with this approach is that it may perpetuate “safety is the SMS Department problem.” In most cases a risk assessment conducted by an individual is not as comprehensive as when a team is involved. Personal perception may be biased. However, when quick reaction to address a hazard in a dynamic environment—such as an airport—is necessary, it is not always possible to convene a panel to assess risks.

40 Brainstorming “Brainstorming is an unbounded but facilitated discussion within a group of experts and stakeholders where the facilitator encourages imaginative thinking” (ECAST 2009). There are two basic rules for hazard identification in brainstorming sessions: • Identify as many hazards as possible; and • Criticism and analysis are forbidden during the session. Advantages: • Good for identifying new hazards in unique systems • Involves airport stakeholders impacted by the safety issue or change • Relatively simple and easy to run • Airport staff with little experience can participate • Can be applied to many airport safety issues • Shared responsibility of outcome. Disadvantages: • In most cases it is relatively unstructured and may not be comprehensive • Depends on the expertise, experience, and profile of participants • Strong influence of group dynamics and experience of facilitator • Can be time consuming • Ties up airport and stakeholder resources • Scheduling can be a challenge when large groups are involved. Brainstorming is the typical SRA technique used to identify hazards by Part 139 airports. EUROCONTROL (2004) suggests that a group of four to six people is ideal for brainstorming. However, surveyed Part 139 airports indicate that panels average 15 participants, with a maximum of 40. Again, it is important to recognize that survey responses did not differentiate between the number of participants in FAA and airport-led SRM panels. Moreover, the use of SRM panels represents a new experience to Part 139 staff, and it has raised interest in experiencing the process. During the early stages of SRM implementation, groups tend to be larger; this is also beneficial for gaining experience and on-the-job training. Some Part 139 airports have divided larger groups into smaller group sessions with represen- tatives of each group in a final, consolidating session. Cognitive science research (Nijstad 2000) indicates that the brainstorming groups’ productivity generally does not grow proportionally with the number of participants. Many foreign airports have a few staff members with expertise in risk assessment who can develop a preliminary SRA that is circulated among panel participants for comments and suggestions. The consolidated results are then presented in a regular meeting for additional and final feedback. Lastly, the final document is circulated for approval. Checklists Lists of common hazards have been developed from past projects and people experienced with the type of project or change. The Pennsylvania Department of Labor and Industry (2013) says there are advantages and disadvantages of checklists. Advantages: • Can be used by nonexpert airport staff • Based on evidence from previous projects or similar changes • Assurance that the most obvious hazards will not be missed.

41 Disadvantages: • Less effective when applying to novel systems • Can inhibit creative thinking • Would miss hazards that were not included in the list • There may be critical gaps when checklists are used by unexperienced people. Formulation of airfield improvements and construction projects are the most common airport activities requiring an SRA when it has any impact on the airside. A typical checklist for airfield construction projects is presented in Appendix E. Safety Performance Indicators According to ACRP Report 1, Vol. 2 (2007), a Safety Performance Indicator (SPI) is any measurable parameter used to point out how well any activity related to safety is performing over time, and to assess the overall SMS health indirectly. SPIs are used to track and identify undesirable trends in safety performance. The analysis of SPI trends is the basic safety assurance tool used by airports that have implemented SMS. The airport selects appropriate parameters, in most cases derived from safety objectives defined by the airport management or related to known safety issues. Data on SPIs are monitored and trends are evaluated to identify the need for risk control actions. Advantages: • Raises safety awareness • In line with the airport’s safety objectives • Addresses specific safety issues. Disadvantages: • Requires regular data collection • Requires consistent data quality • Prone to seasonality bias when monitored over a short period • May require application of statistical techniques. Accident and Incident Investigation Reports Accident and incident investigation reports contain information on hazards, incidents, and accidents and are effective sources for identification of hazards. For undesirable events, sometimes it is neces- sary to conduct a root cause analysis (RCA), which is the analysis of deficiencies to determine their underlying cause (FAA AC 120-79A 2010). Advantages: • Proactive and effective for identifying hazards • Root cause analysis can be relatively simple and easy to use (e.g., 5-Whys technique) • Effective multiplier of the “eyes” of the airport to identify hazards • Airports always have tools/systems for reporting safety issues • Multiple hazards can be identified from safety reports. Disadvantages: • Will only address hazards associated with reported events • Root cause analysis is only effective if applied by a person familiar with the causes • Subject to reporting policies applied by the airport.

42 Structured What-if (SWIFT) This hazard identification technique was adapted from the Hazard and Operability (HAZOP) meth- odology originated in the chemical industry. The technique also involves a multidisciplinary panel of experts chaired by a facilitator; however, different from brainstorming, it is typically conducted at a higher level of the system, with fewer components. The facilitator prepares a list of questions to ask to the panel such as: • What if . . . ? • Could someone . . . ? • Has anyone ever . . . ? Advantages: • Provides a detailed and auditable record of the process • Takes less time than other more sophisticated techniques • Effectiveness will depend on experience of the facilitator. Disadvantages: • Less thorough when looking for details • Relies heavily on expert opinion • Careful thought is required in preparation for the application of this technique. Sometimes a facilitator will use SWIFT techniques during a brainstorming, even when it is not the predominant method used during the session. Safety Audits Safety audits are routinely used under an SMS for safety assurance, to assess compliance with regulatory and internal requirements, and to evaluate the effectiveness of a system, process, or procedure. Advantages: • In addition to identify deficiencies and deviations, audits can detect potential hazards. • Highlight areas of concern for further investigation • Recommend solutions and risk control measures for improvements • Fresh and impartial pair of eyes can identify issues that have been overlooked. Disadvantages: • Intense workload required • Direct objective is not to identify hazards • Airport staff may feel intimidated. BOW-TIE MODEL FOR RISK ASSESSMENT The basic SRA process that has been used by Part 139 airports to conduct safety assessments includes the use of a worksheet or table, and moving to the identification of hazards and applying the five-step SRM process recommended by the FAA. Each hazard and associated risk is individually evaluated using this process. Several hazards and risks can be evaluated, and the worksheet is used to sum- marize the process for all hazards. The method is useful for proactively assessing risks related to formulation, design, and construction of CIPs. More recently, airports in Europe and Asia have been using a well-known risk assessment model called “bow-tie.” The name is related to the shape of the diagram used in the risk assessment. Atypi- cal bow-tie diagram is illustrated in Figure 17.

43 The bow-tie diagram in Figure 17 presents the hazard (e.g., airfield construction), the undesirable event (e.g., runway incursion), the safety events or threats (e.g., construction workers near move- ment areas) and potential outcomes (e.g., aircraft running over construction worker). In addition, two categories of risk controls are illustrated. On the left side of the undesirable event are the proactive or preventive controls called barriers (e.g., safety awareness training to construction workers), in most cases intended to reduce the probability of the undesirable event occurring. On the right side are the mitigation controls (e.g., emergency response plan) intended to reduce the severity of the outcome upon occurrence of the undesirable event. The bow-tie method has many advantages over the more simple method of applying the risk matrix used by Part 139 airports; however it is certainly more complex and may take more time to complete. A typical bow-tie diagram appears in Figure 18. Among the main advantages of the bow-tie method are: • Provides an effective illustration to represent the risk structure and its elements • Increased awareness and understanding of safety risks • The identification of critical risk controls and assessment of how effective these controls are • Possibility to identify control ownership, control function, control type, control criticality and threat exposure • Represents a tool to continually monitor risks and make adjustments to controls when changes impacting the risk of an undesirable event are introduced to the system. FIGURE 17 Bow-tie diagram (Source: ECAST, Guidance on Hazards Identification 2009). FIGURE 18 Basic typical bow-tie diagram (Source: ASM Consultants).

44 The most important advantage of the bow-tie model is that preventive and risk mitigation mea- sures are linked to actions, procedures, and assigned individuals or organizations, thus linking these risk controls with the safety assurance component of SMS. Risk controls, hazards and consequences can be classified according to a variety of categories that can be represented in the bow-tie diagram. Some classifications used for risk controls are illustrated in Figure 19 and summarized in Table 14. Figure 20 is an example of a bow-tie diagram using those capabilities. As mentioned earlier, the bow-tie diagram may require more time for development, and it is not a replacement for the approach currently in use by Part 139 airports for CIPs. However, when dealing with specific safety issues such as runway incursions or bird strikes, the bow-tie model can be an excellent alternative. It is becoming more common with foreign airports that have matured SMS programs. The method for building a bow-tie diagram for safety issues of the aviation industry is well documented at the UK CAA website under “Operations and Safety.” Figure 21 presents a portion of the bow-tie diagram developed by the UK CAA for runway excur- sions. The original, comprehensive diagram is available on the UK website. The original diagram addresses many types of hazards and risk controls, including those under the responsibility of flight safety and air traffic. The intent in presenting this diagram is to show how comprehensive and how much information is available in this type of model. FIGURE 19 Representation of risk controls in bow-tie diagram (Source: UK CAA website 2015). Hazard exposure How frequently is the threat present Control ownership Stakeholder responsible for implementing the control Control function Type of action provided by the control Control effectiveness How effective is the control to reduce risk Control type Category based on purpose of control Control criticality Importance of control to prevent/mitigate undesirable event Source: UK CAA website (2015). TABLE 14 RISK CONTROL CATEGORIES FIGURE 20 Typical bow-tie diagram (Source: ASM Consultants).

FIGURE 21 Part of a bow-tie diagram (Source: UK CAA website, Significant Seven Bowtie Templates 2015).