Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
28 During FAAâs Part 139 SMS Implementation Study in 2011, 14 airports participated in the program with the intent to implement the Safety Risk Management and Safety Assurance components of SMS. Since then, those airports and other Part 139 airports have conducted airport-led SRAs; how- ever, there are still some questions about the process: ⢠Is the risk assessment methodology being used accurate enough? ⢠Is there a better process that is more accurate and faster to perform? ⢠Is there another methodology that does not require involving so many stakeholders? Scheduling these meetings has been a real pain! The best way to answer to these questions is to present risk assessment techniques that have been used by the industry and particularly by foreign airports that already have mature SMS and SRM processes. These techniques are presented in the next section of this report. The majority of SRAs conducted by Part 139 airports were related to airside issues and particu- larly to CIPs. In many situations, the airports hired a consultant to manage and facilitate the SRA. For complex issues, a panel of airport staff and external stakeholders was convened for discussions with the support of a facilitator (Landry et al. 2012). Airport staff, FAA, or consultants led the facilitation during the SRA sessions. The basic safety assessment process for complex safety issues typically involves convening a panel of subject matter experts. This approach was used by Part 139 airports in the Implementa- tion Study and by airports participating in FAA-led panels. FAA ARP also uses a safety assessment screening tool to evaluate the need to convene a FAA-led panel to conduct their SAs. The process of creating the SRM panel involves selecting representatives of airport stakeholders who could be affected by the safety issue or change. A facilitator leads the brainstorming session to identify hazards, assess risks, and define actions to control risks. The process has been used exten- sively in the aviation industry, particularly by air traffic organizations. FAA ATO has gained experi- ence by applying the technique to safety issues in their area of responsibility. The SRA process that has been used by Part 139 airports is comprised of a cluster of risk assess- ment techniques that have improved its effectiveness. FAAâs proposed SMS rule would require Part 139 airports to use a general five-step SRM process but does not go into further detail. Specific techniques are used in each step of the process, as presented in Table 7. Some techniques listed in Table 7 are discussed further in chapter five. Because the process is new and airport staff has had little experience with this type of approach, it creates a significant workload that previously did not exist at Part 139 airports. It is not unusual that airports will have questions about SRM and SRAs at this early stage of SMS implementation; however, the number of questions should decline as airport staff becomes more familiar with the process. RISK ASSESSMENT METHODOLOGIES USED BY AIRPORTS Most foreign airports rely on ICAO guidance to conduct risk assessments. Although the five foreign airports surveyed in this study use panels, only in special cases do they use facilitators for discus- sions and brainstorming. The basic process used by those foreign airports relies on a preliminary risk chapter three UNDERSTANDING THE SAFETY ASSESSMENT METHODOLOGY
29 assessment conducted by a small group or even an individual. The documented results of the risk assessment are circulated among stakeholders for comments and suggestions, and the assessment is presented in regular meetings in which further discussions lead to an agreed list of risk control actions with associated responsibilities. The preliminary risk assessment report is then consolidated with the final changes and circulated again for approval. Some foreign airports have successfully reduced the time required for a panel session by using the same approach, that is, circulating a preliminary risk assessment among the stakeholders. However, a significant disadvantage of this approach is that there is no facilitated brainstorming that can build synergy and obtain buy-in on key decisions. More recently, airports in the United Kingdom and Asia have made an attempt to introduce the bow-tie model (see Table 8) to assess airport safety issues. For example, the bow-tie method has been adopted by Manchester Airport as its standard for SRAs and was included it in its 2015 Aerodrome Manual, which incorporates the SMS Manual. A study (NLR, NLR-CR-2012-582, 2013) developed by the NLR Air Transport Safety Insti- tute for the FAA, identified several risk assessment methods for safety risk management used in the aviation industry. One of the objectives of that study was to identify risk assessment methods that best serve specific programs and can help harmonize risk assessment and risk management across the FAA lines of business. A summary of those methods that could be applied to airport safety issues appears in Table 8. The list is not comprehensive, and other methodologies are available to the industry; however, those techniques may not be practical when applied to airport safety issues and airport staff; some techniques may require the assistance of operational special- ists and statisticians. Many methodologies listed in NLR-CR-2012-582 are seldom used by Part 139 airports. FAA ARP primarily uses the Preliminary Hazard Analysis (PHA) to analyze the development and modification of ARP standards, and airport planning projects. The FAA ARP SMS Desk Reference (2012) suggests alternative methods for specific situations: ⢠Operational Safety Assessment (OSA), in case the planning study does not have extensive operational data to support quantitative analysis for risk assessment; and ⢠Comparative Safety Assessment (CSA) for situations in which multiple alternatives need to be compared. TABLE 7 TECHNIQUES USED BY SRM PANELS SRM Step Technique Used in SRA or SA Session Reference SRA SA 1 - Describe the System 5M (Mission, Man, Machine, Media, Management) ACRP Report 131 (2015) FAA Order 8040.4 (2012) FAA Order 5200.11 (2010) ARP SMS Desk Reference V1.0 (2012) FAA Order JO 1000.37A (2014) ATO SMS Manual V4.0 (2014) 2 - Identify the Hazards Brainstorming, Root Cause Analysis ACRP Report 131 (2015) ACRP Report 1, Vol 2 (2009) 3 - Analyze Risks Brainstorming and Risk Definitions (severity and likelihood) Airportâs SMS Manual FAA AC 150/5200-37A 4 - Assess Risks Risk Matrix Airportâs SMS Manual FAA AC 150/5200-37A 5 - Mitigate Risks Brainstorming, Consensus, and Documentation Airportâs SMS Manual ACRP Report 131 (2015) Source: ASM Consultants.
30 TABLE 8 MOST RELEVANT METHODS FOR RISK ASSESSMENT AND RISK MANAGEMENT FOR USE BY AIRPORTS Method Remark Importance 5M Model 5M = Man, Machine, Medium, Mission, Management model A variation of the 5M Model is the SHELL Model (Software, Hardware, Environment, and twice an L for Liveware (human element), the central element. Used to describe the system that represents the first step in the SRM process. This is one of the techniques used during the SRA process, and details are presented in ACRP Report 1, Vol 2. Preliminary Hazard Analysis (PHA) This is the basic method being used by Part 139 airports in conducting SRAs. It is a cluster of methods, including the 5M Model, that include a hazard analysis worksheet, risk matrix, and sometimes a preliminary hazard list (PHL). Effective in the early stages of formulation and implementation (e.g., airport improvements). Used by all FAA LOBs. PHA primarily aims to support SRM Steps 2 (Identify hazards), and 3 (Analyze safety risk). PHA is easy to use and takes little time compared to other methods for risk assessment. Comparative Safety Assessment (CSA) CSA uses brainstorming to list hazards and assess risks for each alternative considered. The process uses the risk assessment to rank the options for decision-making purposes. Excellent for identifying issues that may require further analysis. Used to evaluate multiple airport development alternatives in FAA SAs. Applied in airport master planning. Operational Safety Assessment (OSA) OSA provides an assessment of hazards and safety requirements for various functional components of a system. It establishes how safety requirements are to be allocated between air and ground components and how performance and interoperability requirements might be influenced. Applied to long range formulation when operational data are not available. The OSA is a two-step process. The first step identifies system physical and functional characteristics as well as air traffic and airport operational procedures. The second step is performing an operational hazard assessment for each component identified in step 1. Collision Risk Models Cluster that includes a variety of Collision Risk Models applied by subject matter experts Includes methods for quantitatively assessing risk of aircraft collisions. Models applicable to airports include those to assess risk of collisions during approaches, overruns, undershoots, and veer- offs. FMEA (Failure Modes Effects Analysis) FMEA comprises a family of techniques to identify potential FMEAs serve for identification and analysis of technical systems. failure modes of a system, the effects of these failures, and the criticality of these failure effects. FMEA is a well-established method; however it is relatively complex, requires subject matter experts, and is seldom used by airports. Fault Tree Analysis (FTA) Cluster of techniques that include Fault Trees, Dependence Diagrams, Reliability Block Diagrams, Functional Flow Diagrams. FTA is a well-established method; however, it requires specific expertise and is seldom used by airports. FTA is considered to be well- accepted for the safety assessment of aviation equipment only. (continued)
31 PHA is the primary methodology used by most Part 139 airports for SRAs. OSA and CSA tech- niques are rarely used in airport-led SRAs. PHA is an effective technique to identify the potential for major hazards at an early stage. It provides the basis for formulation, design, and construction decisions to mitigate the risks identified; however, one important limitation is that PHA may later require further in-depth analysis of risk. PHA is an effective method that can help Part 139 airports perform their internal risk assessments associated with the most common changes and safety issues, for example, CIP. PHA combines tech- niques that allow building synergy within the multidisciplinary team that composes the panel, to better identify hazards and assess risks associated with a certain change. In addition, the technique also serves to get buy-in and sharing of responsibilities on risk control actions derived from the assessment. Method Remark Importance Bow-Tie Model The knot of the bow tie represents a releasing event or a hazard. The left wing shows threats and proactive measures, which improve the chances to avoid the hazard; the right wing shows consequences and reactive measures to mitigate risk consequences from escalating. Developed by Boeing Co. in 1965. The approach has been popularized at the EU Safety Case Conference, 1999, as a structured approach for risk analysis within safety cases in which quantification is not possible or desirable. Airports in the United Kingdom and Asia are using it for common airport risks. Preventive and mitigating measures are linked to tasks, procedures, responsible individuals, and competencies. Risk Register Used for tracking of hazards, including status of control actions, reassessment of risks upon changes, and monitoring performance of control actions. The cluster includes established as well as updated methods. Airports can have their own risk register and it can be quite simple, particularly for smaller airports. Root Cause Analysis This is a structured facilitated team process to identify root causes of an event that resulted in an undesired outcome and develop corrective actions. The process helps identify breakdowns in processes and systems that contributed to the event and how to prevent future events. The purpose of an RCA is to find out what happened and why it happened, and determine what changes need to be made. Reactive approach for risk assessment, the methodology is normally used to investigate accidents and incidents occurring at airports. A common technique used in Root Cause Analysis is the 5-Whys (see ACRP Report 1, Vol 2). Structured What-if Hazard Technique (SWIFT) Structured brainstorming method of determining what things can go wrong based on the panelâs past experiences and judging the likelihood and consequences of Applicable for the review of SOPs and commonly used during SRA sessions, particularly to review hazards under different conditions (e.g., low visibility). One of the most commonly usedthose situations occurring. method by various industries (e.g., Chemical, Oil). Checklist Method Review of a detailed list of prepared questions about potential impacts to safety. Method used for hazard identification. Example list of questions for airfield construction is presented in Appendix E. Source: Developed by ASM Consultants based on NLR-CR-2012-582 (2013). TABLE 8 (continued)
32 BENEFITS OF SAFETY RISK MANAGEMENT PANELS Based on the work by Bircham (2015), some of the key benefits of a well-facilitated safety assess- ment supported by a SRM panel are: ⢠An understanding of the most critical risks to prioritize ⢠A defined risk control plan for each risk ⢠Discussion of risks with airport stakeholders ⢠Consensus about relative importance of risks among all stakeholders ⢠Awareness of different viewpoints ⢠Understanding of where current risk control procedures cannot take the risk down to acceptable levels ⢠Shared responsibilities on risk management. According to the survey conducted in this study, a typical risk assessment session lasts about 2 to 4 hours and involves 15 to 25 people, including managers, supervisors, staff, consultants, and a facilitator. Discussions about each risk normally take about 15 minutes to reach consensus about the risk level and control actions. Beyond the personnel costs, there are also travel requirements and considerable scheduling challenges. The importance of an effective and well-run SRM cannot be overemphasized.
