Transit Safety Risk Assessment Methodologies (2021)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

29   Case Examples This chapter contains five case examples of North American transit agencies including one international transit agency from Canada. The examples include one agency that operates bus-only service. Each case example includes a discussion of: • The transit agency’s operating context. • A description of the methodology used by the transit agency. • Details about the effectiveness of the methodology. • Notable practices. • Benefits. • Challenges. • Lessons learned. • Information about implementation cost(s) of the risk assessment methodology. San Diego Metropolitan Transit System The San Diego Metropolitan Transit System (SDMTS) operates bus and rail services directly and by contract with private operators. SDMTS’s on-demand service is provided by a contractor, as is its freight operation. As part of its rail service, SDMTS has 53.5 miles of light rail with 53 stations and freight operations with temporal separation. SDMTS has almost 100 fixed bus routes across its system, which encompasses about 570 square miles in the urbanized San Diego County with a total of 3,240 total square miles, serving approximately 3 million people in 10 cities. The agency provides 88 million annual passenger trips, or 300,000 trips each week- day on average. To handle the demand, the agency schedules 7,000 trips each weekday and has 128 rail (trolley) cars and 800 buses in its fleet. Rail is divided into Blue, Orange, Green, and Silver lines. The California Public Utility Commission (CPUC) has safety and security regulatory authority over all rail transit and other public transit fixed guideway systems in California. It functions as the State Safety Oversight (SSO) Agency. Working in cooperation with FTA, CPUC staff conduct continuous and ongoing safety oversight of rail and other transit agencies to verify compliance with safety plans and to ensure that the plans meet all state and federal rules and regulations (California Public Utility Commission, n.d.). Safety Risk Assessment Methodology With the establishment of the SMS, SDMTS divided its agency safety plan into the various modes of operation. Although the risk assessment approaches of bus and rail are similar in that both use MIL-STD-882, there are some differences as described in this section. The agency C H A P T E R 4

30 Transit Safety Risk Assessment Methodologies mainly refers to the Hazard Analysis Guidelines for Transit Projects (Adduci et al., 2000) and the section of the Rail Transit Safety Branch Procedure for Hazard Management in the California Public Utilities Commission (2021) Program Standard—Procedures Manual: State Safety and Security Oversight of Rail Fixed Guideway Systems. Bus Risk assessment for the bus side of SDMTS is conducted using the MIL-STD-882 methodol- ogy, which includes severity of the risk and the likelihood of that risk occurring. There are four categories for severity (catastrophic, critical, marginal, and negligible) and six categories for likelihood (frequent, probable, occasional, remote, improbable, and eliminated). These likeli- hood categories are broken down into weekly, monthly, yearly, per decade, and so unlikely that the occurrence may not be experienced; the categories are also broken down by vehicle miles. For example, a likelihood of “probable” is considered a risk that is likely to occur monthly or every 1 million miles. Agency staff indicated that hazard severity can be subjective, saying, “Two reasonable people can come to different kinds of analysis.” The agency provides examples of frequency to help to break this concept down (see Figure 22). SDMTS tries to limit sub- jectivity by looking at the history of the hazard and how many times an accident occurred. The agency staff measure the worst credible mishap like design inadequacies and look at the likeli- hood of the accident happening. In the SDMTS Public Transportation Agency Safety Plan (PTASP), risk levels are defined as high (unacceptable), serious (undesirable with management decision), medium (acceptable with review by management), and low (acceptable without review). The agency tracks hazards in its Risk Register. Hazards that are deemed “acceptable without review” are not required to be entered into the Risk Register. Rail Similar to the bus risk assessment approach, rail uses four categories to define hazard severity (catastrophic, critical, marginal, and negligible). Rail describes rail risk severity as the mea- surement of the worst credible mishap expected to result from human error, environmental conditions, design inadequacies, subsystem or component failure malfunction, procedural Source: San Diego Metropolitan Transit System (2020). Figure 22. Likelihood of risk occurring, SDMTS bus.

Case Examples 31   deficiencies, or a combination. For hazard probability, there are five categories instead of six, which are defined as frequent, probable, occasional, remote, and improbable as shown in Figure 23. The example of frequency is provided in the figure; however, SDMTS also considers likelihood based on events, population, items, or activity. After hazard severity and probability are determined, SDMTS project implementation staff and its Safety and Security Review Committee assess the associated risks using the hazard assessment matrix shown in Figure 24. The bus safety staff use a similar matrix. After assessing the risk, SDMTS rail determines the criticality of implementing corrective actions to reduce the hazard to an acceptable level. As seen in Figure 25, some hazards may be deemed acceptable with or without review while others should or must be mitigated. Source: San Diego Metropolitan Transit System (2020). Source: San Diego Metropolitan Transit System (2020). Source: San Diego Metropolitan Transit System (2020). Figure 23. Hazard probability, SDMTS rail. Figure 24. Risk Assessment Matrix, SDMTS rail. Figure 25. Criticality Index, SDMTS rail.

32 Transit Safety Risk Assessment Methodologies Hazards are tracked in the Hazard Management spreadsheet. The mitigations are discussed in monthly Safety Committee meetings and documented in meeting minutes, which are posted on bulletin boards and on the agency intranet. Contractors are required to report hazards to the Operations Central Control (OCC). Since MIL-STD-882 is the most widely used methodology in the transit industry, it is the preferred method of the CPUC. Benefits The hazard assessment approach is reportedly working well. It establishes a standard method for prioritizing hazards. Having MIL-STD-882 in place provides an avenue for procuring resources based on priorities. Safety staff indicated that low risk can be handled at the depart- ment level while high risk can be accelerated to the executive team where more resources could be allocated for mitigation. During an SSO audit in 2015, concerns were identified related to how hazards and corrective actions were tracked. SDMTS worked to address those concerns through its safety risk manage- ment (SRM) process. Three years later in 2018, the SSO audit found no issues that needed to be addressed and the process is now lauded throughout California as a model for other transit agencies. Challenges One of the concerns faced when SDMTS was finalizing its PTASP was about employee engagement. There was good communication at that time about how the matrices would be difficult for frontline employees to understand. The agency recognized the importance of hazard reporting by frontline employees who are the “eyes of the system” and was worried about asking them to go through the hazard assessment process. Officials decided to make it as easy as possible for frontline employees by having them report hazards via radio to a supervisor. The supervisors handle the documentation of the hazard and the interactions with safety leader- ship. Training these frontline employees to understand the subjective side of hazard manage- ment is one challenge that was mentioned during the interview. “Any employee can rate a hazard as catastrophic or negligible,” staff said. Being mindful of this subjectivity is something the agency practices. Staff said that they try to be open to the feedback and try to understand how the person interprets the hazard. They use a safety com- mittee to track the hazard and communicate to the employee, sharing how hazards are rated. Training is done initially and repeatedly as refresher training to ensure consistency through- out the assessment process. Both terminology and the process as a whole are included in the training. Training specific to reporting hazards is also needed for contracted employees. An important note made during the interview was about the purpose of SMS. The point of SMS is not rating risk and assigning an index for that risk; the point of SMS is to try to categorize and properly define every risk. Then, with limited resources, an agency decides priorities for mitigating that risk based on identified strategies. This interviewee advised not to focus on the “2A” or other risk index and instead to use the categorization as a tool to help prioritize where funding and resources should be directed. How Safety Risk Assessment Is Used SDMTS uses the SRA in the following processes: • As part of its Safety and Security Certification process for major capital projects • As part of its Safety and Security Certification process for any capital project that is safety critical

Case Examples 33   • When investigating a safety event, accident, or incident • On hazards and other safety concerns that are identified through inspections and internal audits • On hazards and other safety concerns reported by employees and/or riders When it comes to capital projects and safety-critical projects, safety is integrated into design, specification preparation, equipment selection, construction, procedures, and operations. SDMTS project implementation staff apply methods of hazard identification, assessment, and resolution to minimize or eliminate accidents and injuries. Rail safety staff also apply the SRM process for new operations or maintenance procedures and any organizational changes. Staff work to identify areas and situations prone to a high frequency of incidents and accidents through existing system inspections and evaluations, reviewing trends, conducting a compara- tive analysis, and evaluating available data. There is a formalized process for safety analyses involving identifying, eliminating, and/or controlling hazards. The analyses are conducted for the following: • Identification of hazards • Assessment of the severity and probability of occurrence of the hazard • Timely awareness of hazards for those who must resolve them • Traceability and control of hazards through all phases of a system’s life cycle SDMTS conducts an annual internal safety and security audit, which includes elements scheduled on a rotation to ensure that all 21 elements are completed during a three-year cycle. Deficiencies or instances of noncompliance are provided to the responsible department, and a corrective action plan (CAP) is created, which includes the required action needed to minimize, control, correct, or eliminate the identified risk and hazard; the schedule for taking the action; and the responsible party. The operations and maintenance superintendents of each depart- ment are responsible for taking all corrective actions and submitting regular progress reports. Hazards reported by employees are recorded in the tracking spreadsheet and communicated to the appropriate safety department, which informs the department responsible for the CAP. The hazard is assessed and tracked to closure. The outcome and new rating are communicated back to the employee through the safety committee activities and minutes. The management of safety risks is performed using a decentralized process in which hazards are assessed and evaluated by the operating departments (transportation and maintenance) with assistance from the System Safety Manager. The SRA process includes all department super- intendents, managers, OCC, frontline employees, and contractors. All frontline employees and contractors are encouraged to communicate hazards directly to the OCC. The safety depart- ment assesses the hazards along with department superintendents and managers. Together, these groups develop the CAPs, which are in place until the hazard has been resolved and the outcome is communicated back to the employee and contractor through the Safety Committee and documented in the minutes that are published after each meeting. Every employee signs a receipt acknowledging the document and instructions on how to obtain access to it through the agency intranet. Implementation Cost The costs of implementing MIL-STD-882 are primarily in the form of initial training. Train operators and supervisors are trained on hazard reporting in their initial and biannual refresher training. Microsoft Excel spreadsheets are used to track and document reported hazards. Other costs have been incurred for software ($21,833/year over four years), which is currently being configured to assist with aspects of SRA. Operational support staff for the rail system have been hired at costs outlined in Figure 26 for a total of $213,921.

34 Transit Safety Risk Assessment Methodologies Sacramento Regional Transit District Background The Sacramento Regional Transit District (SacRT) is the public transit service owned and operated by the city of Sacramento, California. SacRT operates over 80 bus routes (fixed- route, microtransit, and dial-a-ride), 43 miles of light rail serving 52 light rail stations, and Americans with Disabilities Act paratransit services within a 400-square-mile service area in Sacramento County. Data from fiscal year 2019 indicate that SacRT light rail ridership averaged about 40,000 on weekends while weekday bus ridership averaged approximately 37,000 passengers per day. Safety Risk Assessment Methodologies SacRT uses a modified MIL-STD-882 as well as other methods, including the European CSM, bowtie, barrier analysis, and FMEA. SacRT began incorporating methods other than the MIL-STD-882 about five years ago. The safety department at SacRT wrote its SMS manual in-house, and its actions resulted in more familiarity and ownership in the safety system and the SRA process. The manual was written to maximize flexibility. One interview participant described it, saying, “The PTASP was akin to the U.S. Constitution where there is room for interpretation, and where the agency is not locked into a specific model.” The thought was that specificity would be included in the procedures and processes. Prior to five years ago, SacRT approached the SRA differently; there was an acceptable level of risk at that time. That philosophy has changed. Currently, risks to the general public and passengers are assessed first, then the major loss of equipment and then operational readiness. Taking risks that may result in large delays, breakdowns, or disruptions can significantly hinder its key service, and this is not acceptable. In addition, SacRT looks at risks that may not relate directly to safety but could affect the organization overall. An example of these risks might be public perception. For instance, cleaning during the COVID-19 pandemic was something that the agency began doing early. Sometime later, evidence was presented that surfaces were not a factor in the spread of COVID-19; however, SacRT continued to clean and sanitize surfaces to minimize the risk of negative public perception. After the interview, the SacRT interviewee shared documentation that the agency uses, for the project team’s reference. The documentation about FMEA, the fault tree analysis, and ETA are included in the Literature Review section of this report. Other references were mentioned, including AcciMap Analysis (Branford et al., 2009; Branford, 2011; Salmon et al., 2020; Stanton and Salmon, 2020), Human Factors Analysis (Shappell and Wiegmann, 2000; Dekker and Hollnagel, 2004; SKYbrary, 2020), Risk Homeostasis Theory (Curry et al., 2004), and principles of risk-based decision making (Boniface, 2003). Total = $213,921 Figure 26. Trolley operational support costs.

Case Examples 35   Military Standard 882 Benefits of the MIL-STD-882 mentioned by SacRT included the following: • Institutional knowledge of the methodology means that people are familiar with and under- stand the methodology. • It is a good way to communicate results since the MIL-STD-882 is a consistent comparison platform. Therefore, risk to the public, environment, equipment, operations, and so forth calculated using a different SRA can be translated back to the MIL-STD-882 nomenclature. Challenges of the MIL-STD-882 mentioned by SacRT included the following: • There is not much thought involved in the methodology. People using the methodology can simply cut and paste from other documents. • Conducting the assessment can seem like going through the motions, where someone is “just generating paper.” • Losses are more easily deemed acceptable based upon frequency, which might make sense for military operations and systems. However, losses (regardless of the acceptability) must always be reduced to as low as reasonably practicable. • There are factors that are not included in the MIL-STD-882. Staff working at SacRT modified the MIL-STD-882 to address the challenge regarding acceptable risks. To make these modifications, safety staff pulled in the American National Standards Institute (ANSI) standard often used in aviation. The ANSI standard allows users to see the different catastrophic levels because there is a numerical comparison. This view allowed for prioritization of mitigations to address risks that are more critical. European Common Safety Method SacRT is currently converting to the European CSM for the Safety Certification processes. Safety staff see the CSM as more organized, where the person assessing is less likely to “get lost in the weeds” and “can focus on what are truly safety risks.” With the European CSM viewed as a more structured SRA methodology, SacRT reported that it will try the European CSM over the coming months as well as the MIL-STD-882 to compare and make a determination. Benefits of the European CSM mentioned by SacRT included the following: • More formalized and pragmatic than the MIL-STD-882. • Some of the framework, such as what is in the flowcharts, may lend structure to the assessment. • Less subjective. • Better for CM. During the interview, SacRT staff expanded on the perspective of CM. When the agency considers changing procedures or processes, staff expect to conduct an evaluation of the change to help identify potential risks and challenges so that they can develop mitigations. Prior to that expanded perspective, SacRT incorporated CM only if it made a change to the fit, form, or function of a safety-critical item. An example provided was changes in shift times. This type of change is now included as part of CM and assessed as such. With the implementation of SMS, CM is now the item that is still being developed to achieve a well-defined system. Some of the framework in the European CSM, like the flowcharts and evaluation methods, are expected to lend structure to the process. SacRT staff are also open to using something that has been designed elsewhere to see how it works for them. The agency staff have a user group in which they listen to people from other agencies like the Bay Area Rapid Transit (BART), the Los Angeles County Metropolitan Trans- portation Authority, and SDMTS. These agencies report similar flexibility in their CM.

36 Transit Safety Risk Assessment Methodologies Challenges of the European CSM mentioned by SacRT included the following: • Unlike the MIL-STD-882, people outside of the safety department are largely unfamiliar with the methodology. • Examples demonstrating when and how to use the European CSM are not provided by FTA or FRA. In addition, training materials created by the Transportation Safety Institute (TSI) mention only the MIL-STD-882. • Similar to and perhaps as a result of the previous bullet points, the transit industry as a whole is very tied to the MIL-STD-882. Alternate Methods—FMEA, LOPA, Barrier Analysis, and Bowtie SacRT uses several other methods depending on the risk it is assessing. For instance, the FMEA is used when planning a project or process change. SacRT safety staff mentioned an example using FMEA after an incident that stemmed from a lack of a formalized process. There was a desire to resume the activity again, so the staff assembled a multidisciplinary group to perform a FMEA risk assessment. For SacRT, safety staff asked their department to answer the following questions when assessing risk: • Is the risk something that happens once a day, once a month, or less frequently? • What is the rate of the severity if this risk occurs? • Can the risk be detected relatively easily? Agency staff answered these questions on their own to avoid “group think.” The answers of the various staff were similar. Commonality was found in areas where they knew there was a risk but did not know it was a great risk. The controls to override the risk were then considered. SacRT used barrier analysis as a guide at this step in the process. Questions like “What stops you from being able to do that?” prompted them to look at how easily that barrier could be overcome. Barrier analysis fits into the LOPA, where the agency staff look at machinery or processes and determine the various protections. SacRT treats the risk like an organism. Is the risk easily defeated or are there gaps? Oftentimes they find the gaps and discover that they have the tools to address them. SacRT staff sometimes start with one risk assessment methodology and end up with another— put one way, “you need to have more than one screwdriver” for risk assessment. To analyze potential causal relationships, SacRT uses the bowtie method to help differentiate between proactive and reactive risk management. If a preventive action is taken, safety staff consider what it would take to recover (e.g., training or equipment). “How would the event impact our same five categories?” staff asked, and followed up by saying that they advise look- ing at past data and seeing how often the risk happens. A specific example was provided about an employee who slipped, causing a knee injury. The worker’s compensation claim was very high. After analyzing the risks and what it would take to prevent that from happening, SacRT decided to do nothing. As of the interview, no more such events had occurred in the three years following. SacRT staff indicated that they are not restricted to using one SRA methodology. This flexible approach has been occurring only in the past two to three years after completing the SMS. The alternative methods are used to refine the inputs of the MIL-STD-882; then, the agency uses the MIL-STD-882 as the communication platform. Results from other methods are translated into the MIL-STD-882 nomenclature so that risks can be compared. As such, the MIL-STD-882 is how risk is communicated outside of the safety department.

Case Examples 37   Root Cause Analysis When SacRT starts a process, it typically uses mapping to do a high-level root cause analysis. An example was provided about a recent increase in pedestrian trespassers near light rail vehicles. The agency used the root cause analysis to assist staff and asked questions like: • Our processes have not changed, so what is different now? • Why are people trespassing? • Are we, as an agency, responsible for any of the factors that may have led to this accident? • How can we take responsibility to fix it? The agency was able to get to possible root causes that pointed to the following: • More people are experiencing homelessness. • The Centers for Disease Control and Prevention recently suggested that the people experi- encing homelessness not be relocated because of COVID-19. • There are no other places to send people experiencing homelessness. • The COVID-19 pandemic response lacked organization. • Public opinion is negative about people experiencing homelessness. There is a reason that this increase in trespassing is happening. Through this root cause analysis, SacRT examined how the agency can be part of the solution. How Safety Risk Assessment Is Used SacRT uses the SRA in the following processes: • As part of its Safety and Security Certification process for major capital projects • As part of its Safety and Security Certification process for any capital project that is safety critical • When investigating a safety event, accident, or incident • On hazards and other safety concerns that are identified through inspections and internal audits • On hazards and other safety concerns reported by employees and/or riders When asked when different methodologies are used for these processes, SacRT safety staff indicated that if the agency is considering more of a development change or a process change related to equipment, it looks to the FMEA. For changes in procedure or when needing to fix something, the bowtie would probably be used. The FMEA and bowtie are more predictive methodologies. Staff said that they would not use the FMEA and bowtie for a hazard that is immediate. The greatest overall challenge for SacRT in terms of the SRA is risk perception versus catego- rization. Training was mentioned as a key to teaching people how to take a more data-driven approach in their assessment because not everybody is comfortable working with data. Addi- tionally, some people who are new to the SRA process may want all of their concerns to be viewed as critical or catastrophic. Previously, the SacRT safety department would support the other departments handling the SRAs. Now the department is acting more as a coach. The agency, in doing the PTASP, adopted a single plan for both modes (rail and bus) even though FTA regulations are less formal than FRA regulations. The PTASP provides the framework for bus and rail department staff to follow such that the SRAs conducted for each of the different modes are functioning at the same level. With the safety department leading, coaching, and mentoring, “a consensus is being forged.”

38 Transit Safety Risk Assessment Methodologies Resources SacRT provides its training in-house. Staff indicated that training on theoretical problems, or from other industries, is often nebulous or hard to grasp. The relevance needs to be demonstrated. Once an employee has participated as a SME a couple of times, the process becomes easier. The trainers take on many different roles. These people do everything from rail incident investigation to worker injury, bus stations to rail stations. Instead of pigeonholing people into one area where they end up siloed, staff work as part of a system wearing several hats. SacRT senior safety staff conduct train-the-trainer sessions for the SRA. Implementation Cost Although there are costs associated with switching to a different software platform to inte- grate and track risk and CAPs, the software purchase was planned regardless of the transition from a mostly paper-based risk assessment record system to tracking the SRA digitally. In other words, safety risk analysis was not necessarily the driving factor in creating the cost increase. As far as training staff to use new software, SacRT budgets for general safety training, so the funds already exist. The implementation of SMS resulted in about one additional full-time equivalent that is primarily focused on conducting data collection and analysis. The SacRT safety risk analysis process is more uniform now that SMS is in place. Edmonton Transit Service Background The Edmonton Transit Service (ETS) is the public transit service owned and operated by the city of Edmonton in Alberta, Canada. ETS offers bus, rail, on-demand, and—starting in the later part of 2021—microtransit. ETS’s bus fleet of over 1,000 buses and 40 community buses operate on more than 180 routes. Edmonton Light Rail Transit has 18 stations on two lines and 15.1 miles of track. ETS’s Disabled Adult Transit Service includes approximately 100 fully accessible vehicles. All services are operated directly by ETS, with no reliance on service con- tractors. The ETS average weekday ridership is just under 400,000 trips, and the organization recently underwent a major redesign, which resulted in increased service to suburban areas. The ETS interviewee was most familiar with the rail mode. Because the interview focused on rail, this discussion may not entirely represent SRA methods and/or practices that are imple- mented with the other modes operated by ETS. The ETS interview could be scheduled for only 30 minutes, which is about half the length of time covered in the other structured interviews. Even though time was a limiting factor, a significant amount of information was gathered during the call. The ETS interviewee sent additional documentation to the project team to review and shared other references (Adduci et al., 2000; Office of Rail and Road, 2018; Metro North Partners, 2019, 2020; Transport Canada, 2020). The service’s Metroline NW Extension LRT: System Safety Program Plan recommends performing hazard risk assessment following recom- mendations defined by MIL-STD-882E (Acquisition Management Systems Control, 2012), Joint Software System Safety Engineering Handbook (JSSSEH) (Joint Software System Safety Engineering Workgroup, 2010), and Sample Safety Risk Assessment Matrices for Rail Transit Agencies (FTA, 2019a). Safety Risk Assessment Methodologies ETS uses both the Canadian common method and a modified MIL-STD-882.

Case Examples 39   Canadian Common Method The Canadian common method is largely based on the European common method, with minor deviations to better meet the requirements established by Canadian regulatory agencies. Figure 27 is taken from an ETS document that visually represents the common method used by the agency for risk assessment. A somewhat unique stipulation of the common method is the requirement for an independent third-party evaluation. As Figure 27 outlines, the common method may involve the use of quantitative risk assessment methods, which could include methods such as the MIL-STD-882 or the modified MIL-STD-882. ETS recognized several benefits of the Common Method, including • North American transit regulatory agencies have seemingly shifted away from being very prescriptive with safety-related methodologies toward offering agencies more flexibility regarding these methods. ETS stated that they now use the common method because it is more flexible than the other methods. This increased flexibility puts more responsibility on the transit agencies for having good SRA methods and documentation practices in place, but it also treats transit agencies as being experts in their fields—an ideology embraced and appreciated by transit agencies. • Because the common method offers increased flexibility, ETS feels that the method enables a higher degree of innovation for agency staff to implement the SRA. • The common method allows organizations to use demonstrated acceptable safety risks from similar projects as well as explicit risk estimation (quantitative or qualitative). • The common method includes several points of “intervention” that allow transit professionals to consider various solutions to safety risks. Additionally, the process is easily reversed, in the event that a solution is deemed unacceptable. The interview revealed a generally high level of satisfaction with the common method. The most significant challenge noted was in regard to the requirement for an independent third- party evaluation. This requirement may necessitate the acquisition of external consulting services, which may be a challenge for agencies that are fiscally constrained. Modified Military Standard 882 Historically, ETS has relied on the modified MIL-STD-882 for its primary SRA methodology. The common method has only recently been implemented. For each identified potential risk within a light rail transit project, ETS assesses the severity category and probability level. ETS’s modification of the MIL-STD-882 is how it provides a higher level of specificity to probability (Figure 28), as well as an assignment of a real dollar value to the various severity categories (Figure 29). The assessed risks are expressed as a risk assessment code (RAC), which is a combination of one severity category and one probability level. Figure 30 and Figure 31 describe the RAC and the risk levels. Hazards deemed high or serious are redesigned or controlled to bring the risk to a level of acceptability. Hazards deemed medium or low are further assessed for options to minimize their risk where risk minimization is considered feasible. A review of ETS documentation identified two significant benefits associated with the use of the modified MIL-STD-882: • The primary benefit of using the modified MIL-STD-882 approach is the flexibility of hazard management and prioritization in terms of severity and probability. This flexibility facili- tates a rigorous and defensible method for the allocation of critical resources. For example, a hazard with a RAC of “3D” would likely have fewer dedicated resources than one with a RAC of “1B.” In the absence of such a process, allocation of resources becomes arbitrary and potentially less effective.

40 Transit Safety Risk Assessment Methodologies Source: Edmonton Transit Service (2020). Figure 27. Canadian Common Method diagram.

Case Examples 41   Source: Metro North Partners (2019). Source: Metro North Partners (2019). Source: Metro North Partners (2020). Figure 28. Example ETS probability descriptions. Figure 29. Example ETS severity category descriptions. Figure 30. Example of Risk Assessment Code.

42 Transit Safety Risk Assessment Methodologies Source: Metro North Partners (2020). Figure 31. Risk level descriptions. • A secondary benefit is that the System Safety Program Plan (SSPP) “identifies and assigns specific levels of management authority with the appropriate levels of safety mishap severity and probability.” As such, this “methodology holds program management and technical engineering accountable for the safety risk of the system during design, test, and operation and the residual risk upon delivery to the customer (Metro North Partners, 2020, p. 6).” The structured interview revealed that other benefits included the following: • Because the MIL-STD-882 is so widely used, slight modifications to this parent methodology are often well accepted and understood. • Conducting several risk assessments for a single project facilitates the analysis of risks through various lenses, such as “people,” “environment,” “assets/infrastructure,” and/or “production/ operations.” • The establishment of aggregate dollar values for projects facilitates fairly straightforward financial comparisons, which can be enhanced through charts and graphs. Similar to comments made by almost all other transit organizations regarding either the MIL-STD-882 or the modified MIL-STD-882, ETS noted a potentially high degree of subjec- tivity. This high degree of subjectivity has been managed by ETS’s having project-specific SRA workshops that focus on establishing common threshold definitions and confirming what thresholds are being used by specific projects. This action might include assigning a financial value to what qualifies as “catastrophic.” ETS confirmed that the ability to accurately assess risk and probability is correlated with industry experience. Personal experiences can also play into one’s assessment of risk and, particularly, probability. The influence of personal experiences makes it important to normalize frequency by a value (e.g., day, year, number of trips). How Safety Risk Assessment Is Used ETS uses the SRA as part of its Safety and Security Certification process for major capital projects, as well as minor capital projects that are deemed safety critical. Resources For large capital projects, ETS may use external consultants to conduct the modified MIL-STD-882. No other resources were specifically mentioned in the interview.

Case Examples 43   Implementation Cost ETS was not able to provide any costs associated with its SRA process. This task would likely involve individuals from several transit agency departments. The structured interview sug- gested that an accurate assessment of all costs for the SRA was unrealistic, given the short time frame of the structured interview and of the overall project schedule. New Orleans Regional Transit Authority Background The New Orleans Regional Transit Authority (RTA) operates several modes, including bus, rail, on-demand, and ferry. On average, approximately 36,000 riders a day use RTA’s bus services, which serve nearly 40 routes throughout the city. RTA’s on-demand (paratransit) is a shared ride service with an average daily ridership of about 1,200. Streetcar (rail) ridership averages approximately 19,000 daily trips across four routes. The Algiers Point/Canal Street and Lower Algiers/Chalmette ferries connect prominent destinations on both banks of the Mississippi River. Ferry service is operated by a third-party contractor. Safety Risk Assessment Methodologies RTA uses a modified MIL-STD-882 and has relied on this methodology for some time. The risk severity criteria were modified and simplified to be more specific to the agency’s operating characteristics. Severity is scaled to four points (catastrophic, critical, marginal, and negligible), while probability remains at five points (frequent to improbable). Not modifying the severity scale may have been the result of some discomfort with the addition of a fifth item, which could have led to even more subjectivity in this part of the assessment. Safety risk acceptance criteria require “high” and “medium” items in terms of the safety risk index to be addressed through executive leadership action. In the event that the SRA process yields a high risk, an emergency executive team meeting would be called with the focus on immediate resolution. This emergency meeting could result in the stopping of work or halting of service so that people (e.g., employees, riders, or the general public) are not put in harm’s way. If a risk is deemed medium, RTA will work through a larger leadership executive group consisting of executive team and safety committee members. For rail, RTA has a system safety officer who oversees the application of the SRA process, which was recently formalized in the PTASP. The system safety officer serves as an additional layer of oversight to ensure that RTA is following the PTASP. All modes across RTA use the same SRA method. To prevent siloing, RTA empowers middle management involvement. Although relatively new, this process seems to be helping. In devel- oping the SRM section of the agency’s safety plan, staff referred to the guidance material available on FTA’s PTASP Technical Assistance Center (TAC) under the SRM section of the resource library (FTA, 2021). RTA commented that, because FTA/TSI training incorporates the MIL-STD-882, there is a baseline understanding in the industry. As a result of this industry-wide familiarity, RTA can easily reach out to other transit agencies and discuss best practices with this methodology. RTA also commented that recent discussions have centered on how to clearly and concisely present, in the PTASP, how an agency is implementing the SRA. Despite many transit agencies suffering from modal siloing, the SRA process (in general) may facilitate the bringing together of people across the agency who otherwise may not have such an

44 Transit Safety Risk Assessment Methodologies opportunity. The formation of cross-functional teams and committees and their involvement in processes in which they normally would not be involved can work to enrich their perspectives, resulting in a deeper understanding of the importance of the organization operating safely and how they, as individuals, fit into helping to grow the agency’s safety culture. Like all other agencies that use either the MIL-STD-882 or a modified version thereof, sub- jectivity can be an issue. RTA has noticed some disparities in either the probability or severity of risks across various agency teams, departments, or both. Everyone has different perspectives based on industry and personal experience. Such differences are why a common definition of risk is important. Similarly, RTA spoke to the technical nature of the MIL-STD-882 and the difficulty some staff have in fully understanding this methodology. RTA does not have staff dedicated to the SRA, so any time spent trying to understand a challenging process can detract from other priorities. Furthermore, fiscally constrained agencies may not be able to hire external consultants to conduct these required processes. The SRA starts with a basic understanding of what needs to be done after hazards have been identified. While RTA safety staff must be heavily involved in “medium” and “high” risks, department-level ownership of “low” risks is encouraged. Recent organizational process improvements and safety committee restructuring have helped facilitate departmental owner- ship, which is expected to increase over time. RTA reiterated the key importance of agency safety committees to the risk assessment process. The committees must be both held accountable for their role and given the mechanisms to hold employees accountable for safety. RTA has implemented a tiered safety committee structure, with several department-level safety committees feeding information to a single agency safety committee. Although fairly immature, this model has shown signs of success, particularly with the dissemination of safety-related information to frontline non-managerial staff. Another key is finding SRA champions who can effectively engage frontline employees. How Safety Risk Assessment Is Used RTA uses the SRA in the following processes: • As part of its Safety and Security Certification process for major capital projects • As part of its Safety and Security Certification process for any capital project that is safety critical • When investigating a safety event, accident, or incident • On hazards and other safety concerns that are identified through inspections and internal audits • On hazards and other safety concerns reported by employees and/or riders The interview discussion suggests that, in RTA’s experience, the modified MIL-STD-882 may be best suited/easiest to apply to major capital projects. In these projects, transit employees and/or contractors are not yet actively engaged in the work to be performed, so the organiza- tion can proactively address potential hazards. Resources The SRA is resource intensive. RTA is wary of contracting the process. Having ownership of a process leads to the implementation of beneficial change, which can lead to a feeling of

Case Examples 45   pride in a job well done. However, there is perceived value in bringing in a consultant to do the technical assessment for higher-level risks. In the absence of budget, dedicated staff and tools guide the assessment, but the process can quickly become subjective. In the past six months, RTA began new in-house training akin to SMS 101, which empha- sizes frontline risk assessment. The training focuses on RTA operators and begins by reviewing how each and every operator has been trained to conduct their job safely. The training goes on to demonstrate how all these safe operating practices feed into an overall safety management strategy. The goal is to demonstrate how everyone’s role feeds into the overall organization’s system of risk management. The training has been very beneficial. Even though only a handful of these trainings have been offered, the response has been positive, and a more structured class offering is being implemented. RTA stated that online tools would be very beneficial. For example, RTA has looked at soft- ware that is not necessarily SMS driven but related to driver management. Similar cloud-based applications designed to catalog identified hazards would be helpful, particularly for inputting known parameters (e.g., vehicle boardings) and making an initial risk assessment category. Multiple departments could log in simultaneously, which would enable safety to be tracked elsewhere and allow real-time reporting. Implementation Cost RTA commented that the cost associated with implementing the SRA process itself has never been quantified independently from other safety department tasks or agency-wide safety- related processes/activities. Likewise, there is not currently a budget item or account that is solely associated with this process. Final Thoughts RTA believes the industry, which currently lacks knowledge on SRA methods other than the MIL-STD-882 or the modified MIL-STD-882, will benefit from this TCRP project. As it relates to SMS, CFR 673 (PTASP) requires agencies to account for risk assessment and safety assur- ance. RTA believes that the industry would benefit from federal communication that makes it clear that transit agencies have the flexibility to choose methods that work for the agency, given unique operating parameters, and that provides examples of those methods. Golden Empire Transit District Background The Golden Empire Transit District (GET) was formed in July 1973. GET is the primary public transportation provider for the Bakersfield Urbanized Area, California. In fiscal year 2019, GET provided 6.4 million rides on its fixed-route, express, paratransit, and microtransit services. The entire GET bus fleet is powered by compressed natural gas. The fleet consists of 86 40-foot transit buses, two commuter coaches, 25 GET-A-Lift cutaway vehicles, and seven microtransit RYDE shuttles. The fixed-route bus system services 16 routes, and most routes operate seven days a week. The express bus service is provided Monday through Friday between downtown Bakersfield and the Tejon Ranch Commerce Center. The paratransit service operates in the same areas and during the same days and hours as the fixed-route bus system. The on-demand microtransit service RYDE serves only the southwest Bakersfield zone. All services are operated directly by GET, with no reliance on service contractors.

46 Transit Safety Risk Assessment Methodologies Safety Risk Assessment Methodologies As the Bakersfield area is transitioning from a small city to a medium-size city, so is GET. Safety was not at the crux or forefront within the agency in the past as it relates to techniques, projects, and procedures. Prior to the PTASP rule, GET used a method that resembles the European CSM for the SRA without formally identifying it as such. How GET chose to adopt the European CSM in the first place has not been traced and the method is not documented by the agency. Before the PTASP rule became effective, GET developed its first full and admin- istrated safety program. In general, GET tracks accidents and trains drivers to prevent acci- dents from happening. The SRA requirement of the PTASP was new to the agency and GET had no prior experience with the MIL-STD-882 nor any other SRA methods. With the approval and implementation of its PTASP, GET will use only its modified MIL-STD-882 from now on for SRA. GET’s Safety Risk Matrix GET uses a modified MIL-STD-882 and this methodology is relatively new to the agency. The agency assesses prioritized hazards using GET’s Safety Risk Matrix (Figure 32). This matrix expresses assessed risk as a combination of one severity category and one likelihood level—for example, “1A” or the combination of a Catastrophic (1) severity category and a Highly Probable (A) probability level. The matrix also categorizes combined risks into levels—High, Medium, Low, or Very Low—based on the likelihood of occurrence and severity of the outcome. This categorization allows for hazards to be prioritized for mitigation based on their associated safety risk, such as for the purposes of accepting risk: • “High” hazard ratings will be considered unacceptable and require action from GET to mitigate the safety risk. • “Medium” hazard ratings will be considered undesirable and require GET’s safety committee to make a decision regarding the rating’s acceptability. • “Low” hazard ratings may be accepted by the SMS executive with review. Benefits and Challenges To GET, simplicity is a main benefit of the modified MIL-STD-882. The new method could appear cumbersome at the beginning, which is common when standardizing what used to be informal procedures, but this agency has overcome the hurdle. GET found that the best way to encourage employees to report hazards and other safety concerns is to get the identified ones fixed, even small issues. For example, the line near the gate that reminds the drivers to not get too close was faded. GET repainted that line, as well as the lines in the yard that designate park- ing. The agency also worked with the city to fix the “tire slices,” which occur when the grate for the drainage becomes sharp. Completing improvements, even small ones like these examples, encourages more employees to report hazards and other safety concerns because they see that the agency is addressing reported concerns. The safety committee receives reports from all employees, from administrators to mechanics. Although the modified MIL-STD-882 meets the agency’s current needs, GET is not attached to the method. As the agency learns more about the SRA over time, it will consider adopting other methodologies that are easier to use or that suit the agency better. How Safety Risk Assessment Is Used GET has been using the SRA process as a part of its Safety and Security Certification process for major capital projects, on hazards and other safety concerns that are identified through inspections and internal audits, and on hazards and other safety concerns reported by employees

Case Examples 47   and/or riders. For instance, GET conducted the SRA when adding an additional bay for its hydrogen fueling stations. Hazards and other safety concerns are recorded and tracked using a spreadsheet. Because not every employee knows how to use the spreadsheet, a hazard identifi- cation (ID) card was developed as an alternative for reporting. In this way, hazards and other safety concerns are being tracked and later discussed at the safety committee meetings. The safety committee was established following the requirements of GET’s PTASP. Prior to that, there were agency meetings tied to the environmental assessment plan, but not enough commitment to safety. Now the agency has put more emphasis on safety issues, transitioning its Source: Golden Empire Transit District (2020). Figure 32. GET’s Safety Risk matrix.

48 Transit Safety Risk Assessment Methodologies regime from reactive to less formal and more proactive. The safety committee has representa- tion from every department, and drivers are invited as well. It is challenging to cover the cost of having drivers attend the meetings, but GET manages to make it happen. There are a total of 15 committee members. Risk assessment is the primary responsibility of the chief safety officer, and that responsibility is also given to the committee. Resources Funding is the most significant challenge that GET is facing. Once issues are identified, funding is sometimes required to address them. The emphasis on safety has not only increased the agency’s facility and planning budget, but also required additional manpower to work on solutions. The agency does not have a safety analyst at this point. Training requires funding as well. Currently the agency teaches drivers about the hazard ID cards and other new things by catching the drivers on their way to the buses through the agency’s patio. Drivers also some- times get off buses and attend training classes. The cost will grow as the agency expands its on-demand service.