Analysis of Aircraft Overruns and Undershoots for Runway Safety Areas (2008)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

39 Upon developing the risk models, it is necessary to inte- grate them to allow risk assessment of RSAs under specific conditions. A practical application of such models for exam- ining a specific RSA will involve estimating the risk that an aircraft operating under specific airport conditions will chal- lenge the existing RSA and stop beyond the available RSA limits or crash into existing obstacles. This chapter includes a step-by-step procedure to evaluate an RSA using the models and the approach developed in this study. The example presented is for a hypothetical Runway 07/25, but actual NOD for an existing airport was collected from 2002 to 2004 and is used to illustrate the practical application of the approach developed. The procedure repre- sented in Figure 34 will allow assessing the probability that an incident will occur during the aircraft operation and that as a result, the consequences are likely to be severe. The initial process will involve the following steps: The initial process will involve the following steps: 1. Select the specific RSA to analyze and gather information to include dimensions, type of terrain, and existing obsta- cles adjacent to the RSA. 2. Collect (or estimate) a representative traffic sample for the three crash scenarios (LDOR, LDUS, and TOOR) chal- lenging the RSA being evaluated. 3. Divide the RSA into sections comprising specific crash scenarios. Each scenario should include the distance from the threshold and two lateral distances relative to the ex- tended runway axis. 4. Using the frequency models and location models, estimate for every operation the probability for each crash scenario and type of operation. Based on these data, develop the frequency distribution for each crash scenario and for each type of incident (LDOR, LDUS, and TOOR). 5. For each type of operation, determine the percentage of oc- currences having a risk higher than a selected level or TLS. 6. Determine the weighted frequency distribution for all types of incidents together. 7. Repeat the analysis for the remaining runway ends of the airport. 8. Account for the risk exposure and estimate the probabil- ity, in terms of accidents per year, for each type of incident and for each RSA of the airport. 9. Classify the RSA according to the percentage of high-risk operations. Step 1—RSA Details The RSA to be evaluated should be characterized by type of terrain, dimensions, and type, size, and location of obstacles. The analysis is for the approach end of Runway 25 and is a simple example because the obstacles that can cause severe consequences are well defined; the RSA is rectangular and symmetric, and it is surrounded by a body of water, as de- picted in Figure 35. Step 2—Collect Representative Traffic Sample This step will involve gathering normal operations data for the airport. A second alternative for this step is to estimate the expected operations based on the planned flight schedule for the facility. When using historical data, two possible sources are: the Aviation System Performance Metrics (ASPM) data, which identify the takeoff and landing runways in hourly seg- ments, the runway, and direction for each flight; and the Enhanced Traffic Management System Counts (ETMSC), which provides information on traffic counts by airport or by city pair for various data groupings. These same sources were used to obtain NOD for this study. The sample flight data are for operations that may challenge the RSA being evaluated in the event of an aircraft overrun or undershoot. C H A P T E R 4 Practical Application of Models

40 Figure 35. RSA on approach end of Runway 25. Figure 34. RSA risk assessment. The information on flights should be coupled with weather conditions obtained from the airport weather station. The rep- resentative sample (e.g., 1 year) should contain the informa- tion on the parameters used in the frequency models (Table 7). Aircraft operations challenging the RSA at the runway end shown in Figure 35 are those arriving on Runway 25 (LDUSs) or arriving and departing from Runway 07 (LDORs and TOORs). To run the analysis, a sample of the traffic landing and taking off on Runway 07 and landing on Runway 25 is necessary. The sample must be representative of the existing or planned conditions for Runway 07/25. Preferably, the sample should cover at least 1 year of operations, but it may not be necessary to obtain data for each month. A sample comprising at least 4 months of operations, but characteriz- ing the whole year should suffice (e.g., January, April, July, and October), such that different seasons, environmental conditions, and seasonal variations of traffic are represented. Between 2002 and 2004, a sample of historical data from an existing airport was gathered for this example. Data include landing operations for Runway 25, landing operations for Runway 07, and takeoff operations for Runway 07. In the event of an incident, these flights may challenge the RSA for this example. Step 3—Define Crash Scenarios This step still requires some judgment from the analyst. In the example shown in Figure 35, the RSA adjacent to the Run- way 25 approach end is only 100 ft long by 500 ft wide. When applying the risk assessment procedure to a partic- ular RSA, it is important to recognize that not every RSA has a standard or rectangular shape. In many cases, a combina- tion of the shapes and sizes as well as obstacles may exist. Other factors, such as the type of obstacle, will often need a

41 Parameter Value Note Aircraft B738 Boeing 737-800 Equipment Class N Large jet User Class C Commercial Equipment Type J Jet Wind Direction 340 Wind Speed Knts 13 Ceiling ft 500 Visibility Sm 1.0 Temperature C 30 Fog 1 Fog present Icing 0 No icing Elec. Storm 0 No elect. Storm Frozen Precipitation 0 No frozen precip. Snow 0 No snow XWind Knts 13 calculated Rain 0 No rain Foreign Origin/Dest. 0 Domestic Significant Terrain 0 No significant terrain Table 15. Example normal operation on Runway 07, hypothetical airport. subjective assessment and the use of some simplification to run the risk analysis. For the RSA configuration in the example, there are only four crash scenarios that may lead the aircraft to fall into the water if overrunning or undershooting the runway: Crash Scenario 1. During a landing or takeoff operation on Runway 07, the aircraft will overrun the RSA within its 500 ft wide boundaries and fall after running 100 ft. Crash Scenario 2. During a landing on Runway 25, the air- craft will undershoot the RSA within 500 ft wide boundaries and before the RSA end, 100 ft from the threshold. Crash Scenario 3. Same as Crash Scenario 1, but the aircraft will fall into the water outside the 500 ft wide boundaries of the RSA. Crash Scenario 4. Same as Crash Scenario 3, but the aircraft will undershoot before the runway end and outside the 500 ft wide boundaries of the RSA. It should be noted that the small number of crash scenar- ios identified is for this simple example only. The presence of obstacles or the asymmetry of the RSA would lead to addi- tional possible crash scenarios. For example, if the RSA had an ILS structure, two additional crash scenarios should be considered: Crash Scenario 5. During a landing or takeoff operation on Runway 07, the aircraft will overrun the RSA within the cen- ter area of the RSA and hit the ILS structure with enough energy for severe consequences. Crash Scenario 6. Same as Crash Scenario 5, but the aircraft will undershoot in the RSA, before the ILS structure and strike it. In this case, Crash Scenario 1 should be modified so that the aircraft would overrun or undershoot the runways between the RSA 500ft boundaries in a path that avoids the ILS structure and the aircraft falls into the water. No modi- fication would be required for Crash Scenario 2, when the aircraft undershoots the runway before the beginning of the RSA. Step 4—Estimate the Risk For each operation in the NOD sample, it is necessary to calculate the risk. An example calculation for one landing op- eration will be described to help understanding of the process. Suppose one of the landing operations on Runway 07 is characterized by the parameters depicted in Table 15. The information is used in Equation 7 to compute the probability the aircraft will overrun Runway 07 during a landing operation under these conditions, thus challeng- ing the RSA being evaluated. The computation is the following: The probability that an aircraft will not be able to stop dur- ing a landing operation and overrun the runway under these conditions is therefore computed as follows: As expected, the risk of an incident is high for such unfa- vorable conditions: strong cross wind, low visibility, and low ceiling. After calculating the likelihood of an overrun, the next step in the process is to estimate probability the aircraft will stop beyond the RSA or hit an existing obstacle. The area surrounding the RSA is a body of water. In this case, it is possible to assume the aircraft will be lost if it over- runs the RSA and falls into the water. Distance x from the runway end is simply the length of RSA measuring 100 ft, and this is the value to be entered in the normalized x-location model given by Equation 13. It should be noted that the parameter “Δ” from general Equation 25 in this case should be set to 0 because the aircraft will fall into the water if its stop location simply exceeds 100 ft beyond the threshold. The calculation is given by: The probability an aircraft will end in the water if over- running the runway during landing is high because the RSA is very short. P d x e{ } %. .> = =− ×0 004692 1000 824513 81 P d x e x{ } . .> = −0 004692 0 824513 P LDOR e { } . . = + = × − 1 1 8 165 10 9 413 5 b = − + − − −15 456 0 551 0 2 113 0 1 064 0 0 876. . ( ) . ( ) . ( ) . (0 0 445 0 0 857 0 1 832 1 1 639 0 2 4 ) . ( ) . ( ) . ( ) . ( ) .+ − + + + 28 1 1 186 0 1 741 0 0 322 0 0 532 0 ( ) . ( ) . ( ) . ( ) . ( )+ + + − +1 566 0 1 518 1 0 986 0 1 926 0 1 499 . ( ) . ( ) . ( ) . ( ) . (+ + + + 0 1 009 0 0 631 0 0 265 1 1 006 0 0 9 ) . ( ) . ( ) . ( ) . ( ) . − − + + + 24 0 9 413( ) .= −

42 As previously described, it is assumed the aircraft will over- run in a path that is nearly parallel to the runway axis. The probability that the aircraft will overrun the RSA under Crash Scenario 1 is given by a product of three probabilities: the probability of occurring the event (overrun); the probability the aircraft will stop beyond 100 ft from the threshold; and the probability the aircraft will remain within the RSA lateral limits during the overrun. Mathematically: Prob{CS 1} = Prob{LDOR) × Prob{x > 100ft} × (1−Prob{| y | > 250 ft) And Prob{| y | > 250 ft} is that given by: Probability for crash scenario 1 is: 8.165 × 10−5 × 0.81 × (1−0.050) = 6.28 × 10−5 Similarly, the probability for crash scenario 2 is simply given by: Prob{CS 3) = Prob{LDOR) × Prob{| y | > 250 ft In this situation the aircraft only needs to overrun the run- way by a small margin; as long as the transverse deviation is greater than 250 ft, the aircraft will end up in the water. The probability for crash scenario 3 is calculated as follows: P{CS 3} = 8.165 × 10−5 × 0.050 = 4.083 × 10−6 The last step is to compute the total probability the aircraft will overrun the runway during landing and fall into the water with severe consequences: P{LDOR Severe} = P{CS 1} + P{CS 3} = 6.28 × 10−5 + 4.08 × 10−6 = 6.69 × 10−5 According to the FAA criteria described in Table B1-2 from Attachment 1 to Appendix B, the event is probable and such probability is unacceptable, as shown in the FAA Risk Matrix depicted in the same Attachment (Figure B1-1), even when P d y e{ } . %. .> = =− ×0 20174 2500 489009 5 0 there are major consequences. Considering the criteria set by FAA, the risk is considered high and suggests the operation is not safe enough under such conditions. However, the risk estimated for such conditions at any airport should be con- sidered for planning and risk mitigation strategies only as it represents an “average” risk level for such conditions. Step 5—Characterize Risk Frequency Distribution The same procedure described for Step 3 can be used to compute the probability of severe consequences for every landing operation for Runway 07 that is part of the NOD sam- ple. Each operation has a different risk associated with it. If all these risks are estimated, it is possible to build a histogram depicting the distribution of risk, as illustrated in Figure 36. Step 6—Determine Percentage of Operations with Risk Above TLS An example of the probability distribution generated by the prototype software developed under this project is shown in Figure 37. Each bar represents the percentage of operations having a specific risk level. The line represents the percentage of operations having risk higher then the level selected. In this example, if a TLS of 1:10000000 is selected, approximately 9 percent of the operations will be subject to undesirable levels of risk. This is useful, as it evaluates the percentage of opera- tions with risk above a selected TLS. The area in dark bars represents such flights. The same process is used to estimate the percentage of operations having a risk level above the selected TLS for LDUSs on Runway 25 and for TOORs on Runway 07, using the models associated with these types of events. The proba- bility distribution of risks then can be characterized for each type of accident in the vicinity of that RSA. X is the RSA length or distance to existing obstacle Frequency distribution estimated from expected traffic conditions at the airport/threshold and risk models Probability WL > X Fr eq ue nc y Region of Normal Operations And Low Risk Region of Normal Operations and High Risk Risk threshold (e.g. 1x10-7) Figure 36. Frequency distribution of risk.

43 LDOR Probability Distribution of Risk 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 3. 36 E- 09 1. 41 E- 08 2. 49 E- 08 3. 56 E- 08 4. 63 E- 08 5. 71 E- 08 6. 78 E- 08 7. 86 E- 08 8. 93 E- 08 1. 00 E- 07 1. 11 E- 07 1. 22 E- 07 1. 32 E- 07 1. 43 E- 07 M or e Risk Fr eq ue nc y % Ops @ Risk Level % Ops @ Higher Risk Figure 37. Typical frequency distribution of risk from prototype software. Step 7—Total Percentage of Operations with Risk Above TLS The three probability distributions (LDOR RWY 07, LDUS RWY 25, and TOOR RWY 07) can be combined to provide an overall probability distribution that a serious accident will occur in the specific RSA. The percentages for each cell should be weighted relative to the traffic for each type of operation. A hypothetical example is presented to help un- derstanding of the process. Assuming these are the conditions for the airport being evaluated if the TLS is set to 1 × 10−6, results for the approach end of Runway 25 are summarized on Table 16. There are significant differences to risk levels for each op- eration, particularly noting that 34 percent of landings on Runway 07 have risk above the threshold level selected for this analysis. But there also are notable discrepancies in terms of exposure to visibility, ceiling height, and fog between land- ings on Runway 07 and those on Runway 25. Over 30 percent of landings on Runway 07 take place in visibility under 2 statute miles, compared to 1.67 percent on Runway 25. In a related measure, almost 40 percent of land- ings on Runway 07 experienced fog versus less than 7 percent for Runway 25. Moreover, 39 percent of landings on Runway 07 took place in ceiling height under 1000 ft, while the equiv- alent for Runway 25 is only 3.9 percent. These differences are most likely related to the airport’s runway use policy. In fact, data on the airport’s runway usage patterns revealed that landings on Runway 07 are relatively rare. It is reasonable to assume this runway is used for land- ings only for exceptional circumstances, such as adverse wind conditions, and this also would explain the discrepancy in risk exposure for landings on Runway 07 and Runway 25. Using these numbers, the percentage operations with risk above the TLS and challenging the RSA being evaluated can be estimated as follows: Based on the annual number of operations at the airport, the analysis also can be used to estimate the annual rate of accidents or the number of years likely to take for a severe accident to occur. These are useful parameters as they allow comparing different RSAs of the same or different airports to identify and prioritize the RSAs requiring risk management actions or improvements. Steps 8 and 9—Repeat the Analysis for Other Runway Ends In repeating the analysis for the remaining runway ends of the same airport, it is possible to evaluate which RSA poses % ( . . . . . . ) ( . > = × + × + × TLS 0 073 0 34 0 09 0 01 0 3 0 02 0 073 + + = 0 09 0 3 6 9 . . ) . % Operation % of Movements Ops @ High Risk Sample Size Landing on RWY 07 7.3% 34% 487 Takeoff on RWY 07 9.0% 1% 606 Landing on RWY 25 30.0% 2% 2020 Table 16. Summary results for hypothetical airport.

44 higher risk during the year. The assessment may help improve runway use so that risk is minimized for the airport. To com- pare different RSAs, it is preferred to use a parameter that can provide a direct comparison. One possibility is to transform the percentage of operations above TLS into a volume of annual operations above TLS by multiplying the percentage by the vol- ume of flights challenging the specific runway end. Using such a parameter, it is possible to compare multiple RSAs of the same airport, or even for different facilities, to determine and priori- tize which RSAs may require risk mitigation actions. Using another hypothetical example, the following param- eters can characterize the conditions for an existing airport having two runways: 11/29, and 03/21. In Table 17, the approach end for Runway 11 has a higher number of annual operations with risk above the selected TLS. A number of risk mitigation measures may be priori- tized to improve safety for such area. Some safety manage- ment alternatives may include modifying runway use for the airport, installing new NAVAIDS, or improving the RSA, among other procedures. RSA Annual Ops Challenging RSA %Ops>TLS Annual Ops w/ Risk Above TLS Approach End RWY 11 300,000 6% 18,000 Approach End RWY 29 160,000 7% 11,200 Approach End RWY 03 410,000 1% 4,100 Approach End RWY 21 40,000 14% 5,600 Table 17. Summary risk for hypothetical airport.