**Suggested Citation:**"Chapter 3 - Modeling RSA Risk." National Academies of Sciences, Engineering, and Medicine. 2011.

*Improved Models for Risk Assessment of Runway Safety Areas*. Washington, DC: The National Academies Press. doi: 10.17226/13635.

**Suggested Citation:**"Chapter 3 - Modeling RSA Risk." National Academies of Sciences, Engineering, and Medicine. 2011.

*Improved Models for Risk Assessment of Runway Safety Areas*. Washington, DC: The National Academies Press. doi: 10.17226/13635.

**Suggested Citation:**"Chapter 3 - Modeling RSA Risk." National Academies of Sciences, Engineering, and Medicine. 2011.

*Improved Models for Risk Assessment of Runway Safety Areas*. Washington, DC: The National Academies Press. doi: 10.17226/13635.

**Suggested Citation:**"Chapter 3 - Modeling RSA Risk." National Academies of Sciences, Engineering, and Medicine. 2011.

*Improved Models for Risk Assessment of Runway Safety Areas*. Washington, DC: The National Academies Press. doi: 10.17226/13635.

**Suggested Citation:**"Chapter 3 - Modeling RSA Risk." National Academies of Sciences, Engineering, and Medicine. 2011.

*Improved Models for Risk Assessment of Runway Safety Areas*. Washington, DC: The National Academies Press. doi: 10.17226/13635.

**Suggested Citation:**"Chapter 3 - Modeling RSA Risk." National Academies of Sciences, Engineering, and Medicine. 2011.

*Improved Models for Risk Assessment of Runway Safety Areas*. Washington, DC: The National Academies Press. doi: 10.17226/13635.

**Suggested Citation:**"Chapter 3 - Modeling RSA Risk." National Academies of Sciences, Engineering, and Medicine. 2011.

*Improved Models for Risk Assessment of Runway Safety Areas*. Washington, DC: The National Academies Press. doi: 10.17226/13635.

**Suggested Citation:**"Chapter 3 - Modeling RSA Risk." National Academies of Sciences, Engineering, and Medicine. 2011.

*Improved Models for Risk Assessment of Runway Safety Areas*. Washington, DC: The National Academies Press. doi: 10.17226/13635.

**Suggested Citation:**"Chapter 3 - Modeling RSA Risk." National Academies of Sciences, Engineering, and Medicine. 2011.

*Improved Models for Risk Assessment of Runway Safety Areas*. Washington, DC: The National Academies Press. doi: 10.17226/13635.

**Suggested Citation:**"Chapter 3 - Modeling RSA Risk." National Academies of Sciences, Engineering, and Medicine. 2011.

*Improved Models for Risk Assessment of Runway Safety Areas*. Washington, DC: The National Academies Press. doi: 10.17226/13635.

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

13 The analysis of RSA risk requires three models that con- sider probability (frequency), location and consequences. The outcome of the analysis is the risk of accident during runway excursions and undershoots. The three model approach is represented in Figure 15. The first model is used to estimate the probability that an event will occur given certain operational conditions. This probability does not address the likelihood that the aircraft may strike an obstacle or will stop beyond a certain distance. The model uses independent variables associated with causal and contributing factors for the incident. For example, under tailwind conditions it is more likely that an overrun will occur, and this is one of the factors used in the models for overruns. The aircraft performance is represented by the interaction be- tween the runway distance required by the aircraft for the given conditions and the runway distance available at the air- port. Although human and organizational factors are among the most important causes of aircraft accidents, it was not pos- sible to directly incorporate these factors into the risk models. Since this model is specific for the event type, five different models are required, e.g., one for landing overruns and another one for takeoff overruns. The second component is the location model. The analyst usually is interested in evaluating the likelihood that an air- craft will depart the runway and stop beyond the RSA or strike an obstacle. The location model is used to estimate the probability that the aircraft stops beyond a certain distance from the runway. As pointed out in ACRP Report 3 and by Wong (2007), the probability of an accident is not equal for all locations around the airport. The probability of an accident in the proximity of the runways is higher than at larger distances from the runway. Since this model is specific for the event type, five different models are required, e.g., one for landing overruns and another one for takeoff overruns. The last part is the consequence model. This model uses the location models to assess the likelihood that an aircraft will strike an obstacle or depart the RSA and fall into a drop in the terrain or into a water body adjacent to the RSA. In ad- dition, it takes into consideration the type of obstacle and the estimated collision speed to cause severe consequences. For example, an aircraft colliding with a brick building may result in severe consequences even at low speeds; however, the air- craft must be at a higher speed when striking a Localizer an- tenna mounted on a frangible structure for a similar level of severity. The collision speed is evaluated based on the loca- tion of the obstacle and the typical aircraft deceleration for the type of RSA terrain. The ensuing sections provide details for each component of the risk model. The same model is used for all five types of events (LDUS, LDOR, LDVO, TOOR, and TOVO.) The remainder of this chapter discusses the probability and location models. The consequence model is discussed in Chapter 4. Event Probability (Frequency Model) Similar to ACRP Report 3 model development procedures, backward stepwise logistic regression was used to calibrate five frequency models, one for each type of incident: LDOR, LDUS, LDVO, TOVO, and TOOR. Various numerical tech- niques were evaluated to conduct the multivariate analysis, and logistic regression was the preferred statistical procedure for a number of reasons. This technique is suited to models with a dichotomous outcome (accident and non-accident) with multiple predictor variables that include a mixture of continuous and categorical parameters. Also, it is particu- larly appropriate for case-control studies because it allows the use of samples with different sampling fractions, depend- ing on the outcome variable without giving biased results. Backward stepwise logistic regression was used to calibrate the frequency models because of the predictive nature of the research, and the technique is able to identify relationships missed by forward stepwise logistic regression (Hosmer and Lemeshow 2000). C H A P T E R 3 Modeling RSA Risk

To avoid the negative effects of multi-co-linearity on the model, correlations between independent variables were first tested to eliminate highly correlated variables, particularly if they did not significantly contribute to explaining the varia- tion of the probability of an accident. The basic model structure selected is a logistic equation, as follows: where P{Accident_Occurrence} = the probability (0â100%) of an accident type occurring given certain operational conditions; Xi = independent variables (e.g., ceiling, visibility, crosswind, precipitation, aircraft type, cri- ticality factor); and bi = regression coefficients. Several parameters were considered for inclusion in the models. The backward stepwise procedure helps identify those variables that are relevant for each type of event. Some of the independent variables are converted to binary form to avoid spurious effects of non-linear relationships in the model ex- ponent. These binary variables are represented by only two values, 0 or 1. In this case, the presence of the factor (e.g., rain) is represented by 1, and the absence of the factor (e.g., no rain) is represented by 0. One significant improvement relative to the models pre- sented in ACRP Report 3 is the use of tailwind and headwind. These variables were not present in previous models because P Accident Occurence eb b X b X b X _{ } = + + + + 1 1 0 1 1 2 2 3 3+ . . . the actual runway had not been identified for the NOD. The research team has gathered information on the runways used, and the process allowed the calculation of the head/tailwind components to be included in the model. Another major improvement that has increased model ac- curacy was the inclusion of a runway criticality factor. The basic idea was to include a new parameter that could repre- sent the interaction between the runway distance required by the aircraft and the runway distance available at the airport. The log of the ratio between the distance required and the dis- tance available was used. Positive values represent situations when the distance available was smaller than the distance re- quired, and in this case, risk will be higher. The greater the value is, the more critical is the operation because the safety margin decreases. The distance required is a function of the aircraft perform- ance under specific conditions. Therefore, every distance re- quired under International Standards Organization (ISO) conditions (sea level, 15 degrees centigrade) was converted to actual conditions for operations. Moreover, the distances were adjusted for the runway surface condition (wet, snow, slush, or ice) and for the level of head/tailwind. The adjust- ment factors applied to the distance required are presented in Table 2. A correction for slope was not applied to adjust the total distance required. The use of NOD in the accident frequency model was a major improvement introduced by Wong et al. (2006) and was maintained for this study. The analysis with NOD also adds to the understanding of cause-result relationships of the five accident types. The technique used to develop the models is able to iden- tify relationships missed by forward stepwise logistic regres- sion (Hosmer and Lemeshow 2000). The predictor variables 14 Three-Part Risk Model Event probability Location probability operating conditions (airplane performance, type of operation, runway distance available and elevation, weather conditions) RSA characteristics, geometry, presence of EMAS type, size and location of obstacles Consequences Risk Classification Figure 15. Modeling approach.

were entered by blocks, each consisting of related factors, such that the change in the modelâs substantive significance could be observed as the variables were included. Table 3 summarizes the model coefficients obtained for each model. Table 4 summarizes the parameters representing the ac- curacy of each model obtained. The table presents the R2 and C-values for each model. It is important to note that rel- atively low R2 values are the norm in logistic regression (Ash and Schwartz 1999) and they should not be compared with the R2 of linear regressions (Hosmer and Lemeshow 2000). A better parameter to assess the predictive capability of a logistic model is the C-value. This parameter represents the area under the sensitivity/specificity curve for the model, which is known as Receiver Operating Characteristic (ROC) curve. Sensitivity and specificity are statistical measures of the per- formance of a binary classification test. Sensitivity measures the proportion of true positives that are correctly identified as such (the percentage of accidents and incidents that are cor- rectly identified when using the model). Specificity measures the proportion of true negatives that are correctly identified (the percentage of normal operations that the model can cor- rectly identify as non-incident). These two measures are closely related to the concepts of Type I and Type II errors. A theoret- ical, optimal prediction can achieve 100% sensitivity (i.e., pre- dict all incidents) and 100% specificity (i.e., not predict a normal operation as an incident). A perfect model has a C-value equal to 1.00. To assess how successful the models are in classifying flights correctly as âaccidentâ or ânormalâ and to find the appropri- ate cut-off points for the logistic regression models, the ROC curves were defined for each model to calculate the C-value. An example of this assessment is shown in Figure 16 repre- senting the model for landing overruns. The area under the curve for this model represents the C-value and is 87.4%. The C-values for each of the five models developed were above 78% and are considered excellent models. The cut-off point is the critical probability above which the model will class an event as an accident. The ROC curve plots all potential cut-off points according to their respective True Positive Rates and False Positive Rates. The best cut-off point has an optimally high sensitivity and specificity. Event Location Models The accident location models are based on historical acci- dent data for aircraft overruns, veer-offs, and undershoots. The accident location for overruns depends on the type of ter- rain (paved or unpaved) and if EMAS is installed in the RSA. When EMAS is available, during landing and takeoff over- runs, the aircraft will stop at shorter distances, and typical de- celeration for the type of aircraft is used to assess the location probability. Worldwide data on accidents and incidents were used to develop the location models. The model structure is similar to the one used by Eddowes et al. (2001). Based on the accident/ incident location data, five sets of complementary cumulative probability distribution (CCPD) models were developed. With CCPDs, the fraction of accidents involving locations exceeding a given distance from the runway end or threshold can be esti- mated. When the CCPD is multiplied by the frequency of ac- cident occurrence, a complementary cumulative frequency 15 Local Factor Unit Reference Adjustment Elevation (E) (i) 1000 ft E = 0 ft (sea level) Fe = 0.07 x E + 1 Temperature (T) (i) deg C T = 15 deg C Ft = 0.01 x (T â (15 â 1.981 E) + 1 Tailwind (TWLDJ) for Jets (iii) knot TWLDJ = 0 knot FTWJ = (RD + 22 x TWLDJ)/RD (ii) Tailwind (TWLDT) for Turboprops (iii) knot TWLDT = 0 knot FTWJ = (RD + 30 x TWLDT)/RD Headwind (HWTOJ) for Jets (iii) knot HWTOJ = 0 knot FTWJ = (RD + 6 x HWTOJ)/RD Headwind (HWTOT) for Turboprops (iii) knot HWTOJ = 0 knot FTWJ = (RD + 6 x HWTOT)/RD Runway Surface Condition â Wet (W) (iv) Yes/No Dry FW = 1.4 Runway Surface Condition â Snow (S) (iv) Yes/No Dry FS = 1.6 Runway Surface Condition â Slush (Sl) (iv) Yes/No Dry FSl = 2.0 Runway Surface Condition â Ice (I) (iv) Yes/No Dry FI = 3.5 i â RD is the runway distance required in feet ii â temperature and elevation corrections used for runway design iii â correction for wind are average values for aircraft type (jet or turboprop) iv â runway contamination factors are those suggested by Flight Safety Foundation Table 2. Adjustment factors used to correct required distances.

16 Variable LDOR LDUS LDVO TOOR TOVO Adjusted Constant -13.065 -15.378 -13.088 -14.293 -15.612 User Class F 1.693 1.266 User Class G 1.539 1.288 1.682 2.094 User Class T/C -0.498 0.017 Aircraft Class A/B -1.013 -0.778 -0.770 -1.150 -0.852 Aircraft Class D/E/F 0.935 0.138 -0.252 -2.108 -0.091 Ceiling less than 200 ft -0.019 0.070 0.792 Ceiling 200 to 1000 ft -0.772 -1.144 -0.114 Ceiling 1000 to 2500 ft -0.345 -0.721 Visibility less than 2 SM 2.881 3.096 2.143 1.364 2.042 Visibility from 2 to 4 SM 1.532 1.824 -0.334 0.808 Visibility from 4 to 8 SM 0.200 0.416 0.652 -1.500 Xwind from 5 to 12 kt -0.913 -0.295 0.653 -0.695 0.102 Xwind from 2 to 5 kt -1.342 -0.698 -0.091 -1.045 Xwind more than 12 kt -0.921 -1.166 2.192 0.219 0.706 Tailwind from 5 to 12 kt 0.066 Tailwind more than 12 kt 0.786 0.98 Temp less than 5 C 0.043 0.197 0.558 0.269 0.988 Temp from 5 to 15 C -0.019 -0.71 -0.453 -0.544 -0.42 Temp more than 25 C -1.067 -0.463 0.291 0.315 -0.921 Icing Conditions 2.007 2.703 2.67 3.324 Rain 0.991 -0.126 0.355 -1.541 Snow 0.449 -0.25 0.548 0.721 0.963 Frozen Precipitation -0.103 Gusts 0.041 -0.036 0.006 Fog 1.74 Thunderstorm -1.344 Turboprop -2.517 0.56 1.522 Foreign OD 0.929 1.354 -0.334 -0.236 Hub/Non-Hub Airport 1.334 -0.692 Log Criticality Factor 9.237 1.629 4.318 1.707 Night Conditions -1.36 Where: Equipment Class Ref: C Large jet of MTOW 41k-255k lb (B737, A320 etc.) Heavy Acft AB Heavy jets of MTOW 255k lb+ (B777, A340, etc.) Commuter Acft D Large commuter of MTOW 41k-255k lb (Regional Jets, ERJ-190, CRJ-900, ATR42, etc.) Medium Acft E Medium aircraft of MTOW 12.5k-41k lb (biz jets, Embraer 120, Learjet 35 etc.) Small Acft F Small aircraft of MTOW 12.5k or less (small, Beech-90, Cessna Caravan, etc.) User Class Ref: C = Commercial User Class F Cargo User Class T/C Taxi/Commuter User Class G General Aviation Foreign OD Foreign origin/destination (yes/no) - Ref: domestic Ceiling (feet) Ref: Ceiling Height > 2500 ft Visibility (Statute Miles) Ref: Visibility > 8 SM Crosswind (knots) Ref: Crosswind < 2 kt Tailwind (knots) Ref: Tailwind < 5 kt Gusts (knots) Ref: No gusts Thunderstorms (yes/no) Ref: No thunderstorms Icing Conditions (yes/no) Ref: No icing conditions Snow (yes/no) Ref: No snow Rain (yes/no) Ref: No rain Fog (yes/no) Ref: No fog Air Temperature (deg C) Ref: Air temperature above 15 C and below 25C Non-Hub Airport (yes/no) Ref: Hub airport Log Criticality Factor If Log(CF) > 0, available runway distance is smaller than required distance Notes: Ref : indicates the reference category against which the odds ratios should be interpreted. Non-hub airpor t : airport having less than 0.05% of annual passenger boardings. Table 3. Independent variables used for frequency models.

distribution (CCFD) is obtained. The latter quantifies the over- all frequency of accidents involving locations exceeding a given distance from the runway boundaries. Figures 17 to 19 show the axis locations used to represent each type of incident. The reference location of the aircraft is its nose wheel. For overruns and undershoots, the x-y origin is the centerline at the runway end. For veer-offs, the y-axis origin is the edge of the runway, not necessarily the edge of the paved area when the runway has shoulders. For the longitudinal distribution, the basic model is: where P{Location > x} = the probability the overrun/undershoot distance along the runway centerline be- yond the runway end is greater than x; x = a given location or distance beyond the runway end; and a, n = regression coefficients. A typical longitudinal location distribution is presented in Figure 20. P Location x e axn>{ } = â For the transverse distribution, the same model structure was selected. However, given the accidentâs transverse loca- tion for aircraft overruns and undershoots, in general, is not reported if the wreckage location is within the extended runway lateral limits, it was necessary to use weight factors to reduce model bias, particularly for modeling the tail of the probability distribution. The model can be represented by the following equation: where P{Location>y} = the probability the overrun/undershoot distance from the runway border (veer- offs) or centerline (overruns and under- shoots) is greater than y; y = a given location or distance from the extended runway centerline or runway border; and b, m = regression coefficients. A typical transverse location distribution is presented in Figure 21, and the model parameters are presented in Table 5. P Location y e bym>{ } = â 17 Model R2 C LDOR 0.28 0.87 LDUS 0.14 0.85 LDVO 0.32 0.88 TOOR 0.11 0.78 TOVO 0.14 0.82 Table 4. Summary statistics for frequency models. Figure 16. ROC curve for LDOR frequency model. x y Figure 17. X-Y origin for aircraft overruns.

18 x y Figure 18. X-Y origin for aircraft undershoots. y Figure 19. Y origin for aircraft veer-offs. x1 P{Loc > x1} naxexLocationP }{ X rwy end Distance x from runway end Pr ob ab ilit y lo ca tio n Ex ce ed s x Figure 20. Typical model for aircraft overruns. mbyeyLocationP }{ yDistance y from runway bordery1rwy border P{Loc > y1} Pr ob ab ilit y lo ca tio n Ex ce ed s y Figure 21. Typical model for aircraft veer-offs. For Table 5, the following are the parameters represented: d = any given distance of interest; x = the longitudinal distance from the runway end; P{d>x} = the probability the wreckage location exceeds dis- tance x from the runway end; y = the transverse distance from the extended runway centerline (overruns and undershoots) or from the runway border (veer-offs); and P{d>y} = the probability the wreckage location exceeds dis- tance y from the extended runway centerline (over- runs and undershoots) or from the runway border (veer-offs). Figures 22â29 illustrate the models presented in Table 5. EMAS Deceleration Model The analysis tool developed in this research includes the capability to evaluate RSAs with EMAS beds. The details of the development are presented in Appendix E. The same location models for overruns are used when EMAS beds are available in the RSA. However, the bed length is adjusted for each type of aircraft according to MTOW and the EMAS bed length, according to the follow- ing steps: 1. The maximum runway exit speed to hold the aircraft within the EMAS bed is calculated according to the model presented below. The adjusted R2 for this model is 89%. where: v = the maximum exit speed in m/s; W = the maximum takeoff weight of the aircraft in kg; and S = the EMAS bed length in meters. 2. The maximum runway exit speed estimated using the pre- vious regression equation, along with the EMAS bed length (SEMAS), is input in the following equation to calculate the deceleration of the aircraft in the EMAS bed. 3. The runway length factor is then estimated as follows: 4. The runway length factor is then estimated as follows: RLF a a EMAS RSA = RLF a a EMAS RSA = a v S EMAS = 2 2 v W S= â ( )+ ( )3 0057 6 8329 31 1482. . log . log

19 Prob=exp((-.00321)*X**(.984941)) R2=99.8% 0 400 800 1200 1600 2000 Distance X from Runway End (ft) 0.0 0.2 0.4 0.6 0.8 1.0 Pr ob ab ilit y of S to pp in g Be yo nd X Figure 22. Longitudinal location model for landing overruns. Type of Accident Type of Data Model R2 # of Points X 984941.000321.0}{ xexdP 99.8% 305 LDOR Y 4862.020983.0}{ yeydP 93.9% 225 X 751499.001481.0}{ xexdP 98.7% 83 LDUS Y 773896.002159.0}{ yeydP 98.6% 86 LDVO Y 803946.002568.0}{ yeydP 99.5% 126 X 06764.100109.0}{ xexdP 99.2% 89 TOOR Y 659566.004282.0}{ yeydP 98.7% 90 TOVO Y 863461.001639.0}{ yeydP 94.2% 39 Table 5. Summary of location models. Prob=exp((-.20983)*Y**(.486)) R2=93.9% 0 200 400 600 800 1000 1200 Distance Y from Extended Runway Centerline (ft) 0.0 0.2 0.4 0.6 0.8 1.0 Pr ob ab ilit y of S to pp in g Be yo nd Y Figure 23. Transverse location model for landing overruns.

20 Prob=exp((-.01481)*X**(.751499)) R2=98.7% 0 400 800 1200 1600 2000 Distance X from Runway Arrival End (ft) 0.0 0.2 0.4 0.6 0.8 1.0 Pr ob ab ilit y of T ou ch in g Do wn B ef or e X Figure 24. Longitudinal location model for landing undershoots. Prob=exp((-.02159)*Y**(.773896)) R2=98.6% 0 200 400 600 800 1000 Distance Y from Runway Extended Centerline (ft) 0.0 0.2 0.4 0.6 0.8 1.0 Pr ob ab ilit y of T ou ch in g Do wn B ey on d Y Figure 25. Transverse location model for landing undershoots.

21 Prob=exp((-.02568)*Y**(.803946)) R2=99.5% 0 200 400 600 800 1000 Distance Y from Runway Edge (ft) 0.0 0.2 0.4 0.6 0.8 1.0 Pr ob ab ilit y of S to pp in g Be yo nd Y Figure 26. Lateral location model for landing veer-offs. Prob=exp((-.00109)*X**(1.06764)) R2=99.2% 0 400 800 1200 1600 2000 Distance X from Runway End (ft) 0.0 0.2 0.4 0.6 0.8 1.0 Pr ob ab ilit y of S to pp in g Be yo nd X Figure 27. Longitudinal location model for takeoff overruns.

22 Prob=exp((-.04282)*Y**(.659566)) R2=98.7% 0 200 400 600 800 1000 Distance Y from Extended Runway Centerline (ft) 0.0 0.2 0.4 0.6 0.8 1.0 Pr ob ab ilit y of S to pp in g Be yo nd Y Figure 28. Transverse location model for takeoff overruns. Prob=exp((-.01639)*Y**(.863461)) R2=94.2% 0 200 400 600 800 1000 Distance Y from Runway Edge (ft) 0.0 0.2 0.4 0.6 0.8 1.0 Pr ob ab ilit y of S to pp in g Be yo nd Y Figure 29. Lateral location model for takeoff veer-offs. 5. The equivalent length of conventional RSA is then cal- culated: With the equivalent RSA length, the RSA is adjusted for each type of aircraft and is input into the standard location models presented in the previous section. S a a S RLF SRSA EMAS RSA EMAS EMAS= = g Accuracy of Models There were five multivariate logistic frequency models, eight exponential location models, and one log linear deceleration model developed in this study. The accuracy of these models is considered excellent, with C-values ranging from 0.78 to 0.88 for the frequency models. The location models had R2 values ranging from 93.9% to 99.8%, and the deceleration model for EMAS presented an adjusted R2 of 89%.