National Academies Press: OpenBook

Analysis of Aircraft Overruns and Undershoots for Runway Safety Areas (2008)

Chapter: Chapter 2 - Research Approach

« Previous: Chapter 1 - Background
Page 4
Suggested Citation:"Chapter 2 - Research Approach." National Academies of Sciences, Engineering, and Medicine. 2008. Analysis of Aircraft Overruns and Undershoots for Runway Safety Areas. Washington, DC: The National Academies Press. doi: 10.17226/14137.
×
Page 4
Page 5
Suggested Citation:"Chapter 2 - Research Approach." National Academies of Sciences, Engineering, and Medicine. 2008. Analysis of Aircraft Overruns and Undershoots for Runway Safety Areas. Washington, DC: The National Academies Press. doi: 10.17226/14137.
×
Page 5
Page 6
Suggested Citation:"Chapter 2 - Research Approach." National Academies of Sciences, Engineering, and Medicine. 2008. Analysis of Aircraft Overruns and Undershoots for Runway Safety Areas. Washington, DC: The National Academies Press. doi: 10.17226/14137.
×
Page 6
Page 7
Suggested Citation:"Chapter 2 - Research Approach." National Academies of Sciences, Engineering, and Medicine. 2008. Analysis of Aircraft Overruns and Undershoots for Runway Safety Areas. Washington, DC: The National Academies Press. doi: 10.17226/14137.
×
Page 7
Page 8
Suggested Citation:"Chapter 2 - Research Approach." National Academies of Sciences, Engineering, and Medicine. 2008. Analysis of Aircraft Overruns and Undershoots for Runway Safety Areas. Washington, DC: The National Academies Press. doi: 10.17226/14137.
×
Page 8
Page 9
Suggested Citation:"Chapter 2 - Research Approach." National Academies of Sciences, Engineering, and Medicine. 2008. Analysis of Aircraft Overruns and Undershoots for Runway Safety Areas. Washington, DC: The National Academies Press. doi: 10.17226/14137.
×
Page 9
Page 10
Suggested Citation:"Chapter 2 - Research Approach." National Academies of Sciences, Engineering, and Medicine. 2008. Analysis of Aircraft Overruns and Undershoots for Runway Safety Areas. Washington, DC: The National Academies Press. doi: 10.17226/14137.
×
Page 10
Page 11
Suggested Citation:"Chapter 2 - Research Approach." National Academies of Sciences, Engineering, and Medicine. 2008. Analysis of Aircraft Overruns and Undershoots for Runway Safety Areas. Washington, DC: The National Academies Press. doi: 10.17226/14137.
×
Page 11
Page 12
Suggested Citation:"Chapter 2 - Research Approach." National Academies of Sciences, Engineering, and Medicine. 2008. Analysis of Aircraft Overruns and Undershoots for Runway Safety Areas. Washington, DC: The National Academies Press. doi: 10.17226/14137.
×
Page 12
Page 13
Suggested Citation:"Chapter 2 - Research Approach." National Academies of Sciences, Engineering, and Medicine. 2008. Analysis of Aircraft Overruns and Undershoots for Runway Safety Areas. Washington, DC: The National Academies Press. doi: 10.17226/14137.
×
Page 13
Page 14
Suggested Citation:"Chapter 2 - Research Approach." National Academies of Sciences, Engineering, and Medicine. 2008. Analysis of Aircraft Overruns and Undershoots for Runway Safety Areas. Washington, DC: The National Academies Press. doi: 10.17226/14137.
×
Page 14
Page 15
Suggested Citation:"Chapter 2 - Research Approach." National Academies of Sciences, Engineering, and Medicine. 2008. Analysis of Aircraft Overruns and Undershoots for Runway Safety Areas. Washington, DC: The National Academies Press. doi: 10.17226/14137.
×
Page 15
Page 16
Suggested Citation:"Chapter 2 - Research Approach." National Academies of Sciences, Engineering, and Medicine. 2008. Analysis of Aircraft Overruns and Undershoots for Runway Safety Areas. Washington, DC: The National Academies Press. doi: 10.17226/14137.
×
Page 16
Page 17
Suggested Citation:"Chapter 2 - Research Approach." National Academies of Sciences, Engineering, and Medicine. 2008. Analysis of Aircraft Overruns and Undershoots for Runway Safety Areas. Washington, DC: The National Academies Press. doi: 10.17226/14137.
×
Page 17
Page 18
Suggested Citation:"Chapter 2 - Research Approach." National Academies of Sciences, Engineering, and Medicine. 2008. Analysis of Aircraft Overruns and Undershoots for Runway Safety Areas. Washington, DC: The National Academies Press. doi: 10.17226/14137.
×
Page 18
Page 19
Suggested Citation:"Chapter 2 - Research Approach." National Academies of Sciences, Engineering, and Medicine. 2008. Analysis of Aircraft Overruns and Undershoots for Runway Safety Areas. Washington, DC: The National Academies Press. doi: 10.17226/14137.
×
Page 19

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

4The work for this study was planned and structured as follows: 1. Conduct a literature review and a functional hazard analy- sis (FHA) to improve the research team’s understanding of factors causing or contributing to aircraft overrun and undershoot accidents, as well as to identify existing ap- proaches, procedures and sources of data to support the development of risk models. 2. Collect historical accident and incident data from the sources identified and selected, and develop a compre- hensive database of relevant accidents and incidents that included the causal factors, contributing factors, and operation conditions. 3. Collect historical NOD to support the development of risk models. 4. Transform the data to enable comparisons, thus increasing the pool of available information to develop risk models. 5. Develop three sets of risk models for LDOR, LDUS, and TOOR. Each set included: a frequency model for assessing the likelihood of the event, a location model to estimate the probability the aircraft wreckage is located beyond a given distance from the runway, and a consequence model that integrates the previous frequency and location model to evaluate the probability of severe consequences. 6. Develop of a probabilistic approach for the analysis of RSAs that incorporates the models developed in this study. 7. Incorporate the approach and models in prototype soft- ware to demonstrate the feasibility of the analysis approach developed. The research plan followed the diagram depicted in Figure 1. The research team conducted a literature review followed by an FHA to identify important parameters associated with overrun and undershoot events, and collected the necessary information to potentially use as independent variables in the risk models. After those parameters were identified, the research team screened the existing incident and accident databases to locate the events considered relevant for this study. For each event, available reports and docket documents were obtained and were analyzed in search for the relevant information in- cluded in the ACRP 4-01 database. Data was collected and a database was created to save this information in an organized manner. When possible, gaps observed for important parameters were obtained from sources other than accident investigation agencies to com- plement the missing information. A statistical summary of the database was developed and is presented in this report. Statistical tools and software were used to develop risk mod- els for frequency and location for each type of accident. These models incorporate historical flight and weather conditions to evaluate the level of risk exposure for a specific runway. A rational probabilistic approach was developed to integrate these models and to assess the probability of severe consequences for these accidents. Finally, these models were integrated in proto- type software to facilitate the analysis. Literature Review Risk assessments are utilized in many areas of aviation, from designing aircraft systems to establishing operational standards and air traffic control rules. However, there is little information available for assessing the risk of accidents occurring near and at airports. Previous relevant studies for airports can be broadly categorized into four areas: airport design, third-party risk, facility risk, and operational risk. To assess risk from an airport design standpoint, the U.K. Civil Aviation Authority (CAA) Safety Regulation Group conducted a study on aircraft overrun risk, which guides air- ports on overrun risk assessment and provides advice on how to reduce it (CAA 1998). Another study under this category is AEA Technology’s risk assessment of airfield design rules (Eddowes et al. 2001). In this study, the authors reviewed design standards such as run- way length and reference codes, the runway end safety area (RESA), separation distances between runways and taxiways, C H A P T E R 2 Research Approach

and obstacle limitation surfaces. It made concrete recommen- dations for amending the International Civil Aviation Orga- nization ( ICAO) Annex 14 safety areas to achieve a specific target level of safety. In the United States, studies also have been carried out to set criteria for the design of airport safety areas, particularly in California. Garbell (1988) pioneered the accident-potential concept that led to the adoption of safety areas at a number of airports. A 1990 FAA study (David, 1990) compiled data regarding the location of commercial aircraft accidents rela- tive to the runway involved. The database was used to validate the RSA dimensions adopted by the FAA, and it is still effec- tive today. There are only a limited number of general methodologies and models for assessing an airport’s third party risk (Piers, 1996). They are derived principally from studies commis- sioned by the Dutch and British governments and their results are broadly similar (Ale and Piers, 2000). A third family of studies seeks to assess the risk that aircraft operations pose to specific developments near airports. Examples of such studies include one for the U.S. Nuclear Regulatory Commission dealing with the safety of nuclear power plants, as well as a study for the Department of Energy for assessing the risk of an aircraft crash into its nuclear weapons and material storage facilities (Eisenhut, 1973; NRC, 1981). A study on Salt Lake City International Airport inves- tigated the crash probability at a hospital, a school, and a shopping mall nearby (Kimura et al., 1995). The final group of risk assessment studies concentrates on flight operational safety and is not strictly considered airport risk assessment. However, certain elements of these studies are very relevant to airport risk analysis. For example, a study on navigational aids established risk ratios for mostly airport fac- tors that influence the risk of approach and landing accidents (Enders et al., 1996). A related piece of research by the Flight Safety Foundation measured accident risk based on, among other things, airport conditions (Khatwa and Helmreich, 1998). The ICAO’s Collision Risk Model (CRM) calculates the collision probability of an operation with obstacles of known location and size during an Instrument Landing System (ILS) approach. The model is used as a decision-making tool for developing safe approach procedures and for airport planning (ICAO, 1980). One of the core reasons for oversimplification of accident frequency modeling is the lack of data on exposure to various risk factors in normal operations. Without NOD, crash rates related to the presence of risk factors cannot be established. Closing this gap in research is a major achievement of the work conducted by Loughborough University (Wong, 2007) and ACRP 4-01. Appendix A of this report provides information collected during the literature review on the procedures used and re- sources available to pilots during the landing and takeoff phases of the flight. Moreover, it describes how weather conditions, runway conditions, faults, and human errors can affect the operations and lead overruns and undershoots. Functional Hazard Analysis An FHA is a formal and systematic process for the identi- fication of hazards associated with an activity. The purpose of the FHA in the context of this study was to determine rele- vant causal factors of overrun and undershoot accidents and hazards to aircraft associated with airport operations (e.g., landing, takeoff roll, and associated fault sequences) and the physical design of airports. The risk analysis approach utilized in this study is based pri- marily on a review of operational experience, in particular, accident, incident, and normal operation data. The modeling approach adopted for the quantitative assessment of the risks 5 1 - Literature Review 2 - Functional Hazard Analysis 3 - Identification of Relevant Events and Parameters 4 - Accident/Incident Data Collection and Database Development 5 - Development of Approach for Risk Assessment of RSAs 6 - Development of Risk Models 7 - Development of Software for Risk Assessment Figure 1. Research plan.

associated with runway operations was based on the evalua- tion of: • The likelihood of the incident occurring; • The location where the aircraft came to stop, in case of overruns, or its point of first impact, for undershoots; and • The consequences of such an incident (injury and cost of damage). Overrun and undershoot incidents may be considered in terms of the deviation of the aircraft from its intended path. The definition of the deviation for each incident type is sum- marized as follows: • For overrun incidents, the “longitudinal deviation” is de- scribed by the longitudinal distance traveled beyond the accelerate/stop distance available (for takeoff events), and beyond the landing distance available (for landing events). • For undershoot incidents, the “longitudinal deviation” is described by the longitudinal distance the aircraft under- shoots the intended runway threshold. • For both overrun and undershoot events, the “lateral deviation” is the lateral distance to the extended runway centerline. Examples of incident and accident causal factors include human error, as well as incorrect approach speed; deviation of approach height relative to desirable flight path; improper touchdown location; inappropriate runway distance avail- ability, aircraft system faults, improper weight, and aircraft configuration; low friction runway surface conditions (wet, icy, or contaminated); adverse weather conditions, particu- larly tail wind, cross wind, gusting wind, low visibility, and precipitation; and unfavorable runway slopes. Results for FHA of aircraft overruns and undershoots are provided in Appendix B. Database Development A single database that contains a significant number of rel- evant accidents and incidents on and near airports was cre- ated for this study. A second database comprising normal operations data also was developed for this study. The data were organized to facilitate the assessment of each accident type in a coherent manner, rather than based on multiple databases with different inclusion criteria. Before data were collected, some criteria were established for filtering out events available in the database sources that would not be relevant for ACRP 4-01 model development. It is important to describe the criteria used and the reasons for applying them. Filter Applied to the Data Some filtering criteria were used on the data so that events were comparable, as well as to ensure the models developed would represent the objectives of this study. The first filter was an attempt to use information from only specific regions of the world having accident rates comparable to the U.S. rate. Figure 2 depicts accident rates by region of the airline 6 Western-built transport hull loss accidents, by airline domicile, 1994 through 2003* Rates per million departures 1 Insufficient fleet experience to generate reliable rate. * 2003 Preliminary Information United States and Canada 0.4 Latin America and Caribbean 2.4 Europe 0.7 China 0.5 Middle East 3.1 Africa 13.3 Asia 1.7 World 1.0 Oceania 0.0 (Excluding China) JAA - 0.6 Non JAA – 1.2 C.I.S. 1 Source: U.S. Department of Transportation, Federal Aviation Administration. Figure 2. Accident rates by region of the world.

domicile. It was assumed that the information from North America (United States and Canada), Western Europe (Joint Aviation Authorities [JAA] countries), Oceania, and a few selected countries in Asia would be relevant to this study and included in the database. Using this filter criterion, the main sources of data included the following: • FAA Accident/Incident Data System (AIDS); • FAA/National Aeronautics and Space Administration (NASA) Aviation Safety Reporting System (ASRS); • NTSB Accident Database & Synopses; • Transportation Safety Board of Canada; • ICAO Accident/Incident Data Reporting (ADREP) system; • Australian Transport Safety Bureau (ATSB); • France Bureau d’Enquêtes et d’Analyses pour la Sécurité de l’Aviation Civile (BEA) ; • UK Air Accidents Investigation Branch (AAIB); • New Zealand Transport Accident Investigation Commis- sion (TAIC); • Air Accident Investigation Bureau of Singapore; • Ireland Air Accident Investigation Unit (AAIU); and • Spain Comisión de Investigación de Accidentes e Inci- dentes de Aviación Civil (CIAIAC). A more detailed description of each data source is available in Appendix C of this report. In addition, the filtering crite- ria and justification described in Table 1 were applied to the ACRP 4-01 database. Data Limitations There are many quantitative and qualitative limitations to reliable accident and incident data, and these limitations in- variably constrain the depth, breadth, and quality of airport risk assessments (Piers et al., 1993; DfT, 1997; Roelen et al., 2000). This study is no exception. The scope and detail of the analysis are restricted by the availability and quality of the data extracted from available sources. Major data limitations found during the collection process are outlined in the fol- lowing material. Missing Data. Accident investigation records and incident reports consist of a number of standard forms and reports. 7 Filter # Description Justification 1 Remove non-fixed wing aircraft entries. Study is concerned with fixed wing aircraft accidents and incidents only. 2 Remove entries for airplanes with certified max gross weight < 6,000 lbs (<12500 lbs if Part 91). Cut off criteria for lighter aircraft utilized to develop model for overruns on unpaved areas. 3 Remove entries with unwanted Federal Aviation Regulation (FAR) parts. Kept Part 121, 125, 129, 135 and selected Part 91 operations. Some FAR parts have significantly different safety regulations (e.g., pilot qualifications). The following cases were removed: Part 91F: Special Flt Ops. Part 103: Ultralight Part 105: Parachute Jumping Part 133: Rotorcraft Ext. Load Part 137: Agricultural Part 141: Pilot Schools Armed Forces 4 Remove occurrences for unwanted phases of flight. Keep only undershoots and runway excursions beyond the departure end of the runway. We are also keeping veer-off occurrences that were available in the Loughborough University database, but these will not be utilized for developing the risk models. 5 Remove all single engine aircraft and all piston engine aircraft entries. Piston engine aircraft are now used less frequently in civil aviation and therefore have been removed, to increase the validity of the modeling. Moreover, single and piston engine aircraft behave differently in accidents due to the lower energy levels involved and the fact that the major focus of this study is air carrier aircraft. 6 Remove all accidents and incidents when the point of first impact and the wreckage final location is beyond 2000ft from threshold. It would be unfeasible to have an RSA with more than 2000ft beyond the threshold, the gain in safety may not be significant and a previous FAA study used this criterion (David 1990). Table 1. Filtering criteria for accidents and incidents.

Even within these standard areas of interest, it is extremely rare that every field is complete. The NTSB docket files of minor accidents, when available, frequently contain less than a dozen pages of forms accompanied by only a brief synopsis of the occurrence. Information for incidents is very poor. As an example, prior to 1995, the narratives for AIDS reports were limited to 115 characters; reports since 1995 contain a more complete narrative prepared by the investigating inspector. The accident wreckage site often is given only a very crude description without supporting maps or diagrams. Only a small proportion of data fields are systematically recorded for every accident. The amount of missing fields in the database is high, restricting the number of parameters that could be analyzed with confidence. In many cases the report descrip- tions used were coupled with the runway satellite picture obtained from Google Earth to determine the approximate location of the accident and of the wreckage. The reports contain mostly information the accident inves- tigators deemed relevant to an accident’s occurrence. Outside of this judgment, few potential risk factors and measurements are included. This was a major obstacle to developing a data- base that consistently and systematically records a compre- hensive set of risk exposure parameters so they could be included in the models. The data available for analysis and model-building ultimately depended on the agency accident investigation mentality and policies. There are no alternative sources of data, and this issue is particularly critical concerning unconventional or latent risk factors beyond the well estab- lished sources of risk. Parameters such as weight and runway criticality that would require additional calculation often are impossible to compute because of unavailable data. When considering aircraft overruns, there are flights that used more than the nominal runway distance required (take- off or landing) to complete the operation but without departing the runway due to excess runway length available. These cases usually are considered as normal operations and will never feature in accident records. Although these cases could provide additional and valu- able data to model incident location distribution, obtaining normal operations data on actual runway distance used proved difficult to obtain despite extensive efforts from the research team. Finally, the presence of excess runway may alter pilot behavior such that more runway distance is used than other- wise. In a number of occurrences, the pilot did not apply braking to stop the aircraft as soon as possible because the pilot elected to take a specific taxiway exit or was hurried by the traffic control to quickly leave the runway. However, it should be noted that these data limitations are not unique to the current research but are inherent to risk assessment studies that use historical accident data (Piers, 1994; ETSC, 1999). Poor Data Quality. Previous studies using data from accident reports and docket files have reported on the poor quality of data available (Hagy and Marthinsen, 1987). Erro- neous or conflicting information within the same docket is not uncommon. Some cases were identified where the pro- vided wreckage location diagram does not match the text description given. Confusing and inconsistent use of terms and nomenclature adds to the challenges of extracting precise data points. When faced with conflicting data, the research team applied judgment to obtain a final figure according to the best information available. Measurement Difficulties. The measurement of certain parameters suffers from inherent ambiguity in the aviation industry. A prime example is runway condition. There sim- ply has not been an agreed industry standard on reporting runway conditions and determining its relationship with runway friction and aircraft braking performance (DeGroh 2006; FAA, 2006b). The current industry approach is to measure and report runway friction periodically using stand- ard equipment and wet surface conditions. However, it also is common practice to rely on pilots’ subjective reporting, particularly for contaminated runways. Runway surface con- ditions may change rapidly according to precipitation, tem- perature, usage and runway treatment so actual conditions may differ significantly from those reported (FAA, 2006b). Icing conditions, too, also are known to be difficult to deter- mine even though they have an important impact on aircraft performance (Winn, 2006). The weather measured from ground stations may vary sig- nificantly from that experienced by the accident flight (Jerris et al., 1963), particularly if the weather station is located far from the accident location, although this is common only with very remote airports. Another difficulty lies in the dynamic nature of meteorological conditions. Wind strength and direction may change constantly during the course of an approach. It may not always be clear which reading is most relevant. Some judgment was necessary to enter the most appropriate reading into the database. Limited Data for Incidents. The importance of including data from incidents cannot be overemphasized. By excluding incident data, the project would not take into account poten- tially serious occurrences. However, a practical difficulty of incorporating incident data is the lack of it. The quantity and quality of incident data is in even greater doubt than for accidents. 8

Most agencies provide information and reports for accidents and serious accidents only. Many countries have procedures to obtain information on incidents but, except for the United States, these reports are not readily available from Internet sources. For this study, the basic sources of data for nonserious incidents were the FAA AIDS and the NASA/FAA ASRS. One additional difficulty to incorporate the information into this project is the number of incidents reported. Some incidents are not reported because there were no conse- quences. To overcome this obstacle, a study was performed on the distribution of available data to assess the number of unreported incidents and to consider these cases when devel- oping the frequency and location models. A number of miss- ing incidents was assumed, as described in Appendix D, and a weighting factor was applied in the statistical analysis to develop the models. Additional Issues The database used for developing the final risk models in- cludes only those events that may challenge the RSA beyond the runway ends. The criterion utilized is similar to that used by the FAA (David, 1990) and includes those occurrences whereby the point of first impact or the final wreckage location is within 2000 ft from the threshold. Using such criteria, 459 accidents and incidents were selected to compose the informa- tion that was used for developing the risk models. Table 2 sum- marizes the number and type of events by source of data. The main reason for the criterion applied is it would not be feasible to modify an existing RSA to more than 2000 ft in length, compared to the current 1000-ft standard. Most im- portantly, the additional safety benefit for having an RSA longer than 2000 ft certainly would be very small and not jus- tify the costs required for such improvements. Cases when the aircraft veered off the runway but did not challenge the area beyond the runway threshold also were removed, as these were out of the scope of this research. Data for events investigated by the NTSB were gathered from both investigation reports and the related dockets avail- able at the NTSB library in Washington, D.C. Before data were gathered, database rules were developed to assure uni- formity for the information obtained by different researchers contributing to this project. Incident information was collected from NTSB, FAA AIDS and FAA/NASA ASRS databases. Accident data were obtained from NTSB and from aviation investigation agencies from other countries. A significant amount of aviation safety information is available worldwide, in many cases from specific websites. One of the main problems with this, however, is the frag- mentation of the information. Each agency has different search engines, and data are presented in different formats. In most cases, identifying the relevant events fulfilling the crite- ria for this project was quite challenging. Appendix E of this report presents the list of relevant acci- dents and incidents that fulfilled the criteria and filters estab- lished for the study and were utilized for developing the risk models. Supplementary Sources of Information Individual accident reports were evaluated to extract infor- mation. In addition, part of the data was complemented from other sources of information, particularly for aircraft, airport, and meteorological conditions. Based on the aircraft registra- tion, we have gathered information for aircraft involved in accidents and incidents from the following websites: • FAA REGISTRY N-Number Inquiry: http://registry.faa.gov/aircraftinquiry/NNum_inquiry.asp • US/World – Landings.com: http://www.landings.com/evird.acgi$pass*90705575!_h- www.landings.com/_landings/pages/search.html • Airframes.org - Passenger airliners, cargo airplanes, business jets, private aircraft, civil and military. http://www.airframes.org/ • Civil Aircraft OnLine Registers, Official Civil Aircraft Registers: http://www.airlinecodes.co.uk/reglinks.asp?type=Official Airport information, when not included in the incident or accident investigation reports, was obtained from other sources. Basically the following web sources were utilized in this study: • United States: AirNav provides detailed aeronautical infor- mation on airports and other information to assist pilots in gathering information for flight planning. Airport details include airport location, runway information, radio navi- gation aids, declared distances, and other information for pilots. http://www.airnav.com 9 Database Source LDOR LDUS TOOR FAA AIDS (incidents) 14 29 12 FAA/NASA ASRS (incidents) 79 11 9 NTSB (accidents & incidents) 113 51 56 TSB Canada (accidents) 23 1 5 AAIB UK (accidents) 24 0 5 BEA France (accidents) 3 1 3 Other (accidents) 18 0 2 Total 274 93 92 459 Table 2. Summary of events utilized in this study.

• World: The World Aeronautical Database contains de- tailed, aeronautical information on nearly 10,000 airports and more than 11,000 Navigational Aids (NAVAID) worldwide. http://worldaerodata.com/ Many incident reports do not contain weather informa- tion, particularly when it is not deemed to be an important factor in the incident and was obtained from other sources. Weather for normal operations data also has been obtained from other sources, particularly from the National Oceanic & Atmospheric Administration (NOAA) database. NOAA is a federal agency focused on the condition of the oceans and the atmosphere. In many cases, particularly for accidents that occurred out- side North America, search engines available in the websites of accident investigation agencies are not very effective to fil- ter out those events that were irrelevant to this project. Some of the events were identified using these databases and, for a few cases, some accident data has been gathered from inde- pendent accident information websites. Two of the most used during this study included: • ASN Aviation Safety Database: The Aviation Safety Net- work is a private, independent initiative founded in 1996. It covers accidents and safety issues with regards to airliners, military transport planes, and corporate jets, and contained descriptions of more than 10,700 incidents, hijackings, and accidents. Most of the information are from official sources (civil aviation authorities and safety boards), including aircraft production lists, ICAO ADREPs, and country’s accident investigation boards. • World Aircraft Accident Summary: The World Aircraft Accident Summary (WAAS), produced on behalf of the British Civil Aviation Authority by Airclaims Limited, pro- vides brief details of all known major operational accidents worldwide. A typical example of this complementary information was the calculation of wind speed. Since the NTSB database con- tains wind speed and direction but not headwind and cross- wind components, determination of the orientation of the runway used by the accident aircraft allowed the research team to derive the headwind and crosswind components. Wreckage location often is described in words and required translation and interpretation to obtain estimates of location coordinates relative to runway centerline and thresholds. Accident/Incident Database Organization The accident and incident database was organized in Microsoft Access. The system provides some software tools that facilitate the use of the database in a flexible manner. The software includes facilities to add, modify, or delete data, make queries about the data stored, and produce reports summarizing selected contents. The database includes, for each individual event or opera- tion, the reporting agency, the aircraft characteristics, the runway and environmental conditions, result of the opera- tion (accident or incident), and other relevant information such as consequences (fatalities, accident costs) and causal or contributing factors and parameters required to develop the risk models. A unique identifier was assigned to each event, and the descriptions of each field and the database rules are available in Appendix F. The final database includes the cat- egories and fields listed in Table 3. Neither the NTSB nor the FAA routinely compiles data in this manner. Both agencies investigate accidents for aero- nautical purposes to determine ways to improve the design and operation of aircraft and airports and to foster better pilot skills and techniques. If land use factors are examined at all, it is incidental to the primary purpose of the investigation. As previously noted, it was difficult to gather information on incidents because they are rarely investigated to a level that could provide useful information for this study. Also, there are often few consequences associated with incidents. Normal Operations Data Another key approach in this study is the use of normal operations (nonaccident/nonincident flight) data for risk modeling. Various studies already have identified the lack of NOD as a major obstacle to the development of quantitative risk models (Department of Transport, 1979; Piers et al., 1993; Khatwa et al., 1996; Khatwa and Helmreich, 1998; Eddowes et al., 2001; Li et al., 2001). The approach and the data utilized in this project were developed by Wong (2007). In the absence of information on risk exposure, even though the occurrence of a factor (e.g., contaminated run- way) could be identified as a contributor to many accidents, it is impossible to know how critical the factor is since many other flights also may have experienced the factor without in- cident. With NOD, the number of operations that experience the factor benignly, singly, and in combination can be calcu- lated, risk ratios can be generated, and the importance of risk factors quantified. This assessment may allow the prioritiza- tion of resource allocation for safety improvement (Enders et al., 1996). A large and representative sample of disaggregate U.S. NOD covering a range of risk factors has been collected, al- lowing their criticality to be quantified. The basic idea was to use these data and the information on U.S. incidents and accidents as a sample to develop the frequency models only, simply because the NOD represents only events occurring in the United States. The larger dataset comprising both U.S. 10

11 Category Field Level 1 Field Level 2 Accident ID Event ID Accident Class Event Type Researcher Source Country StateLocation City Date Time Basic Info Basic Notes Make Model Series Serial Number Age No. of hours or Years No. of Engines Engine Type Turboprop, Turbofan (Low or High) or Turbojet Max Certified Landing Max Certified Takeoff Max Gross Weight Registration Number Regulations Reference ACFT Regulator Owner Aircraft Data Operator Code IATA Code Latitude Longitude Runway Number Landing Distance Available Takeoff Distance Available Landing Elevation Landing Latitude Landing Longitude Takeoff Elevation Takeoff Latitude Takeoff Longitude Runway condition Runway Grooved Yes/No ARFF Availability A to F Control Tower Yes/No Temporary Construction Works Yes/No Runway Width Runway Slope Surface Material Paved Overrun Length Airport Data Notes Aircraft Damage Destroyed, Substantial, Minor or None Change of Terrain Yes/No Consequence Area No. of Passenger Seats Total No. Of Seats Difficulty in Getting to Wreckage Yes/No Consequences Detailed Consequence Area Table 3. Database structure. (continued on next page)

12 Category Field Level 1 Field Level 2 Aircraft Collision Status Active/Passive/NA Visibility Min. Violation Yes/No Approach Min. Violation Yes/No Approach Category Required Visual/Non-Precision/ILS Cat1, 2 or 3 Approach Category Used Other Aircraft Involved Yes/No Crash Controllability Fully/Partially/No Glide slope Captured Yes/No Go Around Yes/No GPWS Yes/No GPWS type 1st or 2nd Generation Localizer Captured Yes/No Runway Change Yes/No Stabilized Approach Yes/No Takeoff Aborted Yes/No Detailed Info Takeoff Aborted Speed Actual Weight at Crash Was Weight Estimated Yes/No Max Weight for Operation Destination Country Departure Country Diverted Flight Yes/No ELT Fitted and Operational Yes/No Flight Delayed Yes/No Flight Duration Fuel Load Load Factor Operation Type Scheduled Yes/No Landing Distance Required Takeoff Distance Required Takeoff Weight Takeoff Fuel Load Flight Data Weight restriction Violated Yes/No Obstacle Depth Obstacle Height Obstacle Width Obstacle Location X, Y and Z Hit Obstacles Notes Terrain Depth Terrain Height Terrain Width Terrain Location X, Y and Z Hit Terrain Notes No. Passenger Injuries Fatal, Serious, Minor, None No. Flight Crew Injuries Fatal, Serious, Minor, None No. Cabin Crew Injuries Fatal, Serious, Minor, None No. Ground Crew Injuries Fatal, Serious, Minor On Ground Injuries Fatal, Serious, Minor Public Injuries Fatal, Serious, Minor Injuries Total Injuries Event Highest Injuries Notes Table 3. (Continued).

13 Table 3. (Continued). Category Field Level 1 Field Level 2 Ceiling Dew Point Electric Storm Yes/No Fog Yes/No Frozen Precipitation Yes/No Wind Direction Wind Velocity Wind Shear Yes/No Gusts Icing Condition Yes/No Light Level Dawn/Day/Dusk/Night Rain Heavy/Moderate/Light/None Snow Yes/No Temperature Visibility RVR Actual Weather Different than Reported Yes/No Weather General Local Variation Yes/No Tailwind Weather Crosswind Explosion Fire No. Obstacles Hit Runway Exit Speed Total Wreckage Path Length Pilot Actively Avoided POFI Angle POFI Velocity POFI Location X,Y and Z Wreckage Location Longitude and Latitude Wreckage Location X,Y and Z Runway Exit X Runway Touchdown X Touchdown Speed Wreckage Site Elevation Height Above Threshold Approach Speed Wreckage Path Length on Each Terrain Up to 4 segments Wreckage Slope Up to 4 segments Wreckage Info Wreckage Surface Up to 4 segments Power Brake (wheel brakes, spoilers or reversers) Hydraulic Tire Aircraft System Fault Other Low Visibility Rain Wind Shear Tailwind Crosswind Gusts Low Ceiling Strong Wind Anomalies Weather Conditions Turbulence Freezing Rain Other (continued on next page)

and world occurrences was utilized to develop the location models. Incorporating risk exposure information into the accident frequency model enhances its predictive power and provides the basis for formulating more risk-sensitive and responsive RSA policies. Accident frequency models need no longer rely on simple crash rates based on just aircraft, engine, or opera- tion type. As discussed below, factors previously ignored by airport risk assessments and RSA regulations are accounted for using the models developed in this study. Moreover, this normal operations database can be used for future studies. The detailed source and sampling strategy of the NOD database is described in Appendix G. In addition, a small sam- ple of the NOD being utilized in this study is included in that section. A list of sampled airports is shown in Appendix H and the stratified sampling strata is presented in Appendix I. To derive the weights to be applied to each stratum, it was necessary to identify the relevant traffic from Terminal Area Forecasts (TAFs). Details on the calculation of TAF are pre- sented in Appendix J. Normalization of Data The small pool of relevant data available is a fundamental problem to risk assessment in aviation (Caves and Gosling, 1999). Most studies have used data from different airports to develop risk models. However, operation conditions and levels of risk are different at different airports. In addition, only raw distances between the final wreckage location and the runway end have been used to develop current FAA RSA recommendations. To mitigate this difficulty, information available for differ- ent airports was compared by using a normalization procedure, to transform existing data to a standard nominal airport (Kirk- land et al., 2003). To normalize aircraft accident data, the “nor- mal” airport is an airport situated at the International Standard Atmosphere (ISA) conditions, with level surrounding terrain or obstacles and an infinitely long, hard runway. Normalization was conducted for the effects of terrain on wreckage location using the models developed by Kirkland et al. (2003) and the effects of the local atmospheric conditions on the aircraft’s performance, based on standard corrections for aircraft distance required. Major factors that affect the runway distance required for the operation and used in the flight man- ual calculations are runway slope, runway elevation, and the air temperature; however, a correction was not applied to the slope in the RSA due to missing information for the majority of the events. The RSA slope is indeed an important factor on the wreckage distance and this information should be collected and made available in incident and accident investigation reports. Normalization procedures used in this study are presented in Appendix K. 14 Category Field Level 1 Field Level 2 Incorrect Flight Planning Communication/Coordination Visual Illusion Fatigue Pressonitis Human Errors Other Wet Contaminated - Standing water Contaminated - Rubber Contaminated - Oil Contaminated - Ice Contaminated - Slush Contaminated - Snow Contaminated - Paint Contaminated - Other Construction Runway Surface Conditions Down Slope Wildlife Hazards Unstabilized - Low Approach Unstabilized - Low Speed Long Touchdown Unstabilized - High Speed High Above Threshold Takeoff Rejected Approach/Takeoff Procedures Other Aircraft Body Type Wide or Narrow Aircraft Cost 2007 dollar value Human Cost 2007 dollar value Cost Total Event Cost 2007 dollar value Table 3. (Continued).

Ideally, normalization procedures should consider the dis- tance required relative to the runway distance available dur- ing the operation. However, the attempt to incorporate this factor proved difficult due to a lack of available information to compute the distances required, for both the accident data and NOD. In addition, some factors cannot be normalized, such as the pilot’s skill or differences in safety records between coun- tries. Despite these difficulties the technique utilized in this study creates a larger pool of relevant and comparable data for sound model building. Results from models built with normalized accident and normal operations data then can be applied to specific airports through denormalization. Development of Risk Models In this study overrun and undershoot risk models were de- veloped to allow the analysis of RSAs in the light of specific factors related to existing operation conditions. The basic concept was to model the probability distribution for wreck- age location in the proximity of the runway threshold. The concept is illustrated in Figure 3 for overruns and in Figure 4 for undershoots. The illustration for overrun incidents shows a typical prob- ability distribution for the stopping location during a landing or rejected takeoff operation. (The illustration is not to scale and is only intended to help understanding the concept.) The great majority of aircraft will stop within the runway bound- aries, represented by the lightly shaded area of the probabil- ity distribution. However, in a few cases, the aircraft may not be able to stop before the runway end and will stop on the RSA or even beyond. This probability of overrunning the runway is represented by the dark shaded area. It would be best to model the whole probability distribution for the aircraft stopping location, but this information is not avail- able for NOD, and an alternative modeling approach was re- quired in a two-step process: evaluating the probability an aircraft will in fact overrun the runway, and modeling the likelihood the aircraft will stop beyond any given distance from the runway threshold. The same concept was used for the point of first impact (POFI) during landing undershoots. Figure 4 depicts the probability distribution for the touchdown location. Again, for the great majority of operations, the aircraft will touch- down within the runway boundaries, represented by the lightly shaded area of the probability distribution. The dark 15 RSA x y Stop Location Probability Distribution P {Location > x}= e–axm Figure 3. General concept for modeling aircraft overruns. RSA x y Touchdown Location Probabilit y Distribution P {Location > x }= e–axm Figure 4. General concept for modeling aircraft undershoots.

shaded area represents the probability the aircraft will land before the approach end of the runway. In addition, it was necessary to model the probability distribution for the aircraft wreckage location relative to the extended runway centerline. The concept is represented in Figure 5. Using the “x” and “y” models combined will allow users to evaluate RSA off standard dimensions and overall configuration. The distribution only represents the location distribution for aircrafts overrunning or undershooting the runway. An initial analysis of the database and NOD was required to determine which risk factors were available to be built into the parametric models. The generic model and the wider insight gained from the database allowed an initial generic estimate of risk for a given situation to be made and identification of whether there were other significant risk factors that applied. The three-part modeling approach was built for each accident type, as represented in Figure 6. Before developing the final model structure, the available data were evaluated statistically, in order to develop the proper model structure and to ensure the parameters were compatible with model assumptions. Approach Elements Event Probability. The likelihood of an aircraft overrun or undershoot accident or incident depends on the operation conditions, including airport characteristics, weather condi- tions, and aircraft performance. The probability of an accident per movement—the accident rate—is determined from historical data on numbers of move- ments carried out at reference airports and the number of acci- dents that occurred during those movements. Loughborough University has pioneered the use of NOD to include a multitude of risk factors in the assessment of accident probability. Using this approach allowed a far more discriminating analysis than relying solely on the accident rate. As a result, conclusions on RSA risks will reflect better the actual conditions and circum- stances of specific airports. Accident Location. In reality, the probability of an acci- dent is not equal for all locations around the airport. The probability of an accident in the proximity of the runways is higher than at larger distances from the runway. This de- pendence is represented by the accident location model, which is the second main element of the current methodol- ogy. The accident location model is based on historical data. The distribution of accident locations relative to the runway will be modeled through statistical functions introduced by Eddowes et al. (2001). By combining the accident location model with the accident probability, the local probability of an accident can be calculated for each runway end. Accident Consequences. The consequences of an acci- dent are a function of the dimensions of the actual RSA, of the aircraft and impact parameters (such as aircraft size, quantity of on-board fuel, impact angle, etc.), and of the local type of terrain and obstacles. The size of the accident area is not equal for every airport or area within the airport. The influence of the aircraft and impact parameters and the type of terrain on the size of the consequence area, as well as the lethality and 16 y P {Location > y}= e–bxn Figure 5. General concept for modeling aircraft lateral deviation. Risk Operating conditions Operating conditions, terrain Location probability, obstacle location, size and type, aircraft type Three-Part Risk Model Event probability Location probability Accident consequences Figure 6. Modeling approach.

damage of the consequences, are defined in the consequences model, the third main element in the current methodology. For this purpose, lethality is defined as the actual probability of being killed within the consequence area. Aircraft damage is translated as the direct cost of property loss for an accident. Accident/Incident Probability Model To examine the accident propensity associated with differ- ent factors (e.g., environmental conditions), logistic regression was used to develop statistical models for accident/incident occurrence probability. A number of numerical techniques could be used to carry out the multivariate analysis, but logistic regression was preferred. First, the technique is suited to models with a dichotomous out- come (incident and nonincident) with multiple predictor vari- ables that include a mixture of continuous and categorical pa- rameters. Logistic regression also is appropriate for case-control studies because it allows the use of samples with different sam- pling fractions, depending on the outcome variable without giv- ing biased results. In this study, logistic regression allowed the sampling fractions of accident flights and of normal flights to be different. This property is not shared by most other types of re- gression analysis (Nagelkerke et al., 2005). Backward stepwise logistic regression was used to calibrate the three frequency models because of the predictive nature of the research. This technique is able to identify relationships missed by forward stepwise logistic regression (Hosmer and Lemeshow, 2000; Menard, 2001). Due to the more stringent data requirements of multivariate regression, cases with miss- ing data were replaced by their respective series means. To avoid the negative effects of multi colinearity on the model, correlations between independent variables were tested first to eliminate highly correlated variables, particu- larly if they do not significantly contribute to explaining the variation of the probability of an accident. The basic model structure selected for this study is in the following form: (1) where P{Accident_Occurrence} = the probability (0-100%) of an accident type occurring given certain operational conditions; Xi = independent variables (e.g., ceiling, visibility, crosswind, precipitation, aircraft type); and bi = regression coefficients. The use of NOD in the accident frequency model provided a major improvement in the modeling of accident occur- rence, as discussed previously. The analysis with NOD also P Accident Occurence eb b X b X b X { _ } = + + + + 1 1 0 1 1 2 2 3 3+... adds to the understanding of cause-result relationships of the two accident types. This constitutes a causal element in the risk models, so the modeling tool developed can be used to assess risk reduction strategies and estimate future risk levels, given trends in influential factors in an airport context. A previous Loughborough University study on overruns found that the model developed for landing overrun risk using NOD on excess landing distance available is 22 times more predictive than models based on flight type alone (Kirkland, 2001). Additional analyses using NOD have been conducted since, and they continue to show the importance of assessing the criticality of risk factors beyond the simple accident/ movement rate (Wong et al., 2005b, 2006). Accident/Incident Location Model The model structure selected for accident location was used by Eddowes et al. (2001) and is in the following form: (2) where P{Location > distance} = the probability the overrun/ undershoot distance along the runway centerline beyond the threshold is greater than x; x = a given location or distance be- yond the threshold; and a, n = regression coefficients. This dependence is represented by the accident location model, which is the second main element of the current methodology. The accident location model is based on his- torical data on accident locations. The distribution of acci- dent locations relative to the runway was modeled through statistical functions. By combining the accident location model with accident probability, the local probability of an accident can be calculated for each runway end. When plotting the percentage of accidents where the air- craft stopped beyond a certain distance from the threshold, in case of overruns, or first impacted the terrain, for under- shoots, the probability diminishes the greater the distance is, as depicted in Figure 7. The probability and location models will provide a quantita- tive assessment based on operating conditions for a specific airplane landing or taking off at a specific runway. In addition, it is necessary to relate these probabilities with the RSA condi- tions to provide an assessment of the probability that the con- sequences of an incident are severe. This is the final component of the approach, described in the following section. Consequence Model The consequences modeling approach should provide a qualitative assessment of the severity of an accident, based on P Location x e axn{ }> = − 17

the location model and the existing runway characteristics, to include dimensions of existing RSA, airplane weight, location and type of obstacles, and topography of surrounding terrain. The approach used in this project was to model the probability of severe consequences using the frequency and location mod- els, coupled with existing RSA configuration and obstacles. The consequences of an accident depend on several factors that are difficult to model, such as the energy of the crash (speed, aircraft weight, and size), quantity of fuel and occur- rence of fire after impact, type of obstacle (height, depth, ma- terial, size), impact angle, and the local type of terrain. Initial attempts to model consequences focused on the relationship between the raw or normalized distances and the severity of the accident, reflected by the amount of damage and cost of injuries. The overall consequences of the accidents were quan- tified by the total direct costs for injuries and aircraft damage. The approach proved difficult to implement because the re- lationship between accident location and consequences was poor. In many situations the consequences were related to the speed that the aircraft hit an obstacle and the type of the obsta- cle. Information for the former was not available for the great majority of cases. Therefore, the efforts to model consequences were directed to providing a rational approach that incorpo- rated the location model. The basic idea is simple but effective. The higher the speed, and hence the energy when an aircraft hits an obstacle, the greater the consequences. The sturdier the obstacle, the greater the consequences. The larger the obstacle, the greater the probability the aircraft will hit the obstacle. The speed of the aircraft striking the obstacle is related to the distance the aircraft would take to stop if no obstacles were present in the area adjacent to the runway ends. Based on the location model, the terrain type, and the deceleration model developed by Kirkland (2001), the probability the air- craft moving above a certain speed when hitting the obstacle can be estimated. The speed necessary to cause significant damage to the air- craft and potentially severe consequences should be judged based on the type of aircraft, type of terrain, and type of obsta- cle. For this study, only general recommendations are provided to assess the interaction between the aircraft and obstacles. Using some simple assumptions it is possible to evaluate the overall risk of severe consequence accidents. One of the difficulties posed to evaluate consequences of accidents was to integrate the number and severity of injuries with the property loss. It was not possible or practical to eval- uate indirect consequences such as lost revenue, lost work time, disruption of flight schedule, and negative customer re- action to accidents. In this study, estimates for direct costs of accidents are pro- vided as a means to integrate personal injury and property loss. Although it is estimated that indirect costs typically rep- resent four times the value of direct costs, only the latter will be used in this study. The parameters that were evaluated include the cost of the accidents, and number and type of injuries. The relationships between these consequence parameters and potential independent variables include the wreckage path length, the number of obstacles hit during the accident, the location of these obstacles, and land use type for the area beyond the existing RSA. Accident Costs The consequences of accidents, as documented in investi- gation reports, are described in terms of the number of injuries and the level of damage to the aircraft. Although third-party injuries also were accounted for in this study, property loss not related to aircraft damage was not evaluated for the lack of information. Injuries are classified according to ICAO criteria into four groups: none, minor, serious and fatal. The number of passengers and crew members for each level generally is available in the accident reports. Based on the total number of passengers/crew on board the fatality rate for each accident was calculated. Damage to aircraft also is described according to four classification groups: none, minor, substantial, and destroyed. In addition to the raw classification, direct cost of acci- dents was calculated based on the number and type of injuries, as well as the damage to the aircraft. The basic source for accident cost is the Guide for Economic Values for FAA Investment and Regulatory Decisions (GRA, 2004). The objective of this report is to present a set of cost items and quantify the specific values recommended that FAA use in future regulatory evaluations in the conduct of benefit-cost and other evaluations of investments, including certain Airport Improvement Program (AIP) grants, and regula- tions subject to FAA decision making. They also are used by 18 0% 20% 40% 60% 80% 100% 0 500 1000 1500 2000 Distance x from Threshold (ft) Pr ob {d ist an ce > x } Figure 7. Typical trend for wreckage location model.

others, including airports, in benefit-cost analysis of pro- posed investments. The basis for estimating the direct costs is presented in Appendix L. Development of Prototype Software for Risk Analysis As part of this research study, prototype software that in- corporates and integrates the risk models was developed. Users may input raw data and obtain normalized data, run a risk assessment analysis, and obtain denormalized results. The software serves as a tool for risk assessment associated with overrun and undershoot accidents and provides a basic yet useful format for risk analysis professionals to assist air- port operators in evaluating RSAs. Input data include the airport information, target level of safety (TLS), RSA characteristics including dimensions and type of terrain, and multiple historical or planned operations that may challenge the RSA in the event of overrun or under- shoot incident. Output includes frequency distribution of risks for each type of accident and the percentage of flights subject to risk above a TLS. 19

Next: Chapter 3 - Findings and Applications »
Analysis of Aircraft Overruns and Undershoots for Runway Safety Areas Get This Book
×
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

TRB's Airport Cooperative Research Program (ACRP) Report 3: Analysis of Aircraft Overruns and Undershoots for Runway Safety Areas explores overrun and undershoot accident and incident data conditions relating to these occurrences. The report also includes an assessment of risk in relation to the runway safety area and highlights a set of alternatives to the traditional runway safety area. The appendices to ACRP Report 3 are available online.

ACRP Report 50: Improved Models for Risk Assessment of Runway Safety Areas, which was released in July 2011, expands on the researc presented in ACRP Report 3. ACRP Report 50 analyses aircraft veer-offs, the use of declared distances, the implementation of the Engineered Material Arresting System, and the incorporation of a risk approach for consideration of obstacles in or in the vicinity of the runway safety area.

View the Impact on Practice related to this report.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!