Development of a Runway Veer-Off Location Distribution Risk Assessment and Reporting Template (2014)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

9 Availability of Data for Modeling Veer-Off Risk The ultimate objective of this project was to characterize the risk of aircraft veer-off within segments or subareas of the RSA, and areas outside of, but contiguous to the RSA. To achieve these goals, it was necessary to develop risk models based on information collected from accident and incident reports. One of the main concerns during the planning for this study was the availability of information required for modeling veer-off risk, particularly data related to the char- acterization of longitudinal and lateral distances relative to the runway. The analysis of data availability involved a few steps. Initially, it was necessary to develop a general modeling approach to identify what type of data would be required. Following, it was necessary to review accident and incident records screened as veer-offs to identify major gaps in data availability/usability. With the analysis, alternative sources of information were evaluated to close the gaps by either extracting information from these alternative sources, or using it to support inferences based on each report narrative, particularly to characterize the veer-off pathway. Even using this approach, approximately 50% of the records were not used for modeling lateral and longitudinal distances during the veer-off, both because the information was not presented in the report and the narrative did not allow inferences on the veer-off pathway to be made. It is recognized that these inferences may have some impact on the accuracy of the models developed; however, the study has resulted in the development of an analysis methodology available to the aviation industry that is a vast improvement over current status. The research has brought to light the identification of key information that is needed to develop the models and, moreover, to identify the deficiencies that exist in reporting key data. Collection of this data for future veer-offs enhances the accuracy of the models developed in this study. In addition, as part of this project, guidance is provided for improving reporting procedures. This chapter describes the major gaps in data availability and describes additional sources of information used to characterize or infer the veer-off path- way and other parameters. U.S. Databases Three federal databases currently contain information to various degrees on runway veer-offs that have occurred at airports within the United States. Each of these databases is discussed in this section. NTSB The NTSB is the federal agency charged with investigating aircraft accidents and determining probable and contributing causes. As such, the NTSB can do an onsite investigation of any aircraft accident or incident it chooses. In reality, budget and staffing limitations result in the NTSB generally choos- ing to do onsite investigations of accidents/incidents that they believe will have the most impact on improving aviation safety. If an aircraft veer-off results in an air carrier accident involv- ing substantial damage to the aircraft and/or fatalities/serious injuries to the occupants, the NTSB will normally send a team to the site and conduct a major investigation to determine the probable causes and contributing factors of the accident. An RSA that does not meet the FAA standards may be cited as a contributing factor to the accident. The final resting place of the aircraft relative to the runway will usually be documented, along with any of the non-standard aspects of the safety area that contributed to the accident. The location at which the aircraft left the runway and the path that aircraft traveled off the runway may or may not be documented. If an event involves an air carrier incident, the NTSB may or may not send an investigator to the site. For an event involv- ing a general aviation aircraft with fatalities, the NTSB will probably send an investigator; however, site visits would likely not occur for non-fatal general aviation accidents and general aviation incidents. C H A P T E R 4

10 The data collected onsite by the NTSB or received from other organizations, such as the FAA, is entered into the NTSB database. Although some incidents are reported in this data- base, most of the records available involve accidents and are used to support the NTSB’s determination of probable causes and contributing factors. FAA AIDS The FAA is also charged with investigating aircraft acci- dents in conjunction with the NTSB. However, the FAA does not have the authority to determine probable causes. The FAA will make site visits to any accidents that the NTSB vis- its; however, the FAA may also make site visits to many of the accidents and incidents the NTSB does not. In these cases, the NTSB will use the information gathered by the FAA in determining probable causes. The information gathered by the FAA is used to popu- late the AIDS database, which is primarily collected by FAA personnel having a pilot, airworthiness, or flight procedures background. Most of the information collected by them is related to these areas. The information may also be used by the NTSB to populate its database. Although the FAA is a larger organization and more geo- graphically dispersed than the NTSB, there are still a large number of accidents/incidents that do not result in an onsite visit from either organization. In these cases, the NTSB and FAA may conduct a desk investigation during which they gather information through telephone calls, e-mail, and written cor- respondence. This information may be provided by parties such as pilots, controllers, airport officials, state aviation agency employees, and local/state police investigators. Most of the incidents in the AIDS database do not contain explicit information on veer-offs that would be desirable for this study, particularly the veer-off path. In some cases, it is possible to infer some of the necessary information from the write-up (e.g., aircraft came to a stop about 50 feet from the runway just short of Taxiway Bravo). More explicit informa- tion on where the aircraft exited the runway, the aircraft veer- off path, and final resting place would be extremely helpful for the development of risk models. FAA/NASA ASRS NASA administers the ASRS, which is funded by the FAA. Although submitted reports primarily come from pilots and controllers, anyone can report an unsafe condition, incident, procedure, practice, or safety concern through the ASRS. The reporter is assured that he or she will remain anonymous. Also, pilots receive immunity if they report an inadvertent violation to the regulations. The intent of this reporting system is to identify deficiencies and discrepancies that can be corrected before they become accidents. The report narrative is made in a free flowing prose style. The information contained in each report will vary depending upon the reporter. The report represents the reporter’s perspective of what occurred. It is not indepen- dently validated by anyone, which would be very difficult to do so since the identifying information has been parsed from the report. International Databases International databases are obtained from the aviation acci- dent investigation bureau of the country where the accident occurred. The information is particularly comprehensive for major accidents and the availability is similar to that provided by NTSB when the full accident investigation report is avail- able. Data from international databases made up less than 10% of the veer-off records used in this study. Why Should Data Collection Be Improved? Over the years, the FAA has reviewed accidents and inci- dents involving undershoots, veer-offs, and overruns for com- mercial aircraft and found that approximately 90 percent of them come to rest within the prescribed safety area (FAA, 1990). However, the question has never been addressed as to how much risk is involved if a full safety area cannot be obtained for a runway. For example, if a safety area should extend to 250 feet on either side of the runway centerline but on one side, it is cost prohibitive to obtain more than 200 feet of RSA, what is the risk implication if the safety area is nar- rowed on one side? In this situation the 90 percent figure that the FAA cites addresses the entire safety area and here (in the case of the nar- rower safety area on one side) only one small section is being considered. The models and the analysis approach developed under this ACRP effort allow one to assess the risk for this situ- ation. However, the results and accuracy of the models would improve if additional data were included from actual veer-off events. At some airports, achieving the standard safety area may require the expenditure of large sums of money and may result in very little reduction in risk. Better data would allow one to more confidently assess the risk associated with the modified safety area to determine if the investment to meet the standard is worth the benefit in terms of improved level of safety. Potential Improvements to Veer-Off Reporting The intent of this study is not to identify deficiencies or to criticize the procedures used by agencies that manage civil aviation accident and incident databases. However, some

11 conclusions derived from this study have allowed the identifi- cation of potential improvements in data collection that may, in the future, enhance the accuracy of veer-off risk models developed in this research. Most of the improvements that may be achieved are associ- ated with including information to characterize the veer-off path during the runway excursion. The following paragraphs describe each of the three U.S. databases (NTSB, AIDS and ASRS) and the desirable improvements to reporting aircraft veer-off events. NTSB As mentioned earlier, characterization of the veer-off path is not available in many investigation reports from NTSB. Pictures and sketches showing the path would certainly help to characterize the veer-off path and support development of risk models. In most cases, when veer-off location informa- tion is available, it is descriptive and included in the section describing the wreckage of the accident. In other cases, an investigator may not visit the site and little information is collected. For such cases, the pilot or the airport operator may be able to help the NTSB investigator by providing a sketch, pictures, and/or a narrative describ- ing the veer-off path and touchdown location, if the veer-off occurred during landing. With the availability of accident dockets online for events occurring after 1996, sometimes it is possible to find addi- tional information on veer-off path in documents other than the investigation report, particularly for major accidents. Although comprehensive information on weather condi- tions is not available for many events involving non-fatal acci- dents and incidents, it is possible to retrieve METARs when location, date and time are reported. FAA AIDS Veer-off reporting in the AIDS database can be signifi- cantly improved. Although the reports are very objective, details on the veer-off path through the narrative are sel- dom provided. Local time of the incident isn’t available in many reports; however, this data is very important because it allows the weather conditions under which the incident took place to be identified through the associated METAR for the airport at the date and time of the event. Another important parameter that is rarely reported is the approximate touchdown location for landing veer-offs. This information is most likely obtained from the pilots of the air- craft involved in the incident. Moreover, few reports contain information on runway surface conditions, which may be used to estimate the distance required for the operation. NASA/FAA ASRS ASRS database reports have information only on the month and year that the event occurred. Moreover, the reports only contain the 6-hour interval when the veer-off took place and it may be difficult to relate weather conditions to the event, if not included in the report. Another difficulty with ASRS database is that the informa- tion on the runway used may not be available. In this case it is not possible to assess the runway distance available for the operation or to associate wind direction and wind compo- nents during the operation. The aircraft veer-off path during the runway excursion is not available for most of the reports; however, it is some- times possible to infer from the narrative, particularly if it provides information on aircraft speed and location refer- ences (e.g., aircraft departed runway 200 ft from Taxiway D intersection). What Data Needs to Be Collected? In this section, the data that ideally should be collected are identified. Two categories of data can be characterized accord- ing to the need and availability from alternative sources. The first category of information entails data that is essential for developing and improving risk-based approaches and risk models for aircraft veer-off events. The second category involves data that can be helpful to improve models; however, this type of data may be obtained from alternative sources, if necessary. For example, in some cases weather conditions may not be reported, but this information can be obtained from METARs if the airport, the date, and time of the event are reported. Key information with no alternative sources (e.g., the character- ization of the veer-off path) is considered essential data for reporting purposes. Some of this information is already col- lected as part of the accident investigation (e.g., runway iden- tification) but is not always explicitly included in the accident documentation. Knowing the runway designation allows one to use other databases to find other parameters, such as the runway length and wind components. A template for reporting veer-offs was created and is included in Appendix A. The purpose of the template is to identify key information for reporting veer-off events and describe the dis- tances that may be used to characterize the aircraft path during the runway excursion. Figures have been included in Appen- dix A to illustrate the measurements needed to document the travel path of the aircraft from where it leaves the runway to where it stops or reenters the runway. The measurement of the various distances should be as accurate as possible but by no means is the accuracy obtained by a surveyor expected. If engineering plans of the airport

12 layout are available, it may be possible to make the field mea- surement from a known point and then add or subtract the distance to that point found on the engineering drawing. For example, “aircraft left the runway pavement, 150 feet beyond the north edge of the Taxiway Sierra intersection with the runway.” The 150 feet measured in the field would be added to the distance from the beginning of the runway to the north edge of Taxiway Sierra that is measured on the engineering drawing. Although not required, a sketch of the veer-off path is always very helpful. The purpose is to require the minimum data to avoid, as much as possible, overwhelming the investigator or reporter with inputting information that may be optional. For all veer-offs, it is important from a risk analysis perspective to document the aircraft travel path from where it departs the runway to where it comes to a stop. The figures pre- sented in Appendix A depict the various types of veer-off travel paths. Essential Data • Aircraft model (e.g., B737-400). • Airport code (e.g., DVT). • Date and time of event. • Runway used (e.g., 27R). • Type of operation (landing or takeoff). • Runway surface condition: dry, wet, contaminated with water, ice, slush, snow, or other. • Begin roll location: – Longitudinal distance for begin roll (DBR): 77 Landing: approximate touchdown location mea- sured from the beginning of the runway. 77 Takeoff: only if different from the beginning of the runway (always measured from beginning of runway). – Lateral distance for beginning roll (LBR): distance from runway edge, only if touchdown occurred off the runway. • Veer-off path: – Longitudinal distance from the beginning of the runway where the first wheel of the aircraft departed the runway and runway side (DExit). – Longitudinal distance from the beginning of the run- way where maximum veer-off occurred (DMax). – Lateral distance from runway edge (not pavement edge) where the maximum veer-off occurred (LMax). – Longitudinal distance from the beginning of the run- way where aircraft stopped or reentered the runway after veering off (DStop). – Lateral distance from runway edge where aircraft stopped (LStop). Use zero if aircraft reentered runway. • Runway side: right or left relative to direction of operation. • Cause(s) of aircraft damage, if any. The distances defined above are applicable to the aircraft path during the veer-off only. If an aircraft crossed the RSA and entered a taxiway, the stop location should be assumed to be the point where the aircraft entered the taxiway before ini- tiating normal taxiing operation. All distances are measured to the center of the main gears of the aircraft. The template in Appendix A contains figures to illustrate the veer-off distances that will ideally be reported. These fig- ures depict four different veer-off scenarios that are shown as either a left or a right veer-off. Supporting Data In addition to the essential information described in the previous section, it is always beneficial to include the follow- ing data in the report for sake of accuracy: • Runway distance available for the operation. • Runway distance required for the operation. • Weather conditions (temperature, ceiling, visibility, wind speed and direction, gusts, type of precipitation), if any. • Was runway grooved? Yes or No. • RRSA conditions at time of the veer-off: dry, wet, snow, soft terrain, rough terrain. Implementation of Veer-Off Data Collection Procedures One of the main problems with attempting to collect accu- rate data for future runway veer-offs is that in many cases a representative from the NTSB or the FAA may never do an on-site investigation. This is especially true for veer-off inci- dents with minor consequences—in these cases, the RSA with its graded slopes and cleared areas have prevented the aircraft from incurring substantial damage and the aircraft occupants have not been seriously injured. From an accident-prevention perspective there is not much to be gained by sending some- one to conduct an on-site investigation, especially during times when budgets are limited. The reason for collecting additional information such as aircraft veer-off path is for the purpose of risk analysis. As mentioned earlier, the FAA is moving toward a risk-based decision-making processes; however, with the exception of funding, some ACRP studies like this one, the available tools for the industry are still very limited. The prime beneficiary of collecting information on future veer-offs to support risk assessments would be the airport operator who either cannot obtain the full safety area or per- haps needs to modify the current safety area for one reason or another. The airport operator generally has an employee that has access to the airfield after a veer-off occurs, either on duty or subject to call back. Consequently, the cost and time of

13 traveling to the airport that other parties would incur would be minimized if an airport employee takes the measurements and collects the information described in this report. A simple self-administered, computer-based training pro- gram to explain the data that needs to be collected for the various types of veer-offs and how it will be used may be the tool to implement data collection. Also, a standardized data collection form or the template shown in Appendix A could be used by airport operators. In establishing a program for this type of data collection, it would be essential to coordinate this data collection approach at the national level with NTSB, FAA and NASA, so the data may be saved in their respective accident and incident databases. The main question with this type of approach would be what happens to the data once it is collected. If NTSB and FAA were agreeable, it could be sent to them for incorpora- tion into their databases. ASRS information may be entered directly in the narrative of the incident by the reporter. Since NTSB’s database primarily consists of accidents, they may be reluctant to include so many incidents where the only entries would be information from the airport operator’s effort. The FAA would have to either modify the AIDS database to accommodate this data in a uniform fashion or establish a new database, which may represent a major challenge. Alternative Sources of Information The explicit data contained in accident and incident reports for veer-offs rarely provided all the information necessary to develop risk models. In many cases it was possible to infer some of the missing information from the report narratives. Often, information on weather conditions during the incident, if not included in the narrative, was obtained from historical MET- ARs for the specific location, date, and closest time of the event. In ASRS reports, the airport where the event occurred may not be reported and the specific day and time are never reported. If the airport was identified, the mean temperature during the specific month for the time period of the accident was used. The temperature is an important factor needed to adjust the required runway distance for the operation. Runway distances during an incident are seldom disclosed. Only the total runway length is normally available, even for NTSB reports. Historical satellite pictures from Google Earth can be used to measure the landing distance available (LDA) or the accelerate-stop distance available (ASDA). Although the information is available for existing conditions at websites like Airnav.com, runway extensions or reductions may have changed the available distances at the time of the incident. In some reports, the narrative describes airfield compo- nents and structures such as taxiways, hangars, ditches, mark- ers, etc. Identification of specific taxiways was obtained from airport diagrams available at airnav.com. Satellite images were used to identify the location distances associated with these structures. In addition to the official websites with access to accident and incident reports, alternative databases were used to help screen relevant records and to check information inferred from the report narratives. The main sources used were: • Aircraft Owners and Pilots Association (AOPA) Air Safety Institute Accident Database, and • MITRE Corp. Aviation Accident Database. In summary, other sources of information to complement missing data are presented in Table 1. Appendix B of this report presents a summary of veer-off records screened from the listed sources. Summary of Data Available/Usable Table 2 summarizes the data available and usable obtained for this research. The first and second columns list the data- base source and the number of veer-off events screened that are relevant to this study. The following columns contain the number of events that contained some level of information, or none, to characterize the veer-off path of the aircraft dur- ing the veer-off. Assumptions Made As indicated in Table 1, comprehensive information required to characterize the veer-off path is seldom explicitly available in accident and incident reports, except for major accidents when a full investigation report was prepared. To overcome this shortcoming, the research team made inferences and assumptions based on the narratives provided and used the additional data sources listed in previous sections. Although some accuracy was lost due to the assumptions made, the information developed in this study will assist the industry in understanding the mechanisms and the relationship between risk, available safety areas, and the presence of obstacles associ- ated with aircraft veer-off accidents and incidents. The approach will certainly improve knowledge of the rela- tionship between airfield design standards and the risk level involved when standards cannot be met. When necessary, some assumptions were adopted, particularly to estimate the path- way during the runway excursion. In many cases, it was nec- essary to use the narrative to infer the phase of the roll-out at which the veer-off occurred. When information required was not available in the report narrative, the procedures presented in Table 3 were used to obtain the data. In some cases, based on the aircraft speed reported and type of aircraft, the distance from the beginning of the

14 runway was inferred based on average acceleration and deceleration under the runway conditions reported. Other information and references available in the narrative, such as runway markers, taxiways, and other structures (e.g., ditches, hangars) were identified in satellite pictures to infer the distances. In most cases, the runway declared distances are not reported in the accident reports. Current web sources only provide exist- ing declared distances. Historical satellite images were used to measure the declared distance for the operation at the time of the event. Distance measurements when using Google Earth are quite accurate for the purpose of this study. Source Total Number of Events Complete Veer-Off Pathway Available Paral Veer-Off Pathway Available No Veer-Off Pathway Available Actual Esmates Actual or Esmate NTSB 283 77 94 66 46 AIDS 545 8 38 75 424 ASRS 243 0 142 14 87 AAIB 21 3 9 5 4 SACAA 5 0 2 0 3 ATSB 7 1 2 2 2 TSBC 18 9 5 3 1 BEA 8 8 0 0 0 TAIC 1 1 0 0 0 AAIBS 2 2 0 0 0 AAIU 4 2 2 0 0 CIAIAC 4 0 3 1 0 DSB 3 2 0 1 0 TOTAL 1144 113 297 167 567 Table 2. Summary of data available and usable. Source Type of Informaon Retrieved Google Earth www.googleearth.com Historical satellite pictures, measurement of distances, and terrain profiles Weather Underground www.wunderground.com Historical METARs to complement weather condions during the accident/incident Airnav airnav.com Informaon on runway slope, airport diagrams, airport idenfier for U.S. airports Landings www.landings.com Informaon on world’s airports FAA Aircra‚ Characteriscs Database www.faa.gov/airports/engineering/ aircra‚_char_database/ Aircra‚ data Eurocontrol Aircra‚ Performance Database V2.0 elearning.ians.lu/aircra‚performance Aircra‚ performance AOPA Air Safety Instute Accident Database hŒp://www.aopa.org/asf/ntsb/search_ntsb.cfm Screening NTSB records for smaller general aviaon (GA) aircra‚ MITRE Corporaon Accident Database Microso‚ Access database file Comprehensive informaon screened from U.S. accident records (NTSB and FAA AIDS) Table 1. Complementary sources of information for modeling.

15 Informaon Available Temperature Wind (Speed and Direcon) Airport, date and me Historical METAR informaon for me of event Historical METAR informaon for me of event Only date and me (no idenficaon of airport) No assumpon was made No assumpon was made Airport and date (no me) Average air temperature at the airport for the date Wind speed equal to zero if indicated that weather was not a factor Airport and month, with me range (ASRS reports) Average monthly temperature for the me range Assumed zero wind if weather was not a factor. In a few cases, the day of the month was idenfied based on specific weather condions reported in narrave (e.g., strong gusts, precipitaons, etc.) and me range reported Airport and month, but no informaon on runway used Average monthly temperature for the me range Assumed zero wind if weather was not a factor Table 3. Procedures to obtain missing data.