2
Measures of Aviation Safety and Sources of Data

This chapter discusses measures of aviation safety and provides an overview of the various aviation data sources and data that have been available before and since NASA’s NAOMS project was developed.

2.1
MEASURING AVIATION SAFETY

How should the safety of the aviation system be measured? The primary goal of most aviation safety initiatives is to reduce the rate of fatalities during or as a result of air travel. Thus, the most important metrics are the rates of fatal accidents per unit of air travel (number of passenger trips, flight legs, flight miles, and so on). For example, the Gore Commission recommended that “the principal focus should be on reducing the rate of accidents by a factor of five within a decade….”1 Annual reports published by many aviation safety organizations also focus on accident rates and their trends, measured in various ways, and many other studies on aviation safety emphasize fatality and fatal accident rates.2 For example, the FAA published the Aviation Safety Action Plan report entitled Zero Accidents: A Shared Responsibility, in February 1995.3 The FAA’s Safer Skies initiative (April 1998) called for an 80 percent reduction in the rate of fatal accidents by 2007.4 In the various rate calculations, the denominators (unit of exposure) for accident and fatality rates typically have been the number of aircraft departures, the number of passenger departures, or the number of flight hours.5

Fatality rates and fatal accident rates are the most important long-term measures of aviation safety. However, fatal accidents are rare, especially in travel on the major passenger airlines, so looking for changes in fatality rates or fatal accident rates may not give timely feedback regarding the impacts of new equipment, new programs, or changes in the airspace system on the potential risk of air travel.

1

 Gore Commission, Final Report, 1997, p. 4.

2

 Fatality rates measure the rate at which people are killed in aviation and are typically expressed in units of passenger fatalities per million enplanements. Fatal accident rates measure the rate at which aircraft are involved in accidents that produce fatalities and are typically expressed in units of fatal accidents per 100,000 flights.

3

Federal Aviation Administration (FAA), Zero Accidents: A Shared Responsibility, Washington, D.C., February 1995.

4

 FAA, Safer Skies: General Aviation Joint Steering Committee, Washington, D.C., June 9, 2009, available at http://www.faa.gov/about/initiatives/safer_skies/, accessed July 15, 2009.

5

 For a discussion of aviation safety measures, see Clinton V. Oster, Jr., John S. Strong, and C. Kurt Zorn, Why Airplanes Crash: Aviation Safety in a Changing World, Oxford University Press, Oxford, United Kingdom, 1992, Appendix A.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 7
2 Measures of Aviation Safety and Sources of Data This chapter discusses measures of aviation safety and provides an overview of the various aviation data sources and data that have been available before and since NASA’s NAOMS project was developed. 2.1 MEASuRINg AVIATION SAFETY How should the safety of the aviation system be measured? The primary goal of most aviation safety initiatives is to reduce the rate of fatalities during or as a result of air travel. Thus, the most important metrics are the rates of fatal accidents per unit of air travel (number of passenger trips, flight legs, flight miles, and so on). For example, the Gore Commission recommended that “the principal focus should be on reducing the rate of accidents by a factor of five within a decade. . . .”1 Annual reports published by many aviation safety organizations also focus on accident rates and their trends, measured in various ways, and many other studies on aviation safety emphasize fatality and fatal accident rates.2 For example, the FAA published the Aviation Safety Action Plan report entitled Zero Accidents: A Shared Responsibility, in February 1995.3 The FAA’s Safer Skies initiative (April 1998) called for an 80 percent reduction in the rate of fatal accidents by 2007.4 In the various rate calculations, the denomina- tors (unit of exposure) for accident and fatality rates typically have been the number of aircraft departures, the number of passenger departures, or the number of flight hours. 5 Fatality rates and fatal accident rates are the most important long-term measures of aviation safety. However, fatal accidents are rare, especially in travel on the major passenger airlines, so looking for changes in fatality rates or fatal accident rates may not give timely feedback regarding the impacts of new equipment, new programs, or changes in the airspace system on the potential risk of air travel. 1 Gore Commission, Final Report, 1997, p. 4. 2 Fatality rates measure the rate at which people are killed in aviation and are typically expressed in units of passenger fatalities per million enplanements. Fatal accident rates measure the rate at which aircraft are involved in accidents that produce fatalities and are typically expressed in units of fatal accidents per 100,000 flights. 3 Federal Aviation Administration (FAA), Zero Accidents: A Shared Responsibility, Washington, D.C., February 1995. 4 FAA, Safer Skies: General Aviation Joint Steering Committee, Washington, D.C., June 9, 2009, available at http://www.faa.gov/about/ initiatives/safer_skies/, accessed July 15, 2009. 5 For a discussion of aviation safety measures, see Clinton V. Oster, Jr., John S. Strong, and C. Kurt Zorn, Why Airplanes Crash: Aviation Safety in a Changing World, Oxford University Press, Oxford, United Kingdom, 1992, Appendix A. 

OCR for page 7
 AN ASSESSMENT OF NASA’S NATIONAL AVIATION OPERATIONS MONITORING SERVICE In an attempt to find a more timely indication of the effects of programs in improving safety or the impact of developments that might lead to a degradation of safety, analysts have turned to other events that they hope might elucidate changes in aviation safety. One approach has been to monitor and study incidents—events involving damage to an aircraft and/or injury, but in which the levels of damage or injury do not meet the NTSB thresholds that define an accident6—in the hope of more quickly detecting changes that could potentially affect safety. For example, the FAA Accident/Incident Data System (AIDS) contains reports of collisions between aircraft and birds while flights are on approach to or departure from an airport. Most of these collisions have not resulted in sufficient aircraft damage to be considered an accident by the NTSB. Nonetheless, the rate at which such collisions have occurred can be valuable safety information that might reveal an increase in the presence of birds around airports and suggest potential value in establishing programs to deter birds from nesting in areas adjacent to airports. One important problem, however, is that none of these other indices or events that analysts have turned to have been proven to be precursors to accidents or indicators of a pending increase in fatality rates or fatal accident rates. Even changes in the rates of nonfatal accidents or incidents have not been shown to be predictive of or associated with changes in the rates of fatal accidents. Thus, a link between any of these other indicators and the safety of the aviation system is at best uncertain. Another complicating factor in understanding and improving aviation safety is that accidents rarely have a single cause.7 Rather, they are typically the result of a sequence of events involving several malfunctions and/or mistakes. Consider a situation in which an event initiates a sequence of other events that result in damage to an aircraft and/or injury. The same event might occur in another flight but not lead to an accident or injury because something else was done to interrupt the sequence of events. By studying the circumstances and learning how the sequence of events was interrupted, it might be possible to incorporate such information into training or aircraft design and to reduce rates of future accidents. Consider, for example, a flight in which an engine fails during takeoff. With a modern passenger jet aircraft, an engine failure during takeoff should not result in an accident if the pilot takes the correct actions quickly. However, an engine failure places considerable pressure on the pilot, who may not react quickly enough to assess the situation and take the correct action. If one could identify, study, and learn from flights during which actions were taken in time to prevent an accident, it might be possible to improve pilot training so as to reduce the chances of such failures resulting in accidents in the future. 2.2 AVAILABILITY AND SOuRCES OF AVIATION DATA The accident and incident data available from NTSB and FAA are not the only sources of safety data. Two other sources on reports of potentially unsafe events are the Aviation Safety Reporting System (ASRS) and airlines’ Aviation Safety Action Programs (ASAPs). ASRS receives, processes, and analyzes voluntarily submitted incident reports from pilots, air traffic controllers, dispatchers, flight attendants, maintenance technicians, and others. ASRS grew out of the FAA’s Aviation Safety Reporting Program (ASRP), started on April 30, 1975. The FAA determined that the effectiveness of ASRP would be enhanced if the receipt, processing, and analysis of raw data were done by NASA—an independent third party with no enforcement responsibility—rather than by the FAA. This practice would ensure the anonymity of all parties involved, including the reporter, and would increase the flow of informa - tion. Accordingly, NASA designed and administered ASRS through a Memorandum of Agreement executed by the FAA and NASA on August 15, 1975, and subsequently renewed periodically. 8 In 1996, ASAPs were introduced in the flight domain with the hope of encouraging pilots to disclose their errors and, more importantly, the factors 6 Following is the definition of incident from International Civil Aviation Organization (ICAO) Annex 13 and FAA Order 8020.11b: “an occurrence other than an accident associated with the operation of an aircraft, which affects or could affect the safety of operations.” For more information on thresholds between an accident and an incident, see National Transportation Safety Board (NTSB), Form 0. Pilot/ Operator Aircraft Accident/Incident Report, NTSB, October 2006, available at http://www.ntsb.gov/aviation/6120_1_printonly.pdf, accessed June 11, 2009. 7 James Reason, Human Error, Cambridge University Press, Cambridge, United Kingdom, 1990. 8 See FAA, Advisory Circular 00-46D, Washington, D.C., February 26, 1997, available at http://rgl.faa.gov/Regulatory_and_Guidance_ Library/rgAdvisoryCircular.nsf/8e17c23e2f26e8018625726d006ce776/64358057433fe192862569e7006da716/$FILE/AC00-46D.pdf, accessed July 15, 2009.

OCR for page 7
 MEASURES OF AVIATION SAFETY AND SOURCES OF DATA contributing to their errors.9 Both ASAPs and ASRS allow for voluntary reports initiated by pilots and others in the aviation system, and both systems provide some degree of immunity from regulatory action or civil fines. These reports can play a valuable role in understanding and improving aviation safety. However, because they are based on voluntary reporting (resulting in a nonprobability sample; see Chapter 3 for a discussion of types of samples), they do not provide a means to develop statistically valid metrics of aviation safety. Around the same time that NAOMS was developed, the Flight Operational Quality Assurance (FOQA) program was started by several airlines in the mid- to late 1990s. Through their FOQA programs, airlines gather selective digital data from flight data recorders installed on their airplanes and then process and analyze those data to identify particular events. These data can provide extensive information about flight operations. Unlike ASRS and ASAP reports, FOQA data can be available from all flights on all properly equipped airplanes. Unfortunately, these data are not currently available for all segments of the NAS. Flight data recorders are typically not installed on general aviation airplanes, and some operators who do have airplanes with appropriate flight data recorders may not have the resources to gather, process, and analyze those data. However, both recent and future advances in computer and data-storage technology may well reduce the cost of both collecting and analyzing such digital data. It may also become easier to integrate FOQA data with other data sources such as air traffic control data from FAA’s Operational Error Detection Program. FOQA data are limited, however, in that they do not reveal the intentions of the pilot during the recorded events, so they may provide an incomplete picture of the event. More recently, FAA has launched the Aviation Safety Information Analysis and Sharing System (ASIAS). This system is housed and managed at the MITRE Corporation and contains both aviation safety and operations data and a collection of studies of specific aviation safety topics. ASIAS is designed to enable users to perform integrated inquiries across multiple databases, search an extensive warehouse of safety data, and display pertinent elements in an array of useful formats. ASIAS contains both the data sets and the query tools that allow easy access to the data.10 ASIAS is being developed in a phased approach. One can already access an array of aviation safety databases, including FAA Accident/Incident Data Systems, the Air Registry, the ASRS, Bureau of Transportation Statistics, the Near Midair Collision System, the NTSB Aviation Accident and Incident Data System, NTSB Safety Recom - mendations to the FAA with FAA Responses, and the World Aviation Accident Summary. 11 Additional databases planned for inclusion in ASIAS include ASAP data, FOQA data, and other data. 12 (A variety of other data sets are also available to the FAA and to safety analysts that are not included or planned to be included in ASIAS.) The use of statistical techniques to extract pertinent information from currently available data is an attractive approach, as it takes advantage of information that is already available. However, extracting, combining, analyzing, and understanding data from diverse sources involves many challenges. MITRE and others are working to develop additional tools to extract information from ASIAS data. All of the currently available data sources have their advantages and limitations. ASIAS has made considerable progress in allowing a wide variety of these databases to be easily accessed and integrated. If ASIAS continues to be developed as planned, it can become an even more useful source of data for the entire aviation system. 9 See FAA, Advisory Circular 00-58, Washington, D.C., May 4, 1998, available at http://rgl.faa.gov/Regulatory_and_Guidance_Library %5CrgAdvisoryCircular.nsf/list/AC%2000-58/$FILE/AC00-58.pdf, accessed July 15, 2008. 10 FAA, Aviation Safety Information Analysis and Sharing (ASIAS) System, Washington, D.C., available at http://www.asias.faa.gov/ portal/page?_pageid=56,398034,56_398041&_dad=portal&_schema=PORTAL, accessed May 16, 2009. 11 Ibid. 12 FAA, National Aviation Research Plan (NARP) Appendices, Washington, D.C., February 4, 2008, available at http://www.faa. gov/about/office_org/headquarters_offices/ato/service_units/nextgen/research_planning/narp/media/pdf/NARP_08_APP.pdf, accessed July 15, 2009; and Victoria Cox, Statement before the House Committee on Transportation and Infrastructure, Subcommittee on Aviation, March 18, 2009, available at http://www.faa.gov/news/testimony/news_story.cfm?newsId=10433, accessed July 15, 2009.