The NAOMS survey was originally designed to collect information on safety-related events as experienced by all of the frontline operators of the NAS, including pilots, controllers, mechanics, flight attendants, and others. However, the survey of pilots appears to have taken longer and required more resources than expected,6 and surveys of the other groups did not even reach the stage of development. The AC and GA pilots were surveyed from April 2001 through December 2004. NASA decided to discontinue support for NAOMS toward the end of this period, and the survey was transformed into a Web-based tool and handed off to the Air Line Pilots Association (ALPA).7


The NAOMS project was jointly managed by NASA and the Battelle Memorial Institute, NASA’s subcontractor for the project. A total of 29,882 pilots were surveyed as part of the study over the period from April 2001 to December 2004.8 Of these pilots, 25,105 participated in the AC survey. The survey of GA pilots was conducted only for a brief period (approximately 9 months) and involved 4,777 completed interviews. The FAA’s Airmen Certification Database (ACD) was the source from which the sample of pilots to be surveyed was selected.

Each pilot who responded to the survey was asked a set of questions relating to the following: background information, the number of hours and flights that the pilot had flown, the number of events from among numerous possible safety-related events that the pilot had observed, some topic-specific questions, and feedback about the survey. All questions related to events that had occurred within a specific time range (recall period). For most but not all of NAOMS, this recall period was 60 days. Both AC and GA versions of the survey had the same basic structure, with four sections (see Appendixes G and H in this report).

Participation in the NAOMS project was completely voluntary. All data provided by NAOMS respondents were held in confidence. NAOMS maintained records of survey participants, but there is no linkage in NAOMS data repositories between the names of past respondents and the data that they provided.9 To maintain the confidentiality of the survey participants, NASA released only redacted versions of the survey data. Only these redacted data sets were made available to the NRC’s Committee on NASA’s National Aviation Operations Monitoring Service (NAOMS) Project: An Independent Assessment.

More detailed descriptions of the sampling design, the survey questionnaires, the redacted data, and other features of the NAOMS project are provided in the following chapters.


The rest of this report is organized in six chapters that address the various charges in the committee’s statement of task (presented in the Preface).

Chapter 2 discusses ways of measuring aviation safety and describes sources of data on aviation safety that existed before NAOMS or that became available after NAOMS was developed. While fatalities and accident rates are the ultimate measures of aviation safety, the committee notes that there is considerable value in collecting and examining information from certain types of operational data. There were few sources of aviation data when NAOMS was conceived, and many more have become available since then. However, each of these data sources has its advantages and limitations. The FAA’s Aviation Safety Information Analysis and Sharing System (ASIAS) is intended to allow easy access to a wide variety of existing databases. It is being developed as a source of data for the entire aviation system.

Chapter 3 assesses the usefulness of sample surveys in providing information on aviation safety. A sample survey is a scientifically valid and effective way to collect data and track trends about events that are potentially related to aviation safety. It can be used to collect reliable information about all segments of civilian aviation,


See NASA, Final Memorandum on the Review of the National Aviation Operations Monitoring Service, NASA, Washington, D.C., March 31, 2008, p. 3 (also known as the NASA Inspector General’s Report).


Ibid., p. 38.


Battelle, NAOMS Reference Report, 2007, p. 13. Other sources provide slightly different numbers, in part because of reclassifications, different data releases, and so on.


Ibid., p. 8.

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement