The following HTML text is provided to enhance online
readability. Many aspects of typography translate only awkwardly to HTML.
Please use the page image
as the authoritative form to ensure accuracy.
An Assessment of NASA’s National Aviation Operations Monitoring Service
Section A—Pilot background questions and relevant exposure information. The original responses from each pilot are categorized to the following groups:
Time of Interview by Year (4 levels): 2001, 2002, 2003, 2004.
Flight Hours (5 levels): Less than 51, 51-90, 91-130, 131-170, Greater than 170 (hours)
Aircraft size (4 levels): Small or Other, Medium, Large, and Wide-body with only the aircraft that was flown the most being reported
Propulsion (2 levels): Turbofan and Turboprop or Other
Flight Type (3 levels): Domestic, International, and Unknown
Crew Role (2 levels): Captain and First Officer or Other
Amount of Pilot Experience (3 levels): Low, Medium, and High
Mission type (2 levels): Passenger and Cargo or Other
Phase 1 provides pilot responses on flight hours (item 2) but not on flight legs (see Section 6.2).
Section B—Counts for safety events. The responses are provided in their original raw form except for the following modifications:
Responses that were considered to be rare (occurred in less than 0.1 percent of the surveys) were removed and were given in separate tables without linking them to the pilot response for other events.
“High unique” response values were replaced with the next closest numerical value in that field. “High unique response” denotes cases where a pilot’s response for an event count was unusually high in comparison to the responses of the other survey participants.
Section C—Responses to special questions concerning baseline performance measures and “in-close”changes to approach and landing. High unique and rare counts are redacted in the same manner as in Section B. Free-text responses are aggregated into a separate file.
Section D—Pilot feedback on the questionnaire. The numerical values are reported with free-text responses again aggregated into a separate file.
In addition, files of partial raw responses for aircraft type flown, complete set of hours and legs, and career hours flown are provided separately.
Other features of Phase 1 redaction include the following:
Individual pilot responses in Sections A and B are linked by means of a uniquely assigned Random Identification Number. This permits the examination of all the responses from a particular pilot in the analysis.
The main release was cleaned, with those rows having missing data or outliers reported in separate files.2
It appears that the redaction strategies were developed primarily with confidentiality issues in mind. The following comments discuss the advantages and disadvantages from the viewpoint of information value in the redacted data.
“Prior to the recent redaction steps taken, NAOMS air carrier survey responses were evaluated by Battelle at two stages. During initial processing, Battelle refined the set of survey responses using a technique called the Chebyshev process and related criteria to remove 322 responses of doubtful quality to avoid contaminating analyses of the responses. Battelle cites the following specific reasons for their removal: 1) number of flight hours too small; 2) unreasonable ratio of hours-to-legs; 3) unreasonable responses to multiple questions; and/or 4) Section B not completed. These responses are provided below and are identified as the ‘Outlier Survey Responses.’ A second refinement was then made by Battelle during subsequent tabulation activities when the NAOMS project team sought a set of responses with all explanatory flight activity variables present (no null values in Section A fields). This was done to ensure that all tabulations on the responses based on flight activity fields had a consistent total. This resulted in an additional 335 responses being removed. These responses are provided below, identified as the ‘Survey Responses with Unknowns in Flight Activity Fields.’” NASA, NAOMS Air Carrier Survey Responses, available at http://erc.ivv.nasa.gov/news/reports/NAOMS_air_carrier_survey_data.html, accessed June 11, 2008.