refine the survey methodology. These activities are part of a well-designed survey, especially one conducted as a research study to assess the feasibility of survey methodology in aviation safety.

Based on the analyses and findings of the committee, the publicly available NAOMS data should not be used for generating rates or trends in rates of safety-related events in the National Airspace System. The data could, however, be used in developing a set of lessons learned from the project.

The committee encountered several challenges in assessing NAOMS survey methodology and the potential utility of the data. The committee did not have access to the original, unredacted data. Assessing the utility of the NAOMS data based on heavily redacted data is not an easy task, and it is further complicated by NASA’s release of multiple data sets under different redaction methods. Further, as pointed out by the Government Accountability Office’s review of NAOMS: “The project staff and contractors did not assemble a coordinated, clear history detailing the project’s management that would facilitate evaluation of the overall air carrier pilot survey.”¹⁰ The lack of documentation and the delays in obtaining some documents made it difficult for the committee to obtain a full understanding of the steps taken and decisions made (including their rationale) during the NAOMS project. The committee frequently had to rely on statements based on the memory of those involved in the project and, therefore, it received a variety of recollections. Perhaps if decisions had been more clearly documented, there would have been fewer divergent views regarding the various decisions and processes that occurred during the project.