National Academies Press: OpenBook

An Assessment of NASA's National Aviation Operations Monitoring Service (2009)

Chapter: Appendix F: List of Additional Survey Questions with Problems Discussed in Chapter 5

« Previous: Appendix E: Additional Examples of Surveys by Federal Agencies
Suggested Citation:"Appendix F: List of Additional Survey Questions with Problems Discussed in Chapter 5." National Research Council. 2009. An Assessment of NASA's National Aviation Operations Monitoring Service. Washington, DC: The National Academies Press. doi: 10.17226/12795.
×

F
List of Additional Survey Questions with Problems Discussed in Chapter 5

This appendix lists additional questions from the AC and GA questionnaires of the NAOMS survey that were identified by the committee as potentially problematic.

AC2. During the last (TIME PERIOD), how many times did an aircraft on which you were a crewmember perform an evasive action to avoid an imminent in-flight collision with another aircraft that was never closer than 500 feet including evasive action in response to a TCAS advisory? [Question GAC2 on the GA questionnaire asks for the same information.]

In addition to the complex structure pointed out in Section 5.2.3, question AC2 asks the pilot if the aircraft was or was not closer than 500 feet during an evasive maneuver. It would be difficult for pilots to make such an assessment, so the question is likely to result in answers that are not very reliable.

AC3. During the last (TIME PERIOD), how many times did an aircraft on which you were a crewmember experience less than 500 feet of separation from another aircraft while both aircraft were airborne? [Question GAC3 on the GA questionnaire asks for the same information.]

The concern noted in the first sentence for question AC2 above also applies here, particularly if either aircraft had taken an evasive maneuver.

ER1. How many times during the last (TIME PERIOD) did an aircraft on which you were a crewmember divert to an alternate airport or return to land because of an aircraft equipment problem? A. What systems caused the diversion or return to land? [Question GER1 on the GA questionnaire asks for the same information.]

Without post-flight analysis data, the pilot may not have known the actual cause of a failure of any specific system. For example, a pilot could easily have thought it was a hydraulic system failure when it actually was an electrical pump failure.

ER2. How many times during the last (TIME PERIOD) did an aircraft on which you were a crewmember experience a spill, fire, fumes, or aircraft damage due to transporting hazardous materials? C. (How many of these times were the spills, fire, fumes or aircraft damage/Was the spill, fire, fumes or aircraft damage) caused because the hazardous materials in question were out of compliance with regulations?

Although an air carrier pilot is required to know the regulations for “loading” hazardous material on board the aircraft, the pilot is not required to know and would not likely have known the specific regulations for packaging in order to determine that the material was not packaged in accordance with the regulations.

Suggested Citation:"Appendix F: List of Additional Survey Questions with Problems Discussed in Chapter 5." National Research Council. 2009. An Assessment of NASA's National Aviation Operations Monitoring Service. Washington, DC: The National Academies Press. doi: 10.17226/12795.
×

ER4. How many times during the last (TIME PERIOD) did an in-flight aircraft on which you were a crewmember experience uncommanded movements of any of the following devices…? [Question GER2 parts A through F on the GA questionnaire asks for the same information].

Without a post-flight analysis of the data, the pilot is unlikely to know which devices experienced uncommanded movements. Specifically, the pilot would not necessarily know if one surface moved initially and thus caused another or other surfaces to move, the secondary movement being what was observed and reported.

ER7. During the last (TIME PERIOD) how many times did an inflight aircraft on which you were a crewmember experience a total engine failure? [Question GER5 on the GA questionnaire asks the same question.]

A situation that might be interpreted as a total engine failure by the pilot may not, in the final analysis, be considered or reported as an engine failure in FAA or industry databases. For example, an accessory or component failure that reduced thrust or revolutions per minute could cause the perception of an engine failure, but upon analysis, it would be determined that the engine itself did not fail and the event would not be reported as an engine failure in FAA or industry databases because the event did not meet the established definition of an engine failure.

TU2. During the last (TIME PERIOD), how many times did an aircraft on which you were a crewmember encounter wake turbulence that resulted in 10 or more degrees of aircraft roll? [Question GTU2 on the GA questionnaire asks a similar question.]

The actual extent of the roll cannot be reasonably determined by a pilot without post-flight analysis of the flight data recorder. At some airlines, such post-flight analysis has often indicated that the “startle factor” of uncommanded pitch, roll, or yaw change has resulted in pilots overestimating the degree of change encountered. General aviation planes are not typically equipped with flight data recorders, which rules out even the possibility of a post-flight analysis of data in this segment.

WE5. During the last (TIME PERIOD), how many times did an aircraft on which you were a crewmember encounter wind shear or a microburst condition that resulted in an airspeed deviation of 15 knots or greater? [Question GWE4 on the GA questionnaire asks for the same information.]

The extent of the airspeed deviation cannot be verified without post-flight analysis of the flight data recorder. In such atmospheric conditions, it is very difficult for the pilot to assess the airspeed deviation accurately because of the limited time and ability to view the airspeed indicator during a recovery procedure.

There is an additional question—GER7, below—on the GA questionnaire that the committee believes asks pilots for information that they were unlikely to be in a position to have.

GER7. During the last 60 days when you were a pilot or copilot, how many times did you discover that (an airplane/a helicopter) had incorrect or bogus parts installed?

A pilot would not likely know that an incorrect or bogus part had been installed unless the people performing maintenance or repair on the aircraft communicated that information to the pilot.

The following questions from the AC and GA questionnaires were identified by the committee as questions in which there were problems with the structure and/or wording of the question:

AH10. During the last (TIME PERIOD), how many times did an aircraft on which you were a crewmember take off with an out-of-limit center of gravity?

It is unclear if question AH10 asks whether the pilot knew about the center of gravity (CG) condition prior to starting the takeoff roll or whether the pilot did not know about the CG condition until becoming airborne and experiencing flight control surface difficulties. In the first case, the error is with the pilot’s judgment for taking off in an aircraft known to have an out-of-limit CG, and in the second case the error is with the ground crew for failure to load the aircraft within its CG limits. From the standpoint of trying to reduce such potentially unsafe events, distinguishing between these two markedly different causes would be critical.

AH11. During the last (TIME PERIOD), how many times did an aircraft on which you were a crewmember take-off overweight?

Suggested Citation:"Appendix F: List of Additional Survey Questions with Problems Discussed in Chapter 5." National Research Council. 2009. An Assessment of NASA's National Aviation Operations Monitoring Service. Washington, DC: The National Academies Press. doi: 10.17226/12795.
×

It is unclear if the question asks whether the pilot knew about the overweight condition prior to starting the takeoff roll or if the pilot did not know about the overweight condition until becoming airborne and experiencing aircraft performance difficulties. In the first case, the error is with the pilot’s judgment for taking off in an aircraft known to be overweight, and in the second case the error is with the ground crew for failure to load the aircraft within its weight limits. From the standpoint of trying to reduce such potentially unsafe events, distinguishing between these two markedly different causes would be critical.

AH13. During the last (TIME PERIOD), how many times did an aircraft on which you were a crewmember experience an unusual attitude for any reason?

There is no clear definition of what constitutes “unusual” attitude, and the perception of “unusual” is likely to vary considerably among pilots and aircraft types/models. As noted in Section 5.2.3, cognitive testing of such questions should have been conducted during the questionnaire design stage.

AH15. During the last (TIME PERIOD), how many times did an aircraft on which you were a crewmember nearly collide with terrain or a ground obstruction while airborne?

The term “nearly collide” can be interpreted broadly. Some pilots might not consider an event a near collision with terrain unless the Ground Proximity Warning System (GPWS) is activated. Other pilots might apply different criteria or be in aircraft not equipped with a GPWS.

AT1. During the last (TIME PERIOD), how many times was an aircraft on which you were a crewmember unable to communicate with A.T.C. in a time-critical situation because of frequency congestion?

The interpretation of “time critical” is likely to vary across respondents. Some pilots would report “time critical” when they could not communicate with ground control or clearance delivery even though the aircraft was not underway at the time, whereas others would not. Again, cognitive testing of the question would have been appropriate.

AT2. How many times during the last (TIME PERIOD) did an aircraft on which you were a crewmember fly at an undesirably high altitude or airspeed on approach due to an A.T.C. clearance?

What a pilot might consider an “undesirably high altitude or airspeed on approach” is likely to vary significantly with the type of aircraft being flown. This question also combines two different events, high altitude and high airspeed, into a single response.

GE2. During the last (TIME PERIOD), how many times did an aircraft on which you were a crewmember collide or nearly collide with a ground vehicle?

As with Question AH15, the term nearly can be interpreted broadly. Parts A through C of this question ask where the near collisions occurred but provide no definition of near collision. The interpretation of nearly or near collision is likely to vary among respondents and circumstances. For example, if one pilot saw the ground vehicle and could determine that its path was not going to result in a collision even though it would pass close to the aircraft, that pilot might not consider that a near collision. However, if another pilot did not see the ground vehicle until the last minute when it passed close to the aircraft, that pilot might be startled and consider the event a near collision even though the vehicle might not have been any closer to the aircraft in the second situation than in the first.

GE3. During the last (TIME PERIOD), how many times did an aircraft on which you were a crewmember skid, slide, or hydroplane resulting in a significant increase in stopping distance during landing?

There is often some degree of hydroplaning resulting in some increase in stopping distance when landing on a wet runway, particularly one with puddles of standing water. A pilot’s perception of what was significant could well vary depending on the circumstances. For example, an increase in stopping distance might be considered significant on a short runway, whereas the same increase in stopping distance might not be considered significant on a long runway.

GE10. During the last (TIME PERIOD), how many times did an aircraft on which you were a crewmember nearly

Suggested Citation:"Appendix F: List of Additional Survey Questions with Problems Discussed in Chapter 5." National Research Council. 2009. An Assessment of NASA's National Aviation Operations Monitoring Service. Washington, DC: The National Academies Press. doi: 10.17226/12795.
×

experience a ground collision with another aircraft while both aircraft were on the ground?

As with question GE2, this question provides no definition of what constitutes nearly. Parts A through C of this question ask where the near collisions occurred, but provide no definition of near collision. Refer to the explanation above, provided for question GE2.

TU1. During the last (TIME PERIOD), how many times did an aircraft on which you were a crewmember encounter severe turbulence that caused large abrupt changes in altitude, airspeed, or attitude? [Question GTU1 on the GA questionnaire asks for the same information.]

The interpretations of the terms severe turbulence or large abrupt changes in altitude, airspeed, or attitude can vary considerably. Even if definitions had been provided, it is unlikely that the extent of the turbulence or change in altitude, airspeed, or attitude could be verified with needed accuracy without post-flight analysis of the flight data recorder. In addition, question TU1 asks the respondent to combine multiple events into a single response.

WE1. During the last (TIME PERIOD), how many times did an aircraft on which you were a crewmember lack accurate weather information when crewmembers needed it while airborne? [Question GWE3 on the GA questionnaire asks for the same information.]

The question does not provide a definition of what constitutes accurate weather information. The perception of the extent to which the weather information was accurate or sufficiently accurate for their needs is likely to vary among respondents. As noted before, cognitive testing of the question would have been appropriate.

Suggested Citation:"Appendix F: List of Additional Survey Questions with Problems Discussed in Chapter 5." National Research Council. 2009. An Assessment of NASA's National Aviation Operations Monitoring Service. Washington, DC: The National Academies Press. doi: 10.17226/12795.
×
Page 62
Suggested Citation:"Appendix F: List of Additional Survey Questions with Problems Discussed in Chapter 5." National Research Council. 2009. An Assessment of NASA's National Aviation Operations Monitoring Service. Washington, DC: The National Academies Press. doi: 10.17226/12795.
×
Page 63
Suggested Citation:"Appendix F: List of Additional Survey Questions with Problems Discussed in Chapter 5." National Research Council. 2009. An Assessment of NASA's National Aviation Operations Monitoring Service. Washington, DC: The National Academies Press. doi: 10.17226/12795.
×
Page 64
Suggested Citation:"Appendix F: List of Additional Survey Questions with Problems Discussed in Chapter 5." National Research Council. 2009. An Assessment of NASA's National Aviation Operations Monitoring Service. Washington, DC: The National Academies Press. doi: 10.17226/12795.
×
Page 65
Next: Appendix G: Air Carrier Questionnaire »
An Assessment of NASA's National Aviation Operations Monitoring Service Get This Book
×
Buy Paperback | $51.00 Buy Ebook | $40.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The National Research Council of the National Academies was requested by the National Aeronautics and Space Administration (NASA) to perform an independent assessment of NASA's National Aviation Operations Monitoring Service (NAOMS) project, which was a survey administered to pilots from April 2001 through December 2004.

The NRC reviewed various aspects of the NAOMS project, including the survey methodology, and conducted a limited analysis of the publicly available survey data. An Assessment of NASA's National Aviation Operations Monitoring Service presents the resulting analyses and findings.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!