Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 58
58 Guidebook for Conducting Airport User Surveys
It may then be necessary to take corrective action, either by coaching an interviewer who
missed something during training or by retraining the entire group if the trainer failed to con-
vey a point well enough. The survey planning team would be wise to budget for both of these
occurrences, particularly if the questionnaires or procedures are complicated or difficult.
It is also important either to train backup interviewers, who may or may not be deployed, or
to provide for a second training session for new hires if people need to be replaced. As it is diffi-
cult to find people who are willing to be trained without any assurance that they will have the
opportunity to participate in the survey, the latter approach is more typical. However, a second
training session can present scheduling difficulties because attrition of the trained interviewers
can occur throughout the survey. Another problem arises with recruiting and training additional
interviewers at short notice if they have to be issued with security badges, because this process
can be quite time-consuming. Therefore, it may be better to recruit and train enough interview-
ers at the start of the survey to allow for some attrition, and adjust the length of the survey period
and the hours worked by each interviewer to manage the attrition that actually occurs.
Finally, it is important to note that when mistakes happen, the decision about whether to retain
and tabulate work that is imperfect is both a research and a policy decision. The research decision
has to do with the necessity of having enough interviews to achieve the desired statistical preci-
sion and power; the policy decision has to do with the importance of a given piece of information.
It is also important to consider just how imperfect less than "perfect" work really is.
4.7 Pre-Tests and Pilot Tests
Pre-tests of questionnaires and pilot tests of survey procedures provide an opportunity to
identify any problems with question wording or other aspects of the questionnaire as well as the
planned arrangements for performing the survey. Such tests allow corrections or adjustments to
be made before the main data collection effort.
4.7.1 Pre-Test of Questionnaires
The goal of a pre-test of the questionnaire is to make sure the survey questions can be asked
easily by the interviewers, are clear to the respondents, and produce the desired information. Pre-
tests also provide an opportunity to identify unanticipated responses or situations and to adjust
the question response categories or the survey script as necessary.
An initial pre-test may be performed on a convenient group of people who were not involved
in developing the instrument, such as staff of the survey sponsor. However, while this may help
improve the question wording and identify unanticipated responses that require changes to the
question response categories, it is not a truly representative test of the questionnaire. It should
be followed by a pre-test involving a sample of the intended respondents of the survey, per-
formed in the same way as planned for the survey itself. This pre-test will increase the chances of
identifying unanticipated responses or situations. The questionnaire designers should be
involved in conducting the pre-test, because they will be able to identify unanticipated responses
or any misunderstanding of the intent of a question better than someone who was not involved
in the development of the questions.
The results of the pre-test should be subject to careful analysis to identify any apparent difficul-
ties with the question wording or survey flow, such as missing or incomplete answers, illogical
responses, or incorrect skips. In particular, responses of "Other" to categorical questions, where the
respondent has provided an explanation, should be examined to identify any misunderstanding of
the categories or commonly occurring responses that should be added to the designated categories.
OCR for page 59
Survey Design 59
If electronic data collection devices are to be used in the survey, the programming needs to be
thoroughly tested and debugged before the pre-test, or the validity of the pre-test may be com-
promised by errors in the programming. A rigorous testing plan for the software should be
included in the project schedule. This program testing, done by people other than the program-
mers, will attempt to ensure that all possible response options and question branches are tested.
This can be quite time-consuming and can be facilitated by preparing a set of response scripts
that test all possible branches from each question. These test scripts ensure that certain response
options are not overlooked, and can also be used to check that changes made to fix any problems
have not affected other parts of the program. The survey schedule should allow enough time to
adequately test the programming and make any needed revisions before the first pre-test.
4.7.2 Pilot Test of Survey Procedures
The primary purpose of a pilot test of survey procedures is to identify any problems with the
planned approach to conducting the survey and to refine the estimates of the number of field
staff required. Because a pilot test typically involves performing the survey on a representative
sample of potential respondents, it presents an opportunity to conduct a pre-test of the survey
questionnaire, and the two tests are often combined.
Other objectives of a survey pilot test include the following:
· Testing the survey sampling strategy.
· Validating the sampling plan.
· Identifying problems with survey logistics.
· Checking the data quality.
Feedback from the pilot test should include both field observations by supervisory staff and
issues reported by field staff in debriefing sessions, supplemented by an analysis of the survey
responses collected. In the case of interview surveys, noting the start and end time on each
response will enable this analysis to examine the average time to perform a survey as well as the
interval between ending one interview and starting the next. This will provide information on the
amount of time spent waiting for subjects to become available to be interviewed as well as the time
required to move between locations. Field staff should also note refusals to participate and record
the reason where given or apparent, such as a language barrier or cell phone conversation.
The pilot test results should be compared to the expected number of responses from the sampling
plan. This comparison may require collecting data on the potential number of respondents, such as
the number of passengers on flights surveyed in airline gate lounges. The results should be exam-
ined for data quality, including missing or incomplete information, apparently illogical or incon-
sistent responses, or difficulty interpreting free-text write-in answers. Where addresses or other
location data are collected, these should be examined to ensure that the location can be identified.
While this analysis addresses many of the same issues as the survey pre-test, it focuses on the
conduct of the survey rather than the design of the questionnaire. Results of the pilot test analy-
sis can be used to identify issues requiring particular attention in training field staff for the data
collection phase.
4.7.3 Scheduling of Pre-Tests and Pilot Tests
Pre-tests and pilot tests should be performed far enough in advance of survey data collection
that there is time to adequately analyze the results and resolve any issues, as well as perform any
additional testing required. This process is likely to require several weeks to do well. Any changes
to the questionnaire have to be finalized before the forms are printed for the data collection phase
