Cover Image

Not for Sale

View/Hide Left Panel
Click for next page ( 64

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement

Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 63
Survey Design 63 The second language issue concerns international passengers. As the number of people flying internationally continues to grow, more and more languages are being spoken at airports, par- ticularly at major gateway international airports. This prevalence of multiple languages is a real problem for surveys and one for which there is no ready solution. The survey planning team must determine if this will be an issue and attempt to arrive at a workable solution. Where inter- national passengers are primarily from a small number of countries, bilingual or multilingual interviewers could be scheduled to survey the appropriate flights. It will not usually be feasible to translate questionnaires into many languages, but if most passengers who cannot speak English speak a common language, a translation in this language could be developed. If the survey team is conducting a strict sampling process, the language barrier may interfere with this process and lead to some bias in the results. A related issue is that travelers whose first language is not English may appear to understand the questions, but in fact do not fully under- stand them and give incorrect answers. It is important that interviewers be trained to identify such situations and that they devote time to ensuring that the respondents understand the ques- tions and they understand the responses. A benefit of using a multilingual questionnaire is that it will allow the survey team to exam- ine whether the characteristics of non-English-speaking respondents are significantly different from those of English-speaking respondents, and thus how important it is to make special arrangements to include these respondents in future surveys. 4.9 Use of Electronic Data Collection Devices There is a growing interest in using hand-held electronic data collection devices (EDCDs) for performing interview and observation surveys. These devices have a number of distinct advan- tages over printed forms, as discussed in the following subsection, but also raise several issues that need to be carefully considered. 4.9.1 Advantages and Disadvantages One of the most obvious advantages of using EDCDs is the elimination of the data entry task and the associated potential for errors to be introduced in the data. EDCDs store the recorded data directly in a database that can typically be downloaded and combined with the corresponding data from other devices. Questions are displayed and responses are entered using a program running on the device. While the capabilities of commercial software for data capture vary, most, if not all, programs provide the capability of exporting the data in a format that can be read by or imported into commercial statistical analysis software packages and spreadsheet software. This capability can represent a significant cost and time savings, as well as eliminate a potential source of error. Some survey sponsors have raised the concern that the absence of a paper form means that there is no opportunity to go back and check the validity of the data in the data file. This is true, but no different from the use of paper forms. There is no way with paper forms to check whether the response that was recorded on the form is in fact what the respondent said or intended. The use of paper forms does allow the data entry task to be checked for errors, but this is an irrele- vant consideration with EDCDs because the task is eliminated. The electronic data files should of course be backed up regularly so that they can be restored if they become corrupted. There is also the possibility that failure or loss of an EDCD could result in the loss of the data stored on the device. This possibility is no different from the possibility of loss or accidental destruction of paper forms before the data have been extracted from them. Modern EDCDs are quite robust and preserve their data if they lose power. Survey procedures should provide for

OCR for page 63
64 Guidebook for Conducting Airport User Surveys periodic downloading of the data on EDCDs. Some devices can do this while in use through a cell phone or Wi-Fi capability. In the final analysis, if a survey sponsor wants a paper backup to the electronic data file, one can always be printed out. One caveat is that with paper forms, well-trained interviewers sometimes make notes on the forms to clarify responses or suggest cautions in interpreting the response. Depending on the software, programming EDCDs to provide the same capability may be possible, but entering these comments typically takes longer on an EDCD than on a paper form. A difficulty with using EDCDs is that they cannot be used to survey passengers who are hur- rying to catch a flight, because (unlike printed forms) they cannot be handed out to respondents to take with them. Inability to survey such passengers can result in biased information if these passengers have different characteristics from those interviewed. This issue can be addressed by providing interviewers with mail-back forms to hand to passengers in this situation. Because the response rate from mail-back forms is typically quite low, the number of such forms handed out should be recorded on the EDCD (including the serial number of the form and where distrib- uted) so that appropriate weights will be assigned to those forms that are returned. Use of EDCDs introduces additional costs for purchasing or renting the devices and the soft- ware and programming the devices. There will also be logistical issues regarding availability and recharging of devices. The requirement to be able to use EDCDs efficiently can limit the appli- cants when selecting interviewer staff, to some extent, and can introduce the need for additional training. These issues are discussed in the following subsections. Survey Design and Data Quality An important advantage of EDCDs is that they allow more complex branching based on responses to prior questions and can tailor subsequent questions to information already pro- vided. While branching can also be done with paper forms, it can quickly become unwieldy and confusing if there are more than a few such branches. There is no such limitation with an EDCD program and the resulting logic is transparent to the user. Thus survey questions can be tailored to individual situations and the questionnaire can include questions that apply to only a few respondents without affecting the questions asked of others. The ability to tailor questions on the basis of prior answers can also shorten the length of time required to conduct the survey by skipping questions that are not applicable to particular respon- dents. Also, with paper forms the questionnaire may appear very long to the respondent and adversely affect the response rate, but the length of the survey is not evident to the respondent when using EDCDs. However, for this reason it is generally advisable to inform respondents of the likely time required to complete the survey. Another advantage is the ability to do real-time data checking and ask follow-up questions to clarify apparently inconsistent responses. This ability allows the interviewer to correct misunder- standings or mistakes while the respondent is still available. In other cases it may provide an expla- nation for what appears to be inconsistent information but is in fact an unusual or unanticipated combination of circumstances. This is a powerful feature. As more detailed information becomes available about air passenger travel patterns, for example, it has become clear that the range of traveler behavior is much wider than is commonly assumed. Therefore, trying to fit the full range of behavior into only a few simplistic categories can lead to a misunderstanding of the situation. Cost and Technical Support Considerations Although EDCDs provide powerful capabilities, the units themselves can be fairly expensive and a large number may be required for a given survey. Survey firms and airports conducting fre- quent surveys will generally acquire their own equipment and allocate the cost over many differ- ent surveys. It may be cost effective for infrequent survey sponsors to rent units. In this situation,

OCR for page 63
Survey Design 65 it would be desirable for the survey planning team to purchase a few units for questionnaire devel- opment and post-survey use, but rent additional units for the period of the survey. There is also the cost of the data entry software to be considered. Depending on the software used, license fees can cost as much as the units themselves. However, the cost of the equipment and software has to be balanced against the saving in data entry costs, both for the current survey and for future sur- veys that will use the equipment. While the technology involved in using EDCDs will be familiar to anyone with a personal dig- ital assistant (PDA), some technical support will be necessary, particularly for programming the units and downloading the data. Survey firms with experience using EDCDs should be able to provide this level of technical support, either with their own staff or though a subcontract with a technical specialist who has appropriate programming experience. 4.9.2 Choice of Equipment The choice of equipment for EDCDs involves tradeoffs between cost, capability, and ease of use. With increasing capabilities in terms of memory, screen size, and battery life, standard PDAs are becoming widely used for survey data collection. Custom devices that have been designed specifically for use in survey data collection are also available. These generally have larger screens and keyboards but are typically bulkier and heavier, and thus harder to use. The ability to hold the unit in one hand while entering data with the other is an attractive feature of PDAs. Consideration should also be given to the need for staff to hold the units for extended periods of time while standing; therefore, compactness and light weight are distinct advantages. On the other hand, a small screen limits the font size and the number of response options that can be displayed on one screen, while a compact keypad is more awkward to use and more likely to lead to mistyping. The principal factors to consider in selecting equipment are Screen size and clarity. Data storage capacity. Software that can be used. Use of keypad versus stylus. Key size and keypad layout. Battery charge life and recharge time. Weight and handling. Data export capability. Wi-Fi or cell phone capability. Cost. Laptop and tablet computers have been used for some survey applications where complex, graphically intensive screen displays need to be shown to respondents. However, these are awk- ward to hold for extended periods and would require either that respondents and interviewers be seated during the interview or that interviewers be provided with a mobile cart to support the unit. 4.9.3 Programming Considerations Selection of software to program EDCDs involves the choice between purchasing commercial software and developing a custom program using a suitable programming language. Commercial software is available that has been designed for programming surveys on EDCDs, particularly PDAs. The more advanced software generally includes the capability to present the usual types of questions and response formats, including categorical check-box questions and free text input, as well as to program skips to different questions based on the responses to prior questions. However, this software often has limited ability to display sophisticated graphics or custom screen layouts. Thus for applications that require this capability, it may be necessary to

OCR for page 63
66 Guidebook for Conducting Airport User Surveys develop a custom program. In this situation most survey sponsors will retain a survey firm that already has this capability. Selecting an appropriate commercial software package involves deciding which program fea- tures and capabilities are required and which packages support the EDCDs being considered. The choice of EDCD and software should be approached as an integrated decision, because such issues as screen size and keypad design interact with how the software displays the questions and allows the responses to be entered. As with any other software and hardware, capabilities are continually evolving and hardware costs are dropping, so the survey planning team or contractor will need to assess the currently available technology before making a purchase. Some questions to consider when reviewing commercial software follow: What question formats are supported? Can the software combine different types of question format (e.g., both check-box responses and free-text answers) in the same screen display? How does the software handle questions that have more response options than will fit on a single screen? What are the limits to the number of response options a given question can have? Can the user customize the code that is used for each categorical response for a given question? Can the user customize the order that categorical responses appear on the screen? What are the limits to the number of questions? Can the software use an existing data file to control what response options for a question are displayed based on prior responses (e.g., listing hotels in a given city)? Can the software display response options based on the first few characters entered (e.g., sug- gesting city names based on the first two or three letters)? Do the options get refined as more letters are entered? Can the user restrict the format used for numerical entries (e.g., require that a zip code have five digits and no decimal point)? Can the user specify a template for numerical entries (e.g., telephone numbers as xxx-xxx-xxxx)? Can the user specify range and consistency checks to be made on numerical responses? Does the software provide specific capabilities to enter dates and times? Does the software support touch-screen responses? Can the software display a keyboard on screen for entering free text? Can the software display graphical features? Can the software allow the user to select a location on a graphical feature (e.g., a location on a map)? Does the software allow logic branches, and how easy is it to program logic branches? How much memory or disk space does the program occupy? How much memory or disk space is required to store each survey response, and how does this vary with the number of questions? What data download formats does the software support? 4.9.4 Survey Logistics Other considerations also need to be taken into account in using EDCDs. There will need to be a secure room where the equipment can be left when not in use and where the data can be down- loaded. It will usually be necessary to have spare batteries that can be charged while the units are in use or to have spare EDCDs to use while others are being recharged. Downloading the data from the units will typically require a desktop or laptop computer. Access to a telephone line or Wi-Fi link will allow the downloaded data to be transferred to a central database on a regular basis. Because the units themselves are fairly expensive (several hundred dollars each), appropriate steps

OCR for page 63
Survey Design 67 should be taken to ensure their security while in use. Units should be checked in and out to sur- vey staff at the start and end of each shift, and staff should be trained not to let them out of their possession while they are in use. It may be prudent to provide each unit with a short tether that can be clipped to the interviewer's apron or vest to help prevent loss and damage. 4.9.5 Staffing Considerations Use of EDCDs will generally require staff training. Some older interviewers may have less experience handling small electronic devices, may have difficulty using the small keypads or see- ing the screen display fonts, and may require additional practice to become comfortable using the devices. The other staffing consideration relates to programming and technical support. The program- ming can be subcontracted if nobody on the survey planning team has the necessary skill and expe- rience, but this will incur additional cost and administrative effort. The team will need to coordi- nate closely with the programmer to ensure that the finished program performs satisfactorily. Although technical support in the field does not require the same level of specialized skill as pro- gramming, and will generally be within the capabilities of survey supervisory staff, some training may be required beforehand if the field supervisors have not performed this function previously. 4.9.6 Summary Although EDCDs offer a number of distinct advantages, experience in their use for airport user surveys has been mixed. Problems can arise if their use has not been carefully planned and thought through. As with any evolving technology, some of the difficulties that arose with the early hardware and software have been overcome. Even so, the effective use of EDCDs requires some technical skills and experience. Survey sponsors should consider on a case-by-case basis whether the advantages justify the effort involved. In general, hand-held EDCDs are only prac- tical to use with interview surveys. Printed forms will typically be required for self-completed surveys. Surveys with the following characteristics tend to favor the use of EDCDs: A long or complex set of questions with multiple branches. The need to tailor questions to the respondent's answers to earlier questions. Questions with a large number of categorical response options. A large sample size (resulting in high data entry costs). The desire to include data consistency checks during the survey interviews. Availability of survey staff with experience in the use of EDCDs. The survey is likely to be repeated frequently (spreading the cost of programming over several surveys). On the other hand, surveys with the following characteristics may be more appropriately per- formed using printed forms: A fairly simple questionnaire with limited branching. A relatively small sample size. Straightforward questions that do not require data consistency checks. Respondents with limited time to answer questions, who may need to be given the survey form to complete and mail back later. Respondents need to enter information or complete the questionnaire themselves. Lack of staff with appropriate experience in the use of EDCDs. No plans to repeat the survey in the near future.