Guidebook for Conducting Airport User Surveys (2009)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

125 Many of the issues related to planning and designing tenant surveys are common to other types of airport user surveys, and the reader will be referred to those sections of the guidebook where applicable. Airport tenants include a wide range of organizations, such as concession operators (retail, food and beverage, car rental, courier, entertainment, etc.), airlines, government agencies, orga- nizations providing aircraft and aviation services, organizations handling cargo and mail, gen- eral aviation aircraft owners and fixed-base operators, and often non-aviation businesses as well. The particular tenants to survey will depend on the goals and purpose of the survey. 7.1 Purpose of the Survey and the Data to Be Collected Tenant surveys are conducted for a variety of reasons. Typical examples, and the types of data collected, include the following: • To obtain information to determine the economic impact of an airport. The types of informa- tion collected typically include data on gross revenues, wages, and taxes; numbers of employ- ees by part-time or full-time status and type of work; and the value of capital assets and capital expenditures. • To determine satisfaction with the services provided by the airport in its role as landlord. This survey could include tracking tenant satisfaction and service requirements, identifying key fac- tors that influence overall tenant satisfaction, gathering feedback about the quality of the services provided by the airport and its efficiency in meeting tenant needs, and identifying opportunities for enhancing tenant satisfaction and value. This type of survey should be conducted by a third party, and responses should be kept confidential and released in aggregate form only. • To obtain information on customer service through techniques such as “mystery shopper” sur- veys. The survey could also include price comparisons with identical or similar off-airport shops and services (e.g., restaurant chains). The results are used to make specific improvements to concessions. As with all airport user surveys, the first step in conducting a tenant survey is to outline its goals and purpose. (See Chapter 2 for a discussion on specifying goals, defining the purpose of a survey, the importance of doing so, and who should be involved.) 7.2 Survey Methodology The options for conducting tenant surveys include using Internet, fax-in, or mail surveys (or a combination of these) or on-site visits. Because the number of tenants is generally relatively small, C H A P T E R 7 Tenant Surveys

all tenants in the categories of interest are usually surveyed, especially if the results will be compiled for tenant subgroups. Where there are many tenants, selection of a sample is appropriate. Survey- ing all tenants will help ensure an adequate response rate. Mystery shopper surveys must obviously be conducted on site. These surveys are very different from other tenant surveys and are discussed separately in Section 7.7. 7.2.1 Internet Surveys Internet surveys are ideal for conducting tenant feedback and economic impact surveys. The airport should have contact information for all tenants, which will generally include email addresses. There may be exceptions in the case of general aviation aircraft owners or small, inde- pendent concession operators, who may have to be surveyed using another method. All tenants to be surveyed are sent an email asking them to participate in the survey and pro- viding a link to the questionnaire on the World Wide Web. (See Section 1.4 for a discussion on setting up a Web-based survey.) It is often a good idea to notify tenants in advance of the survey by email or telephone and to follow up with non-responding tenants, again by email or prefer- ably telephone. Using Web-based surveys with initial contact via email has the following advantages: • The survey is relatively easy to set up and administer. • The use of email allows the survey invitation to be easily forwarded to the appropriate person in the company. • It allows respondents to complete the survey at a time convenient to them and to do so in more than one session, if they wish (depending on the software used). • It allows easy tracking of the number of responses and requires no data entry. • Response rates are usually much better than with mail surveys. The main disadvantages are that some effort may be required to compile a tenant email address list and some tenants may not have email addresses or easy access to the Internet. A mail or fax- in survey could be used for those tenants without email addresses. Many Web-based survey tools allow for mailed and faxed replies to be entered manually so that they can easily be incorporated into the Web-based survey results. However, response rates for Web-based surveys are not as high as with on-site visits. 7.2.2 Mail or Fax-in Surveys Mail and fax-in surveys are similar in approach to Internet surveys but require each tenant’s mailing address rather than email address. Questionnaires are mailed to each tenant with a pre- paid reply envelope, and responses are either mailed or faxed back. The advantage of mail surveys is that all tenants will have a mailing address. The main disadvan- tages are that response returns are often slow, the response rate is typically low, and responses must be entered into a database for analysis. Non-responses are usually followed up by telephone, which is more costly than sending bulk reminders via email but is generally more effective, particularly if the emails have not been reaching the correct person. 7.2.3 On-Site Surveys On-site surveys are the only viable method for mystery shopper surveys (discussed in Section 7.7). On-site surveys can also be used for tenant feedback and economic impact surveys, but Internet or mail surveys are generally preferable. Tenant staff available on site will often not be able to provide 126 Guidebook for Conducting Airport User Surveys

Tenant Surveys 127 the required information and may be busy with customers, so interviews should be done by appoint- ment. Tenant staff may also have to check company records to respond to some questions, thus some questions might go unanswered with only a single interview. Generally, tenants are contacted by mail or telephone to advise them about the survey and possibly schedule an appointment. Send- ing the questionnaire in advance gives the tenants a chance to assemble any information needed. The advantage of on-site surveys is that a deeper understanding of the issues is possible through in-person interviews. The major disadvantages are that scheduling all the required interviews is often difficult and on-site surveys are more costly to conduct. 7.2.4 Confidentiality As confidentiality is an issue for tenant surveys, it is recommended that they be conducted by respected third parties. Assurances should be given to the tenants that the information will be kept in strict confidence and provided to the airport operator in aggregate form only. 7.3 Sampling Methodology In the case of Internet or mail surveys, questionnaires or survey requests will generally be sent to all tenants at the airport. However, including all tenants could be costly, especially for on-site interviews at large airports. If only a sample of tenants is required, a stratified sample of tenants should be surveyed where the tenants are grouped by category—for example, concessions, air- lines, government and security agencies, individual aircraft owners, and other organizations. The sample size will depend on the purpose of the survey, the desired level of accuracy, and the num- ber of tenants in each category. For random sampling, required sample sizes to achieve an accuracy of better than ±5 percent- age points for a categorical variable are given in Table 7-1 for a range of total numbers of tenants at the airport. As discussed in Section 3.4, the required sample sizes vary depending on the propor- tion of the population in the category of interest. This proportion will not be known at the time of planning the survey and must be estimated, at least approximately, based on past surveys, experi- ence of other airports, and/or knowledge of the airport. Required sample sizes are given for three values of this proportion: 0.5, 0.25 and 0.1.29 The largest samples are required when the proportion of the population in the category of interest is 0.5. Because most surveys ask multiple questions with various, unknown proportions in each category, the sample size corresponding to a propor- tion of 0.5 should be used unless the survey sponsor is primarily interested in questions where the Table 7-1. Sample sizes required for accuracy of better than 5 percentage points in a categorical variable. Sample Size Required for Category Proportion: Total Number of Tenants at Airport p = 0.5 p = 0.25 p = 0.1 30 28 27 25 50 44 43 37 75 63 60 49 100 80 74 58 200 132 118 82 400 196 168 103 Note: Assumes random sampling. 29 For proportions (p) greater than 0.5, the required sample size is the same as for the proportion 1 – p. For example, for a proportion p = 0.75, the required sample size is the same as for p = 0.25.

128 Guidebook for Conducting Airport User Surveys proportion is lower. Further examples for determining the required sample size are provided in Appendix B. Depending on the purpose of the survey, it may be appropriate to sample a greater proportion of some subgroups than others. If a similar level of accuracy is desired for each subgroup, a higher sampling fraction would be used for smaller subgroups or for subgroups where the variance of the characteristic of interest, say economic impact, is greatest. Thus, subgroups with a high economic impact such as airlines would have a higher sampling fraction than owners of individual private aircraft. For economic impact surveys, the treatment of non-responses can significantly affect the results. The economic impact of individual tenants varies greatly, particularly between groups of tenants. Use of only the data from responding tenants would lead to underestimation of the total impact, and expanding the sample results on the basis of the average response for each group of tenants could result in large errors, depending on the tenants that did not respond. Every effort should be made to obtain some response from all tenants, even if only basic information—such as the num- ber of employees or gross revenues—is obtained from some organizations. This information can then be used to develop weights to expand the data received from each group of tenants to repre- sent the total group. If no information can be obtained from some tenants, the airport will always know something about the size of each tenant. In many cases they will know the number of employ- ees from issuing security badges. They will also know the size of the area leased and in many cases will have traffic or revenue data from required reports or concession fees. This information can be used to estimate the relevant economic measures for the non-respondents, based on the data from the responding tenants. Mystery shopper surveys will generally be conducted on all food and beverage and retail con- cessionaire tenants (see Section 7.7). 7.4 Questionnaire Wording and Length Tenant feedback and economic impact surveys should take less than 20 minutes, preferably much less, including finding the data. There is always a temptation to ask for more detailed infor- mation than is required; however, this can significantly affect the response rate. The survey should be restricted to requests for information that a knowledgeable person within the orga- nization will know immediately or be able to obtain easily. As discussed in Section 4.3, deter- mine in advance exactly what information is required and how the response from each question is going to be used. A sample tenant questionnaire is provided in Appendix I. 7.5 Measures to Obtain Adequate Response A high response rate is important for obtaining accurate, unbiased results. The following measures could be taken to improve the response rate: • Include the name of the company or organization conducting the survey and the survey sponsor in the introduction to the survey. The introduction should clearly state the purpose and how the results will be used, highlighting aspects that could benefit the respondents as airport tenants. • Make it easy for the tenants to respond. Use of email to distribute the questionnaire allows it to be easily forwarded to the appropriate person. Try to keep the requested data to things the respondent will know immediately or be able to find out easily.

Tenant Surveys 129 • Alleviate confidentiality concerns. The survey should be conducted by a third party with an assurance that all responses will be kept confidential and information will only be released in aggregate form so that responses from individual tenants cannot be identified. It may be use- ful to include some information about the survey organization to show that it is a respected organization with much to lose by breaking the confidentiality commitment. • If conducting a Web-based survey, use software that allows respondents to save partially com- pleted responses and complete them at a later time. • Follow up with non-responding tenants. Initially this can take the form of email reminders, but should include a telephone call if nothing has been received after two or three reminders. The telephone call would be a good opportunity to attempt to obtain at least an estimate of the number of employees or gross revenue for those declining to complete the survey. • Give recipients adequate time to respond. The deadline for responding should allow sufficient time for people on vacation, leave, or work-related trips to respond when they return. Two weeks is a reasonable period, increased to three weeks in July and August or around Christmas (although it is preferable not to schedule tenant surveys during these periods if possible). Reminders should be sent weekly, with a final reminder within 48 hours of the survey’s closing time. 7.6 Survey Budget Tenant surveys are relatively inexpensive to conduct using an Internet approach, but costs can increase significantly if initial response rates are low and many telephone follow-up calls are required. The budget for tenant surveys will include the following components: survey design and plan- ning, including development of tenant lists; questionnaire design, testing, loading onto a survey website; follow-up of non-respondents; checking and analysis of responses; and reporting and presentation of findings. (See Chapter 2 for a more detailed discussion of budgeting.) Costs will vary depending on the collection method, sampling method, size of the sample (less so for Web-based surveys), difficulty in developing tenant lists, and the initial response rate. How- ever, for a typical Web-based survey with 10 to 20 questions where email addresses can be pro- vided by the airport and telephone follow-up is only required for 25% of tenants, costs might be approximately: • $8,000 for a survey of 50 tenants. • $12,000 for a survey of 100 tenants. • $15,000 for a survey of 150 tenants. • $18,000 for a survey of 200 tenants. If the tenant survey is conducted on a regular basis, some items—such as survey questionnaire development and implementing it on the website—may entail minimal costs after the first sur- vey. The start-up costs that could be avoided in subsequent surveys are typically in the $2,000 to $5,000 range. Costs could be less if the initial response rate is very high or if non-respondents are followed up via email only. However, if response rates are low, the confidence in the results will be reduced and there will be additional costs for the analysis of non-responses. 7.7 Mystery Shopper A mystery shopper survey is a very different type of tenant survey from the tenant feedback and economic impact surveys discussed above.

A mystery shopper survey is part of an overall program to assess the performance of concessions and identify deficient areas and specific improvements that could be made. The purpose of a mys- tery shopper survey is to anonymously evaluate customer service, operations, employee integrity, merchandising, and product quality of airport concessionaires. Product returns are also conducted where possible. Each concession is “shopped” at least twice—once during a busy period and once during a slower period. By shopping the store at least twice at different times, the survey staff will likely encounter different employees. Mystery shopper surveys are usually conducted on an ongo- ing basis every three or six months. The types of information collected on each concession include the following: • Staff attributes—such as the greeting and assistance given, friendliness, courteousness and appearance, helpfulness, knowledge of the store’s products, and whether a uniform or name badge was worn. • Product selection—range, quality, display, items available and easy to find, prices clearly marked either on or in front of items. • Purchase/checkout—length of queue, time in queue and being served, whether cashier stated purchase total and offered a bag and receipt, and whether transaction was accurate. • Returns—whether returns are accepted, length of queue, duration of transaction, staff knowl- edge of the return process, accuracy of refund, whether customer was asked why the product was returned, and whether staff suggested another product where appropriate. The mystery shoppers assess the concession while making a purchase or return, and immedi- ately after leaving the store answer a series of questions related to each attribute, with most attri- butes simply requiring a numerical ranking, a yes/no answer, or time duration. Price comparisons can also be made with similar off-airport stores or with other airports. Usually more than one shopper is used and each assesses a concession once. Some of the ques- tions call for judgment (e.g., “friendliness of greeting”), and using a number of survey staff reduces dependence on a single opinion. It is essential that survey staff go through a training session with the supervisor, so that they fully understand exactly what they should be looking for when rating an attribute and there will be some consistency in the results. The shoppers also should provide comments or explanations where relevant, especially if poor ratings are given. These comments will be particularly useful when identifying specific improvements to be made. When analyzing the results, both the average score and any variation between scores should be considered. One low score may not be cause for concern, but if a number of shoppers give a low score, or other related attributes also receive low scores, further examination should be con- ducted. These concessions should be surveyed again to determine if the poor rating is a trend or an anomaly. The main advantage of mystery shopper surveys is that they provide assessments of the detailed attributes of concessions. Respondents to air passenger or employee surveys generally will not be able to recall these attributes in such detail or will not be able to answer this type of question due to time constraints. Mystery shopper surveys can be used in conjunction with pas- senger satisfaction surveys to identify areas with deficiencies as well as specific improvements that could be made. The major disadvantages of mystery shopper surveys are that a very limited number of shop- per surveys of a concession are made, many of the assessments rely on judgment, and the rating is dependent on the person making the assessment. Thus if there is significant variation in ser- vice quality over time, or in how the mystery shoppers rate the same attribute, the resulting rat- ings may not be accurate. Results can be skewed if an infrequent event occurs during a particular survey. The variation between mystery shoppers can be reduced with good training and by select- ing survey staff that are typical of the profile of airport concession users. 130 Guidebook for Conducting Airport User Surveys

When using the results of a mystery shopper survey, it is important to take these potential lim- itations into consideration. 7.8 Summary Surveys of tenants are usually conducted to collect information for studies of the economic impact of airports or to determine tenant satisfaction with the airport as landlord. The best method for conducting these types of surveys is to send invitations to participate by email with a link to a Web-based survey. All tenants will usually be surveyed using this approach, with follow-up emails and telephone calls to those not responding. The surveys should be conducted by respected third parties with assurances that the data will remain confidential. Tenant surveys are relatively inexpensive to conduct compared to other airport user surveys. Mystery shopper surveys are a different form of tenant survey. Their purpose is to anonymously collect detailed information for assessing the performance of airport concessions. This informa- tion is used to identify each concession’s strengths and deficiencies as well as specific improvements that could be made. Tenant Surveys 131