Guidebook on Conducting Airport User Surveys and Other Customer Research (2021)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Conducting Customer Research P A R T   I Introduction Airport managers undertake research to address a wide variety of questions about airport customers and other members of the airport community, and a broad assortment of research methods is available for these purposes. Some research concepts are applicable to nearly all of these customer research efforts. The eight chapters in Part I discuss topics that are applicable to most types of research studies. These chapters are summarized briefly in the following. Chapter 2: Customer Research Methods Chapter 2 provides an overview of quantitative and qualitative research methods, considers how these two broad approaches differ, and discusses the advantages and disadvantages of each methodology. Chapter 3: Research Planning Chapter 3 outlines the steps that are involved in planning a research study. They include defining the research purpose, assembling a planning team, selecting the appropriate research method, planning the research schedule, and developing a project budget. Chapter 4: Statistical Concepts Chapter 4 presents and explains the concepts of sampling and statistical accuracy that need to be considered in the design and implementation of surveys. Included are the difference between a census and a survey, considerations of statistical accuracy, sampling methods, sample size calculations, and the weighting of survey data. Chapter 5: Survey Design and Implementation Chapter 5 describes the process of designing and implementing a survey after the initial steps discussed in Chapter 3 have been completed. Topics include defining the population of interest, formulating a sampling plan, designing a survey questionnaire, collecting the data, and training interviewers. Chapter 6: Qualitative Methods Chapter 6 discusses what are referred to as “qualitative research methods,” namely methods that are not based on numbers or concepts of statistical accuracy. It first considers the advantages

and disadvantages of these methods and then describes four key study types: focus groups, individual depth interviews, observation, and mystery shopping. Chapter 7: Monitoring and Enhancing Customer Service Chapter 7 explores the concept of monitoring and enhancing customer service at an airport. It presents and discusses three major approaches: customer relationship management (CRM), experience management (XM), and customer experience (CX) programs. Other topics addressed in this chapter include working with feedback loops, prioritizing experience factors, using dash- boards to present research findings, and utilizing research results. Chapter 8: Targeted Studies of Specific Issues Chapter 8 considers the design and implementation of targeted studies that address specific issues identified in ongoing monitoring studies. It discusses the types of research questions that may arise, the sources of these questions, cautions relative to decisions about whether to conduct additional research, planning of a targeted study, possible study types, and utilization of the research results. Chapter 9: New and Developing Data-Collection Techniques Chapter 9 explores a variety of new and developing data-collection techniques that have emerged since the publication of ACRP Report 26, including the use of social media, Wi-Fi micro-surveys, real-time customer monitoring, and the acquisition of location data. 10 Guidebook for Conducting Airport User Surveys and Other Customer Research

Customer Research Methods 2.1 Introduction This chapter provides a brief overview of the range of customer research methods available. Each of these methods is discussed in more detail in the subsequent chapters, as indicated in the following sections. The intent of the descriptions in this chapter is two-fold. First, the chapter provides a high- level overview of the range of customer research methods in use at different airports, although not necessarily at any one airport. Second, the descriptions offer users of the guidebook a means to help decide which of the different research methods may be most relevant to the questions or issues that they face and direct them to the relevant chapters of the guidebook for more detailed guidance and examples of the use of those methods. Airport customer research methods fall into two broad categories: quantitative and qualita- tive. The former generally gives numerical results, such as satisfaction scores or the percentage of customers with a particular characteristic, while the latter gives more descriptive results, such as the reasons why customers found a particular facility or service unsatisfactory. The difference between these two approaches is discussed in more detail in Chapter 3: Research Planning. Airport user surveys form the primary quantitative customer research method at most airports. Sections 2.2 and 2.3 introduce the different types of airport user surveys. General considerations in planning airport user surveys, as well as other customer research studies, are discussed in Chapter 3, while guidance on survey design and implementation is provided in Chapter 5. A more detailed discussion of the role of surveys in customer research, as well as airport planning and development, survey concepts, and the principal survey types and methods, is provided in the introduction to Part II of this guidebook, which contains chapters addressing each type of survey. In addition to surveys, airports commonly undertake other ongoing monitoring activities, including programs to track customer feedback, as discussed in Section 2.4, and analysis of social media posts, discussed in Section 2.5. These monitoring activities will typically have both a quantitative and qualitative focus. Qualitative customer research methods include focus groups and observation, as described in Sections 2.6 and 2.7 and discussed in more detail in Chapter 6: Qualitative Methods. The near universal use of smartphones by airport users has created opportunities for con- ducting research into airport users by tracking their locations using signals from their smart- phones, as described in Section 2.8 and in more detail in Chapter 9. Finally, airport user research can involve outreach to airport stakeholders and the larger community served by the airport, as discussed in Section 2.9. C H A P T E R   2 11  

12 Guidebook for Conducting Airport User Surveys and Other Customer Research In any quantitative application of customer research methods, but especially of airport user and other surveys, the issue of statistical confidence arises. Are the results suggested by the analysis statistically valid, or could they just have occurred from chance? A related issue that arises in planning surveys (and to an extent in other customer research methods) is how large a sample size is necessary to produce results of a desired level of statistical confidence. This turns out to be a non-trivial question, and those involved in planning, analyzing, and present- ing the results of such research need to have at least a basic familiarity with statistical terms and concepts. Chapter 4: Statistical Concepts is intended to provide an introduction for those with no training in statistics or who would like to refresh their understanding of the topic, and to provide a convenient reference for those with greater familiarity. 2.2 Air Passenger Surveys Surveys of air passengers are the most well-established and probably still the most widely used customer research method. This reflects the fact that if an airport is interested in questions about the perceptions and characteristics of the airport users, the most obvious way to answer these questions is to ask the users directly. Air passenger surveys fall into two broad categories: those addressing customer satisfaction and those designed to gather information on air traveler characteristics. Of course, sometimes both aspects are addressed in a single survey. Indeed, customer satisfaction surveys often ask questions about respondent characteristics to analyze how satisfaction varies with the charac- teristics of the respondents. Whatever the motivation for conducting an air passenger survey, common issues arise that need to be considered. These make up the steps involved in survey planning discussed in more detail in Chapter 3 and issues of survey design, implementation, conduct, analysis, and docu- mentation described in more detail in Chapter 5. Surveys primarily designed to gather information on air passengers also sometimes gather information on greeters and well-wishers, either by asking departing air passengers about well-wishers who accompanied them to the airport or by directly asking greeters and well- wishers who happen to be intercepted in the course of an air passenger survey. Surveys that only target greeters and well-wishers are less common. Issues involved in surveying greeters and well-wishers are discussed in more detail in Chapter 10: Air Passenger Surveys (Section 10.11). It should be noted that any given individual may make multiple air trips through an airport in the course of a year. While it is quite unlikely that any one individual will be surveyed more than once in a given survey, particularly a survey of relatively short duration, in the event that this does occur, they will be counted as a different air passenger each time, since the circumstances of their air trip will be different each time. 2.2.1 Customer Satisfaction For many years, airports have conducted surveys to measure and track customer satisfaction. These are commonly conducted fairly frequently, such as quarterly or annually, with a relatively small sample size (several hundred to perhaps a thousand respondents), since it is felt that most questions apply to all travelers, and a larger sample is not needed. Since the primary motiva- tion for conducting these surveys is to track changes in satisfaction over time, the surveys are performed fairly frequently so that appropriate actions can be taken in a timely way to address any issues uncovered by the survey results. The required size of the survey sample will depend on both the desired accuracy of the survey results and the extent to which results are desired for

Customer Research Methods 13 subsets of the overall customer population being surveyed. Issues of how to determine required sample sizes are discussed in Chapter 4. More recently, interest has been growing in being able to benchmark the customer satis- faction at an airport to that at peer airports, as well as to use the results of customer satisfac- tion surveys at different airports to identify actions that airports can take to improve their own customer satisfaction. This in turn has led to an interest in standardizing the survey questions and methodology. In response to this interest, Airports Council International (ACI) has established the Airport Service Quality (ASQ) program, through which regular surveys are conducted, typically quarterly, at participating airports using a standard survey question- naire and methodology. Results are shared with the participating airports. A number of other organizations also conduct surveys among users of different airports worldwide and use the results to develop airport customer satisfaction rankings that they publish. Perhaps the best known of such surveys in the United States is the North American Airport Satisfaction Study performed by JD Power Associates using an online survey. Globally, the World Airport Survey conducted by the United Kingdom research firm Skytrax claims to be the world’s largest airport customer satisfaction survey. These organizations are generally doing this for commercial reasons by making the details of the survey responses for a given airport available for purchase by that airport. Surveys of this type are known in the survey profession as syndicated research studies. Customer satisfaction surveys, including the ASQ program, are discussed in more detail in Chapter 10. Section 10.5: Issues with Online Surveys includes a discussion (Section 10.5.1) on the use of survey panels, which are generally used in syndicated research studies. 2.2.2 Characteristics of Air Travel Parties Surveys to obtain data on characteristics of air travel parties for airport planning, design, and management purposes, as well as transportation planning at a local, regional, statewide, or even national level, are a well-established, indeed essential, airport user research technique. Such surveys are used to obtain data that are not easily obtained or may not be possible to obtain through other means, such as trip purpose, household income, traveler demographics, and details of airport ground access or egress travel. In addition to factual information about the survey respondents, these surveys sometimes include questions about traveler perceptions and likely use of proposed or potential new facilities and services that the airport or other agencies may be considering. Questions of this sort raise important methodological issues that need to be taken into account if the results are to be considered reliable. These are discussed in more detail in Chapters 5 and 10. Whereas surveys of customer satisfaction reflect the individual opinions of the survey respondents, surveys of characteristics of air travel parties are designed to obtain data on all members of the air travel party. Care is needed to avoid double counting in cases where survey responses are obtained from multiple members of the same air travel party (such as when survey questionnaires are distributed to all adult passengers waiting in an airline gate lounge). Similarly, care is needed in interpreting responses to questions about individual characteristics, such as age, which may not apply to other members of the air travel party. Because of the potential use of the data collected by such surveys for regional or statewide transportation planning purposes, local, regional, or state transportation planning agencies sometimes undertake these surveys, usually in cooperation with the airports involved. Another reason for conducting air passenger surveys is to gather information, for use by tourism or convention agencies, on visitor spending in the region for use as input to economic

14 Guidebook for Conducting Airport User Surveys and Other Customer Research impact studies or information on visitors’ activities while they are in the region. Although visitor spending and activities while in the region are distinct from spending on air travel, airports may be a convenient location to conduct such surveys. Typically, surveys designed to collect information on characteristics of air travel parties have been undertaken less frequently than customer satisfaction surveys since there appears to have been a belief that these characteristics tend to be fairly stable over time. In addition, they are often conducted as part of a planning study, such as an update to an airport master plan or regional transportation plan, which typically only occur every few years. However, there is a growing recognition that characteristics of air travel parties may in fact be changing over time, and therefore, airports may need to perform such surveys more frequently, similar to customer satisfaction surveys, to identify trends and changes in a timely way. A good example of this issue would be airports that undertook groundside planning studies based on survey data that were collected prior to the advent of ride-hailing services (e.g., Lyft and Uber) and as a result failed to fully anticipate the resulting changes in ground access mode use. 2.3 Employee, Concessionaire, Tenant, and Other Surveys Employee, concessionaire, tenant, and other surveys are conducted for reasons similar to air passenger surveys. As well as being concerned about air passenger satisfaction, airports should care about the job satisfaction of their own employees and the views of concessionaires and other tenants of the airport management and its administrative processes. In addition, airport access and egress travel by the personnel of all the organizations operating at the airport contributes to the traffic generated by the airport and may influence the viability of ground transportation facilities and services as well as require provision of specific facilities such as employee parking lots. Airports sometimes undertake surveys of other stakeholders or the broader community, including residents of the area served by the airport, local businesses, or organizations involved in handling air cargo at the airport. Of course, these other types of airport users may also make air trips, and a given individual may use the airport in more than one capacity. However, each type of survey collects informa- tion from its respondents when those respondents are acting in a particular capacity. 2.3.1 Employees In addition to the employees of the airport itself, the wide range of other organizations operating at the airport, including airlines, government agencies, contractors, concessionaires, and other airport tenants, also have employees who work at the airport. In the case of airlines, flight and cabin crew frequently make ground trips to and from the airport even if they are based elsewhere. Larger airports may host specialized facilities, such as airline maintenance bases, which can account for a significant number of employees. Although airports may be primarily concerned about the job satisfaction of their own employees, in some cases they may care about the job satisfaction of employees of other orga- nizations at the airport as well, particularly those in customer-facing positions such as airline counter and gate agents and concession employees. Although the airport is not directly respon- sible for those employees, their attitudes and customer engagement can influence the overall air passenger satisfaction with the airport experience, often to a greater extent than the airport’s own employees.

Customer Research Methods 15 In recent years, the ASQ program has added an airport employee survey to the range of surveys that are offered to airports. Conducting surveys of airport employees, particularly those of other organizations, can be more challenging than conducting surveys of air passengers due to dispersed job locations and the different shift patterns employees work to provide coverage for their duties up to 24 hours per day and seven days per week. There are also issues of the willingness of employees of other organizations to participate in a survey sponsored by the airport rather than their own employer and how to contact potential survey participants. The issues that arise in performing employee surveys are discussed in more detail in Chapter 11: Employee Surveys. 2.3.2 Concessionaires and Other Tenants Concessionaire and other tenant surveys refer to surveys of those organizations rather than surveys of their employees as described in the previous subsection. Surveys of these organi- zations typically address concessionaire and tenant satisfaction with their relations with the airport management and its procedures, including the availability of adequate facilities and support services as well as services needed by their own employees. The fact that these orga- nizations are generally in a contractual relationship with the airport can make the design and conduct of such surveys somewhat sensitive and challenging. A class of organization included under “other tenants” is government agencies, such as the FAA, TSA, and U.S. Customs and Border Protection, that have employees based at the airport. These may not strictly be tenants since they may own their own facilities or be provided facilities at no cost, but there will generally be some form of agreement between the agency and the air- port. However, the relationship between the airport and these agencies is different from other tenants in that, with some limited exceptions, the airport does not have the option of ending its contractual relationship and seeking another organization to perform the same function. A detailed discussion of the issues that arise in performing these various types of surveys is provided in Chapter 12: Concessionaire and Other Tenant Surveys. 2.3.3 Other Airport Stakeholders Airports sometimes undertake surveys of area residents or the local business community. The motivation for such surveys can range from a desire to better understand how the airport is viewed by the local community to gathering information on the use by local residents or businesses of other airports that serve the region. Surveys of area residents are discussed in Chapter 13, and surveys of local businesses are discussed in Chapter 14. Another group of airport stakeholders is the various organizations involved in the air cargo sector, including airlines, integrated carriers (such as FedEx and UPS), freight forwarders, air cargo road haulers, and major shippers. Planning airport facilities to meet the needs of these organizations and the air cargo traffic at the airport requires information on air cargo activity and the air cargo market that is not routinely reported and can only be obtained by conducting surveys of the firms involved. Air cargo surveys are discussed in Chapter 15. 2.4 Tracking Customer Feedback Many airports maintain customer research programs that track and evaluate customer feedback from a variety of sources in order to understand passengers better, examine internal processes, and assess customer satisfaction. Some of these sources are electronic, and some are

16 Guidebook for Conducting Airport User Surveys and Other Customer Research manual. They can include monitoring comments on social media and programs to record and analyze complaints as well as comments received via email, telephone, or in person to airport staff or volunteers. Customer feedback programs have several common components, including an emphasis on action; a direct link between the feedback and the organization’s response; an emphasis on understanding customer motivations, journeys, and pain points; and a process for closing the loop with concrete actions. Loops are typically referred to as outer, which addresses the customer population as a whole, and inner, which responds to the individual customer. One weakness of many customer experience programs is that they focus on what to do with the feedback rather than the relative strengths and weaknesses of different sources of customer feedback, which can present a dilemma on how to use conflicting information. For example, comments on social media or made in person to airport staff or volunteers may be positive or negative, while airport users are unlikely to take the trouble to send an email or make a telephone call to report a positive experience, so feedback from these latter methods generally reflects a negative experience. The answer to this dilemma is to make use of feedback from different methods in ways that reflect their strengths, as discussed further in Chapter 7: Monitoring and Enhancing Customer Service. One type of research that is critical for enhancing customer service is regular monitoring of service quality. Both airport-designed and standardized surveys, such as the ASQ surveys, are used to assess and identify priorities for attention and track the effectiveness of manage- ment actions. Overall satisfaction is influenced by many experience factors. Airports can determine what these factors are by using statistical analysis techniques, by asking customers to identify specific factors that were most important to their overall satisfaction, or by conducting targeted studies to better understand factors affecting metrics that received ratings that were lower than acceptable. Some airports have developed sophisticated dashboards and other graphics that collect most or all of their customer satisfaction measurements in a single presentation. One airport that provided one of the case studies undertaken during the project integrates 10 separate metrics into a system designed to understand the customer experience. Another case study airport has developed a dashboard that contains two types of feedback: solicited or voiced feedback and unsolicited feedback. Solicited feedback includes the results from ASQ surveys as well as instant feedback from a separate survey that provides trend analysis and text analysis in real time. Unsolicited feedback is processed through a comments/inquiry management system that consolidates feedback from phone calls, letters, the airport’s website, on-site comments, and emails and is combined with data from an analytical tool that is used to capture customer feelings and expectations from social media. Details of both these dashboards are discussed in Chapter 7. 2.5 Social Media Data Analysis of social media posts is an important component of many airports’ customer research programs. By analyzing social media posts, airports can identify topics about the airport that are being mentioned and determine whether these topics are being talked about in positive, negative, or neutral terms. Airports have used social media posts to help identify problems that should be addressed as well as to identify outstanding employees who deserve recogni- tion. More broadly, airports have used social media platforms to engage with customers and

Customer Research Methods 17 promote the airport. Airports use social media as one of several platforms for disseminating customer surveys. These and other examples of how airports are using social media for customer research are covered in Chapter 9. 2.6 Targeted Studies of Specific Issues In addition to the types of ongoing or periodic surveys discussed in Sections 2.2 and 2.3 and the monitoring activities described in Sections 2.4 and 2.5, airports often identify issues that suggest the need for more targeted studies. These issues can arise when questions for which there are no available answers are raised by managers, when one or more results of the monitor- ing studies indicate customer dissatisfaction, or when negative customer feedback is received from less-structured sources such as social media or comment cards. Additional, targeted infor- mation then becomes of importance and potential use. When targeted studies are being designed, it is particularly important to determine what information will be needed to address the problems or questions that have been identified. The information that needs to emerge at the back end of the study, when the results are received, will be driven by the research questions asked at the front end. Good research questions will identify high-priority information needs, will yield answers that address these needs, and will translate directly into appropriate management action. Whether the goal is enhancing customer satisfaction levels, increasing revenue, or improving planning, truly actionable information needs to be the ultimate goal of any targeted research. Chapter 8: Targeted Studies of Specific Issues discusses tools for undertaking this process. Targeted studies can also be conducted using qualitative methods. The most popular of these methods is the focus group, in which a small group of members of the target audience (passengers, employees, area residents, or other constituencies of interest) are brought together in a specialized room to discuss an issue in depth. Typically, focus groups contain eight to 10 people, last about 2 hours, and yield a transcript of some 40 to 50 pages of verbatim narrative that is analyzed using qualitative techniques. A detailed discussion of focus groups can be found in Chapter 6. Another qualitative research method that can be used for targeted studies is in-depth inter- views, which are typically undertaken to obtain information from representatives of stakeholder organizations or others with specialized knowledge. The use of in-depth interviews is discussed in more detail in Chapter 6. 2.7 Observation Observation research quite literally means watching what people do without communicating with them. It can be largely qualitative or exploratory, as when researchers are trying to figure out possible behaviors in a given situation, or it can be somewhat quantitative, when behaviors are being tallied and recorded. Many observation studies use a hybrid of these two approaches. Observations can be made by people or electronically, such as by tracking user cell phones, and both of these strategies are used in airports. The topic of this section is human observation. Electronic tracking is largely quantitative, although qualitative information may sometimes be inferred from the observed behavior and is discussed in the following section. Further details can be found in Chapter 6. A key advantage of observation research is that it does not rely on self-reported behavior and is thus reliable given a well-trained and attentive observer. There are also no issues with respect

18 Guidebook for Conducting Airport User Surveys and Other Customer Research to cooperation; because people are not being asked to participate, they are not offered the option to refuse. The major disadvantage of observation research is that it is limited to behavior; motivations and attitudes are not captured. Thus, if researchers need to know why people do what they do or how they feel about something, this will not be an appropriate technique. Often, primarily observational studies also include an interactive component. Perhaps the most common use of observation at airports is assessing wayfinding. Observers either follow passengers on their trips from, for example, security screening to a particular concourse or gate, or observe them at strategic decision points along the path. Passengers may then be inter- viewed at decision points that are viewed as being potentially problematic. Photographs may also be used to enhance the reporting, such as by showing unclear signage. Rather than have observers watch the subjects of a study directly, the subjects’ behavior can be recorded using video or time-lapse photography and the resulting recordings watched or analyzed using automated techniques. This can have a number of advantages, one of which is that it can collect data over a prolonged period of time or at inconvenient times (such as at night) without needing to have human observers present for the entire time. The recording can then be watched later at a more convenient time or in a more convenient environment. This approach can also allow interesting or complex situations to be replayed for closer study and may be perceived by the subjects as less intrusive than being watched by human observers. Recent advances in image processing techniques can allow situations of interest to be identified in a prolonged video or time-lapse recording without requiring a human observer to spend time watching the entire recording. However, there are important privacy considerations that need to be considered when filming subjects without their explicit consent. A particular type of observation study that airports may conduct of its concessionaires is termed “mystery shopping,” in which unidentified staff attempt to purchase goods or services at airport concessions and report on their experience, as discussed further in Chapter 6. 2.8 User Location Tracking Airports use a variety of user location tracking data from cell phones, beacons, and Wi-Fi. Cell-phone data are one of the most common sources, since the location of each cell-phone handset is tracked continuously by the cell-phone service providers. Cell-phone data have been used to track passengers as they travel through the airport and provide information on TSA queue lengths and wait times and concessions visited by customers. It should be noted that all the user-location tracking techniques involve communication with a user’s smartphone, but there are differences in how this communication is done and how the location tracking data are made available. Some airports have been creative in using location tracking data for targeted marketing. For example, when cell-phone owners who live within a smaller airport’s catchment area cross the geofence around a larger competitor airport, they are sent a push advertisement by the cell-phone provider or a third party providing this service. This helps increase aware- ness that the smaller airport could be an alternative option for future flights. These and other examples of how user-location tracking has been used by airports are covered in Chapter 9. 2.9 Stakeholder and Community Outreach At the beginning of a study, researchers often need to coordinate with internal stakeholders within the airport organization about the study’s content to make sure everything that will be included is something that needs to be known rather than simply being nice to know. Reaching

Customer Research Methods 19 out to these stakeholders at the conclusion of a study for input into the implications of the results as well as the appropriate actions to be taken is also customary and could be considered a best practice. Outreach to external stakeholders, such as airlines, government agencies, and local business organizations, is less common but may be helpful in a variety of circumstances. Examples include: • Making presentations and conducting telephone interviews as well as follow-up surveys in connection with a branding project, • Gathering data for economic analysis and impact studies, • Obtaining input for master planning efforts, • Determining new markets that should be targeted as part of air service development, • Surveying stakeholders about the airport’s corporate reputation, • Holding dialogues with stakeholders to determine where the airport’s and concessionaires’ services need to be improved, • In order to help attract new businesses, working with economic development organizations to understand who is coming to the area, and • Learning what types of research activities stakeholders have undertaken and the results of this research. Outreach to the wider community, such as residents or individual businesses in the area served by the airport, is considerably less frequent than outreach to external stakeholder organi- zations. However, it does occur, often in connection with airport planning activities. Examples include public forums in support of the airport’s master planning process, public forums to discuss aircraft noise issues, and community forums about airport facilities. Some airports have also established standing advisory committees.

20 3.1 Introduction Thorough planning is essential to the success of any research effort. Unfortunately, this task is often allotted insufficient time and attention, resulting in inadequate or inaccurate data. This chapter describes the factors that need to be considered in planning a successful research project. These factors are as follows: • The purpose of the study • The planning team • The involvement of external organizations • The research method • The budget • The schedule 3.2 Research Purpose The importance of clearly defining the purpose of a research project from the outset cannot be overstated. In essence, this is the answer to the question, “Why are we doing this?” If the replies from the people commissioning the research are vague—“The information would be interesting,” “It would be nice to know,” or “We need to understand our customers better,” for example—the best next step is to stop. To be useful, research should yield results that are actionable. It is often helpful to begin a research effort by considering the end or result of the process. This approach suggests that an airport first consider the actions it wants to take and the decisions it wants to make on the basis of the results, determine the information that will be required to take those actions or make those decisions, and only then start to consider the target population and questions that will provide the necessary information. This approach also helps to prevent errors of both omission and commission, which are often not discovered until the end of the research process. An error of omission is identified when someone asks, “Why didn’t we ask that question?” and when the unasked question represents information necessary for effective use of the results. A good example (which actually occurred in an air passenger survey in the San Francisco Bay Area) would be failing to ask how long departing air passengers residing in the region served by the airport would be away on their trips. The survey data were intended to be used for airport ground access studies, and the air trip duration of course determines the cost of parking a car while away on the trip, which is an important consideration in the modeling of ground access mode choice. Research Planning C H A P T E R   3

Research Planning 21 An error of commission has occurred when the research does not obtain the information that the designer intended. An example would be asking air passengers, “Where did you begin your trip to the air- port today?” with the goal of obtaining trip origin location information for airport access trips. Unless an exhaustive list of possible response options is provided, it is unclear what the question might mean by the word “where.” Typically, the purpose or purposes of a research study will be outlined using goals and objectives. Goals are broad, general statements of the information the study is intended to obtain; objectives are more specific. For example, the goal might be to gather information on air passenger ground-transportation use; the objectives might be to determine where passengers began their trip to the airport, which ground access modes they used, and why they chose those particular modes. Well-defined goals and objectives will pro- vide the necessary guidance for the development of useful and appropriately worded questions. 3.3 The Planning Team A strong leader or project manager will be needed to help the research team clearly define and articulate the purpose of a study, oversee its design, determine how and when it will be conducted, ensure successful implementation, and facilitate the appropriate use of the results. Although planning by committee can be awkward and time-consuming, it is essential to involve those who will need to approve key decisions on issues such as scope and budget as well as everyone who will be involved in actually using the findings. It may also be helpful to include representatives of external organizations who could benefit from the results, and who might, therefore, provide important input or perhaps even assistance. It is also important to consider the inclusion of those, such as analysts and modelers, who will work to turn the data that emerge from a research study into meaningful information. Finally, if the study involves a survey that will be conducted by intercepting passengers, it may be necessary to coordinate with those planning other such studies at the same time. Having two surveys conducted at the same airport at the same time can be seriously problematic. Exactly how many personnel are required will depend on the size and complexity of the study as well as the availability of personnel. It is also possible that in some cases, one person may have the necessary expertise to play multiple roles. Depending on the structure of the organization and the nature of the research, team members may come from the same department or from various departments. Either way, it is important that all staff and managers involved understand the importance of the research and the role of each team member. The process of coordination with other departments and external agencies should include soliciting comments on the scope and content of the planned study. These comments can be important to the success of the research because they might reveal one or more of the following: • Some of the information being sought already exists within the organization or in other organizations. Best Practice “We look at it quarterly because our trackers are always on. We meet with our agency every quarter to make sure that we are reflecting exactly what the passenger journey, or the passenger experience, is so that we are capturing the most important things.” —Research participant Research Success “This year’s strategic focus was the end-to-end passenger journey . . . what is most important to our customer. . . . We identified the airports in the ASQ that perform better than [subject air- port] in any one of three categories. . . . I got to lead the effort in choosing a team to go out to these various airports. That was a new initiative for us, to get outside of an airport and find an industry leader and see what they’re doing and see if there’s any way we can bring back what they do.” —Research participant

22 Guidebook for Conducting Airport User Surveys and Other Customer Research • Other studies are scheduled to take place in the future that might conflict with the planned study or cause an undesirable respondent burden on smaller groups of airport users. • Inclusion of additional questions that would not conflict with the planned research design but would produce substantial added value might be desirable. • Structuring or actual wording of questions to allow comparisons of the results with those from other research efforts might be helpful. 3.4 Research Method Perhaps the most important preliminary step in conducting a research project involves the selection of the appropriate method. There are two fundamental types of methods: quantitative and qualitative. Among the quantitative methods, by far the most prominent is the survey, and much of this guidebook is devoted to planning, designing, and implementing various types of surveys. Among the qualitative methods, which are discussed in Chapter 6, probably the most common is the focus group. Quantitative methods—methods that yield predominantly numeric information—are the most suitable when the primary goal of the research is to obtain answers to “what” or “how many” questions. Examples might include use of ground access modes, trip purpose, amount of time spent at the airport, and numbers and types of concession purchases. Qualitative methods, which yield largely narrative information, are most appropriate when the study is focused on asking “how” or “why” questions. Examples include how passengers found out about ground access options, why they selected a particular mode, reasons for arriving at the airport when they did, and how they feel about the concession options. Perhaps needless to say, research purposes will often include both quantitative and qualitative areas of inquiry. The optimal approach to such a situation is to give serious consideration to including both types of research in the study if this is financially feasible and if time permits. In this case, the main decision is which should come first, the qualita- tive study or quantitative one? Generally, qualitative research will precede a quantitative study because the former often will reveal areas of inquiry that a quantitative study should be used to measure more precisely. Unfortunately, how- ever, there are also instances in which quantitative results prompt the question, “Why did they say that?” Whether this will lead to yet further qualitative research will be a function of the importance of the answer and the availability of resources. In cases where it is not possible, for whatever reason, to conduct both types of studies, planners need to give careful consideration to the type of information that will be most important for decision making. This is not necessarily an easy choice to make, and it is therefore critical to involve in the selection process everyone who will be responsible for acting on the research results. Finally, when making the decision about method, it is also important to consider that quantitative information, while often not particularly helpful in understanding why people do what they do, is methodologically more definitive than qualitative information. If the basic rules of sampling are followed (see Chapter 4), quantitative results can be deemed accurate within a clearly defined margin of error. Qualitative results, on the other hand, will be less definitive because they are based on relatively small sample sizes. Best Practice “A lot of times, the way I approach research, I like to do subsequent follow-up research. My plan is to develop some focus groups based on the survey results we get to allow us to dig deeper into some of the questions. Once I present findings to team, I always like to encourage those additional research questions.” —Research participant

Research Planning 23 3.4.1 Survey Method One key step in planning a survey is choosing the most suitable survey method. The survey methods discussed in this guidebook include passenger intercept, mail, telephone, and online. For surveys of businesses, in-person, in-office interviewing is also considered as an option. Choosing the appropriate method for any survey involves consideration of a variety of factors. Perhaps the key trade-off, however, is between cost and quality. As a general rule, the higher the quality desired, the more expensive the survey will be. Quality in this case includes the following: • Data quality. Will the questions be understood and elicit the desired information? Will all of the questions be answered? Will the answers be accurate? Will answers to any open-ended questions be clear and complete? • Response rate. Will the survey methodology achieve a high enough response rate (the pro- portion of those asked to participate in the survey who actually do so) that the results can be generalized to the population of interest? It is clear from the literature that respondents often differ from nonrespondents in material ways, and of course the survey team will know little if anything about those who do not respond. Although experts differ on this point, one rule of thumb is that the response rate needs to be over 50% for a researcher to be reasonably confident that the results are representative. While lack of response bias (which is the difference in responses between respondents and the popu- lation being surveyed) is more pertinent than a high response rate per se, the former is often difficult or impossible to ascertain. The over-50% metric indicates that researchers have at least heard from a majority of potential respondents rather than a minority. Regardless of what is considered an acceptable response rate, the lower the response rate, the more caution needs to be used in interpreting the data. All other things equal, response rates vary widely by survey method. The main survey methods and their advantages and disadvantages are outlined in the following. Passenger intercept surveys are most commonly conducted with departing passengers in gate areas. They can also be undertaken with arriving passengers as they leave security, wait at baggage claim, or stand on the curb. The former approach is widely recognized as more straightforward, easy to undertake, and representative; in some airports, surveys of departing passengers also ask about arrival experience. By far the majority of intercept surveys are conducted by trained interviewers, who ask the questions and record the responses using either paper and pencil or, more recently, tablets. Potential participants are selected using predetermined sampling rules that identify who the interviewer should approach, thus typically yielding a highly represen- tative result. The main advantage of intercept interview surveys, particularly in gate areas, is the potential for a high response rate because passengers typically have little else to do. In addition, high data quality results from the use of professional interviewers. Another potential advantage, depending on the purpose of the survey, is that passengers will have concluded their journeys through the airport at this point and will thus be familiar with the services and amenities they needed or wanted to experience. The primary disadvantage of interview surveys is their cost, which can be substantial. Self-administered surveys that are handed out, completed by the respondent, and then returned, either in person or by mail, are Best Practice “Length of questionnaire is key. Working with stakeholders, they want to throw a lot of questions in. You need to ensure the questionnaire is focused on what is needed versus nice to have. Keeping the questionnaire focused, the data are going to be better than a watered- down version of a questionnaire that is time sensitive to those that are moving through quickly.” —Research participant

24 Guidebook for Conducting Airport User Surveys and Other Customer Research frequently used for air passenger, employee, and tenant surveys. The key advantage of this approach is its relatively low cost because one interviewer can hand out a large number of questionnaires in a given time period. Disadvantages of this approach include lower response rates and inferior data quality. Length and complexity are also concerns; generally, airports try to keep such surveys short and simple to maximize the number of responses and the complete- ness of the information they get back. Mail surveys are infrequently used by airports but could be useful for surveying tenants that are spread out across the airport’s premises or even based off-site. Their main advantage is their relatively low cost. Key disadvantages are similar to those of surveys that are handed out: low response rates, inferior data quality, and the need to keep the survey short and self-explanatory. Telephone surveys play major roles in surveying residents and representative businesses in the area served by the airport. The key advantages of this method are the ability to obtain a rep- resentative sample of a large and dispersed target population and a high level of quality control due to the use of professional interviewers. Disadvantages include their relatively high cost and the need to allow a long enough survey period for an interviewing call center to maximize the response rate. Online surveys have become increasingly popular in recent years, in large measure because they can be conducted at extremely low cost. They are also often relatively easy to program and deploy with software that is readily available and inexpensive or even free. This software allows researchers to upload questionnaires to the Internet and send out email invitations to participate. Respondents enter their answers online, and the software stores them in an electronic database. Another approach more recently adopted by airports is to offer passengers online surveys when they access the Wi-Fi service. Although some airports have made taking such surveys a prerequisite for using Wi-Fi, many have found that this requirement causes sufficient discontent that they have reverted to optional survey completion. Others offer basic Wi-Fi service to all passengers but only provide high-speed Wi-Fi in exchange for participation in a survey. Some airports also offer online surveys at kiosks positioned throughout the premises. Regardless of how online surveys are deployed, online survey software offers a considerable degree of quality control. It takes respondents to applicable questions based on their answers to previous questions, and it offers only potentially valid responses. Tabulated responses and simple graphics are often created by the software, and the data can be downloaded for more detailed analysis. Frequently, respondents in locations other than the airport can save partially completed responses and return later to finish them, which can increase the response rate. Some programs also allow researchers to track respondents and nonrespondents so that reminders can be sent only to those who have not yet responded. Online surveys have three main disadvantages. First, it is still the case even today that not everyone has access to the Internet at home or at work. Second, response rates to online surveys are generally the lowest of any type of survey, reducing the likelihood of obtaining a representa- tive result. Third, online surveys have to be fairly short and simple. The reasons for this include varying technological capabilities among respondents, the absence of anyone to clarify issues the respondent may have, and people’s general expectation that online transactions should move rapidly. A final consideration with respect to online surveys is that while some results offered by outside consultants come at a low cost, they are frequently based on convenient, nonprobability samples and thus are not necessarily representative of the target audience. Obtaining repre- sentative samples for online surveys can be challenging. It is always important in assessing

Research Planning 25 research findings to consider the extent to which the methods used in conducting the underlying study yielded representative and hence useful results. Other issues that need to be considered when selecting a survey method include the following: • Speed. How quickly does the research need to be completed? As examples, mail surveys take a long time to come back; surveys that are handed out to passengers are usually either returned almost immediately or not returned at all. • Complexity. How complicated are the inquiries? The more complicated the questions, the more important it becomes to have a person available to clarify things and answer any questions the respondent may have. • Flow control. Does the order in which questions are asked need to be controlled? If so, then methods such as handouts and mail are excluded. An example of the need to control flow might be asking what airports in the area come to mind, then following up by naming and inquiring about any airports not mentioned. The second question in effect gives away possible answers to the first. • Visual aids. Are visual aids needed? An example might be a stated preference survey on the likely use of a proposed inter-terminal transportation mode, in which respondents are shown images of the proposed mode so they have a clear understanding of what it would be like. Methods such as landline telephone surveys that do not permit the display of such aids would be excluded. • Confidentiality. How important is confidentiality to the study? In many cases, airport surveys are not asking particularly sensitive questions, and confidentiality is not a major issue. Passenger surveys, however, frequently ask about the origin of the trip to the airport, which could be a home address. In this case, confidentiality is obviously of high importance. Some survey methods are more confidential than others. Mail surveys can be designed with no identifying information on the survey questionnaire so that respondents can remain anonymous; in-person interviews and online surveys perhaps raise the most concerns about confidentiality, although the latter can be made quite secure with the proper hardware and software. The main advantages and disadvantages of each survey method are summarized in Table 3-1. 3.4.2 Qualitative Methods Selection of a qualitative method is significantly less complex than the selection of a survey method, primarily because the airport’s information needs will drive the choice fairly directly. Focus groups will be most appropriate when there is a benefit to be derived from the interaction among customers and when a detailed understanding of an individual customer’s perspective is not necessary. In-depth interviews will be most appropriate in fundamentally the opposite circumstances, when information derived from customer interaction is not likely to be particu- larly useful and when an individual’s point of view needs to be understood in depth. They can also be useful when it might be difficult to obtain candid opinions from a group, such as one composed of potentially contrary consumer segments or of competitive businesses. Both of these methods can obtain the perspectives of a reasonably representative cross-section of a particular customer type (passengers, area residents, business representatives, employees, and concessionaires or tenants). The key difference in this respect is that in focus groups, the entire cross-section, or a reasonable facsimile thereof, will be present at the same time. Use of the remaining qualitative methods—observation and mystery shopping—is essentially dictated by what is to be studied. If something needs to be seen to be studied, then observation will be the correct technique. If an assessment of what is transpiring in retail establishments is called for, then mystery shopping is an appropriate approach.

26 Guidebook for Conducting Airport User Surveys and Other Customer Research 3.5 Schedule Ultimately, the research schedule will be determined by the date when the results are needed in final form by those commissioning the study and intending to use its findings. The planning team will then need to work backward from that date to select a start date that will accommodate all of the research tasks involved. If possible, it is wise to err on the side of caution and assume things will take longer to accomplish than optimists believe. Airports are dynamic environments, and disruptions of the best-laid plans can occur without warning. Caution is particularly in order when a research effort is new to the airport and has not been previously attempted. In this case, contingencies may arise that may have been anticipated by the more experienced but do not necessarily come to mind among the less experienced. When possible, it would be advisable to consult with representatives of other airports that have already undertaken what the airport is planning before finalizing the schedule. One caution that is unique to airports is the need for all on-premises survey staff to be screened and badged before they can begin work beyond security. This is important not only because of the elapsed time involved in background checks, but also because of the possibility that not all new staff will pass the check. This in turn may necessitate recruiting and screening additional staff and essentially starting over if there are no contingency plans in place. Survey Method Advantages Disadvantages Intercept interviews • High response rate • High data quality • Reasonably representative sample • Allows for complex surveys • High cost Mail • Lower cost than all but online and hand-back surveys • Useful for short and simple surveys • Lower response rate than intercept interviews • Lower data quality than intercept or telephone interviews • Periodic reminders are advisable • Returns come in slowly • Cost of outbound and return postage Mail-back, and hand-back intercept • Lower cost than intercept interviews • Useful for short and simple surveys • Lower response than intercept interviews • Lower data quality than intercept or telephone interviews • Mail-back returns come in slowly • Cost of return postage Telephone • Allows representative sample of large and dispersed populations such as residents and businesses • High data quality • Moderately high cost • Requires a call center with trained interviewers • Can be time-consuming to implement Online • Lowest cost of all methods • Automated data processing • Requires access to the Internet • Lowest response rate of all methods • Lower data quality than intercept or telephone interviews • Difficult to obtain representative samples Table 3-1. Advantages and disadvantages of major survey methods.

Research Planning 27 Other factors that should be considered include: • Whether there are internal committees or other groups from which approval will be required but that meet only periodically; • The extent to which seasonal representation is important to the results and, if so, how this will be accommodated; • Whether other planned research might interfere or conflict with the study; • How flight delays, flight cancellations, and gate changes will be handled in terms of survey sampling and interviewer reassignment; • The degree to which particularly hot, cold, or otherwise inclement weather could have an effect, such as for surveys conducted curbside; • Whether there are special events at certain times of year that could skew the results; • Whether there are planned training exercises that might result in terminal evacuations or abnormal passenger movements and could, therefore, skew the results; and • The extent to which periods such as the Thanksgiving and Christmas holidays might be unintentionally included when they should be avoided. 3.6 Budget The final step in the planning process is the preparation of a research budget. Ideally, this takes place after defining the study purpose, determining the research method, and considering the study content. If a budget has already been established, it is important to determine whether it is realistic. If it is not, the project manager may need to change the scope of the research or seek additional funding. Tables 3-2 through 3-6 outline all of the activities involved in various types of surveys as well as in focus groups, in-depth interviews, observation, and mystery shopping; the types of direct costs that need to be considered for each method; and other issues that may need to be assessed when developing a research budget for a particular type of study. Airports can use these tables to work internally or with vendors to develop an approximate budget for their project. Allocation of tasks between airports and consultants or vendors will vary from airport to airport and may also vary across projects within airports. For each project being under- taken, airports will need to consider to whom various tasks will be assigned before developing an appropriate budget. After a preliminary budget has been prepared, it may be necessary to obtain approvals to fund and conduct the study. This may involve persuading senior management that the research will be a worthwhile expenditure. Presentation of the merits of the undertaking could include the following: • Purpose of the study – Goals and objectives – Population to be studied – Information to be collected – Types of results and analysis expected – Questions to be answered by the research – Decisions or actions to be supported by the results • Management of the research – Composition of the planning and implementation teams – Research method and rationale – Planned schedule

28 Guidebook for Conducting Airport User Surveys and Other Customer Research Tasks Direct Expenses General Considerations • Planning meeting(s), including travel • Questionnaire design and review • Project management • General administration/coordination • In-house planning • Security clearance/badging, including travel • Technology procurement • Interviewer training, including travel • Interviewer briefing • Pretest/pretest debriefing, including travel • Pilot test/pilot test debriefings, including travel • Sample design • Interviewer scheduling • Programming/program testing • Interviewing and supervision of interviewers • Client contact/progress reports • Data entry (if applicable) and exports • Cleaning/checking narrative responses • Codebook development, coding, and code checking • Data cleaning • Data tabulation and analysis, including both initial data runs and full tabulations • Report preparation, proofing, and editing • Presentation preparation, proofing, and editing • Presentation to client, including travel • Planning meeting travel; may include mileage and parking, per diem, airfare, and hotel • Translation services • Technology acquisition (smartphones/tablets) and data plan • Field service or online software charges • Conference room rental for interviewer training/briefing • Training materials • Printing of questionnaires (if applicable) • Travel expenses for interviewers and supervisors • Report/presentation materials • Presentation to client; may include mileage and parking, per diem, airfare, and hotel • Security clearance • Locations of interview at airport • Interview length • Bilingual or multilingual interviewing staff (if applicable) • Overtime due to flight schedules • On-site space for managing operations • Online software requirements and access • Number of open-ended questions to clean, code, and transcribe • Level of data analysis • Cost of the study – Proposed budget – Components of the budget – Opportunities for adjustment, if needed – Cost relative to potential costs of unsupported decisions or actions Assuming approval of the budget, with revisions as necessary, it may well turn out that costs will require further adjustment as the project evolves. There may be unanticipated issues that call for additional expenditures, or there may be unexpected areas where savings are possible. As appropriate, further management approval may be needed during the study. Advising managers of these eventualities in advance will prevent unnecessary surprises. Finally, it is also important to note that research is often under-budgeted, especially by those with limited or no experience using a particular method. Serious consideration should always be given to including a line item for contingencies or the unexpected. Table 3-2. Activities, expenses, and considerations for intercept surveys.

Research Planning 29 Tasks Direct Expenses General Considerations • Planning meeting(s), including travel • Questionnaire design and review • In-house planning • Pretest/pretest debriefing • Sampling • Interviewer briefing • Interviewing • Client contact/progress reports • Codebook development, coding, and checking • Programming/program testing • Data cleaning • Data tabulation and analysis, including initial data runs and full tabulations • Report preparation, proofing, and editing • Presentation preparation, proofing, and editing • Presentation to client, including travel • Planning meeting travel; may include mileage and parking, per diem, airfare, and hotel • Translation services (if applicable) • Sample acquisition • Call-center charges • Report/presentation materials • Presentation to client; may include mileage and parking, per diem, airfare, and hotel • Number of completed surveys anticipated • Interview length • Anticipated completion rate • Number of open-ended questions to clean and code • Level of data analysis Table 3-3. Activities, expenses, and considerations for telephone surveys. Tasks Direct Expenses General Considerations • Planning meeting(s), including travel • Questionnaire design and review • Invitation design, review, and reminders • In-house planning • Pretest/pretest debriefing • Sampling • Programming/program testing • Survey launch and reminders • Respondent contact/undeliverable emails • Client contact/progress reports • Codebook development, coding, and checking • Cleaning/checking transcriptions • Data cleaning • Data tabulation and analysis, including initial data runs and full tabulations • Report preparation, proofing, and editing • Presentation preparation, proofing, and editing • Presentation to client, including travel • Planning meeting travel; may include mileage and parking, per diem, airfare, and hotel • Translation services (if applicable) • Sample acquisition • Call center or online software charges • Report/presentation materials • Presentation to client; may include mileage and parking, per diem, airfare, and hotel • Number of invitations • Personalized versus generic invitations and reminders • Anticipated response rate • Questionnaire length • Number of open-ended questions to complete and code • Level of data analysis • Online software requirements Table 3-4. Activities, expenses, and considerations for online surveys.

30 Guidebook for Conducting Airport User Surveys and Other Customer Research Tasks Direct Expenses General Considerations • Planning meeting(s) • Design participant screening questionnaire • Design moderator’s outline • Review and revise instrument • Review recruitment effort • Travel to focus group site (time) • Management and oversight (time) • Moderate group(s) • Client attendance • Transcript review • Report preparation, review, and revision • Facility charges, which may include participant recruitment, facility rental, hosting, refreshments for participants, meals for clients, and audio- or videotaping • Cooperation fees for participants • Printing of moderator’s outlines for observers • Design and production of boards/illustrations or other presentation material • Meeting transcription(s) • Travel to focus group site; may include mileage and parking, per diem, airfare, and hotel • Number of groups • Location • Duration of group meetings • Number of observers • Special equipment and setting requirements Table 3-5. Activities, expenses, and considerations for focus groups. Tasks Direct Expenses General Considerations • Planning meeting(s), including travel • Interview guide design and review • In-house planning • Pretest/pretest debriefing • Selection of interviewees • Interviewer briefing • Interviewing • Client contact/progress reports • Codebook development, coding, and checking (if applicable) • Transcript review and coding (if applicable) • Report preparation, proofing, and editing • Presentation preparation, proofing, and editing • Presentation to client, including travel • Planning meeting travel; may include mileage and parking, per diem, airfare, and hotel • Translation services (if applicable) • Interviewing facility charges, if applicable • Audio- or videotaping • Report/presentation materials • Presentation travel; may include mileage and parking, per diem, airfare, and hotel (if applicable) • Interview locations (office, home, or formal facility) • Number of completed surveys anticipated • Interview length • Anticipated completion rate • Number of open-ended questions to clean and code (if desired) Table 3-6. Activities, expenses, and considerations for in-depth interviews.

Statistical Concepts 4.1 Introduction An understanding of the concepts of sampling and statistical accuracy is fundamental to an understanding of such issues as the size of the sample to be used in a survey and the accuracy of the resulting findings. This chapter is intended to introduce statistical concepts for those who may not have had any formal statistical training or may wish to refresh their understanding of the topics addressed. It may also be useful as a reference for those who are already familiar with the considerations addressed in the chapter. For those interested in additional informa- tion or the derivation of the relationships presented in the chapter, a more detailed discussion is provided in Appendix A. There are a number of basic statistical concepts that are necessary to understand in order to follow the discussion in the subsequent sections of this chapter. While many of these concepts use terms that are in common usage, in statistics these terms have a particular meaning that is important to understand. Some of these terms are defined and explained in the “Overview of Basic Statistical Concepts” text box. 4.1.1 Overview of Statistical Concepts This section is intended to provide a brief overview of the statistical concepts discussed in this chapter for readers who may not need the level of detail presented in the following sections but would like a broad understanding of the issues involved. It may also be helpful to establish the context for the discussion in the rest of the chapter. Most surveys attempt to obtain responses from a subset of a larger group of interest, such as all air passengers using an airport or all airport employees. This subset is termed a “sample,” and the larger group is termed the “target population” (or simply the “population”). The intent is that the responses from the sample of respondents reflect the corresponding characteristics of the target population. This naturally raises two questions: • How accurate are the survey results? • How large a sample is required to achieve a desired level of accuracy? Because the sample does not include the entire population (by definition), there will be differ- ences between the survey responses for any given characteristic (say trip purpose) among the members of the sample and the true proportions of that characteristic among the population. If the sample is very small, the likelihood that it will accurately reflect the true proportions among the population is also quite small. As the sample size increases, the likelihood that it will reflect the true proportions of given characteristics also increases, or put another way, the results of the survey become more accurate. C H A P T E R   4 31  

32 Guidebook for Conducting Airport User Surveys and Other Customer Research The key concept here is the idea of the likelihood or probability of the sample results reflecting the true proportions (or average value in the case of a numerical response, such as the length of time before flight departure that an air passenger arrived at the airport) of the population. The difference between the proportion or average value given by the sample responses and the true values for the population is termed the “sample error” (or simply the “error”). Statistical techniques provide a way to estimate the size of the likely error for any given sample size. The second key concept is that the greater the variability of a given characteristic in the population, the greater the probability that the sample error will be larger than a specified amount for a given sample size. It should be clear that if all the members of the population were to have the same value for a given characteristic, then a sample of one would give a completely accurate value for that characteristic. In reality, of course, any given characteristic takes many values. The relative proportions of each of these values are termed the “distribution” of those values. As the values of the characteristic become more diverse in the population, so it becomes less and less likely that the values found in a sample of a given size will correspond to those in the population within any given margin of error. Overview of Basic Statistical Concepts Distribution In any set of data, each item in the dataset has a particular value. The distribution of the data in the dataset refers to the proportion of the items that take each of the possible values in the dataset. With discrete data (e.g., the number of people in a travel party), each possible value (1, 2, 3, etc.) will occur for some proportion of the total number of items in the dataset. For continuous data (e.g., the time taken to drive to the airport), there are effectively an unlimited (or at least very large) number of possible values. Therefore, the distribution is defined in terms of a functional relationship, typically plotted as a graph or expressed as an equation. The relationship can be used to determine the proportion of values within a given range. Average The average value of a set of data (also referred to as the mean of the distribution) is defined as the sum of the values of each item in the dataset divided by the number of items. This corresponds to the common usage of the term “average.” Usual usage is to refer to the average of a set of data and the mean of a distribution, although the concepts are identical. Variance The variance of a dataset or distribution measures the spread of the values about the average or mean value. It can be thought of as the average of the squared difference between each value and the mean of the distribution. The differences are squared so that larger differences have greater importance than smaller differences, and negative and positive differences do not offset each other. Standard deviation The standard deviation of a dataset or distribution is defined as the square root of the variance. This expresses the spread of the values around the average or mean of the dataset or distribution in the same units as the data. Standard error The standard error of a sample statistic (e.g., the average or mean of a sample drawn from a population) is calculated as the standard deviation of the sample divided by the square root of the sample size. It is a measure of the expected accuracy of the statistic and reflects the fact that if the sample were to be drawn from the population multiple times, the statistic would be slightly different each time. More details can be found in textbooks on general statistics, such as those listed in the bibliography.

Statistical Concepts 33 The likely accuracy of an estimated statistic (such as the average value) for a set of sample data can be expressed as the standard error of the statistic. It can be seen from the definition of standard error given in the text box that a doubling of the sample size will reduce the standard error of an estimated statistic by about 29% (1 − 1/ 2). Therefore, there are two aspects to expressing the accuracy of any survey result: the margin of error being considered, and the probability that the actual error (which is of course unknown) will lie within that range. This probability is termed the “level of confidence,” and the margin of error is termed the “confidence interval.” Thus, one might express the expected accuracy of a particular survey result as being within a range of plus or minus 5% with a 95% confidence level. As one is willing to accept a lower level of confidence, the confidence interval becomes smaller. Put another way, as one reduces the width of the desired confidence interval (say to plus or minus 2%), the associated confidence level becomes less. In other words, one is less confident that the true error lies within that smaller range. The calculation of the confidence interval (the margin of error) for a given confidence level depends on both the sample size and how the sample is chosen. For any given sample size, the highest accuracy (smallest margin of error) is obtained if the members of the sample are randomly selected from the population. However, in most cases, this is almost impossible to achieve due to the practical constraints on how a survey can be conducted. Obviously, one cannot gather all the air passengers using an airport into one place and then randomly select those to interview. Therefore, other sample selection methods are used. To the extent that these methods change the likelihood that any given member of the population is chosen for the sample, the accuracy of the results will be reduced. This brings up a third consideration in selecting the sample size to achieve a desired accuracy (in addition to the desired confidence interval and level of confidence): the sampling approach used. An important consideration is whether the variation of the characteristic of interest in the population differs between the sample and the population. Even with a non-random sampling method, such as sequentially selecting passengers waiting in an airline gate lounge, if the distri- bution of the characteristic across the sample is the same as across the population, the accuracy will be the same as if a true random selection had been made. This emphasizes the importance of considering whether the characteristics of interest in a survey appear to differ between the sample and the population as a result of the way that the sample is chosen. Where this is thought likely to be the case, the statistical techniques to estimate the required sample size for a given confidence interval and confidence level need to be modified accordingly. 4.2 Concepts of Census and Sample Surveys In general, a survey will collect information from a sample of individuals from the target population. In some cases, it may be appropriate to survey the entire population, in which case the survey is termed a census survey. A census survey is generally appropriate for collecting information on small populations when a very high level of accuracy is required and when there are no significant constraints due to budget, survey resources, or the time period when individuals are available to be surveyed. A census survey might be appropriate, for example, for a survey of tenants at the airport, but not for a survey of air passengers. For a sample survey, a sample of respondents is selected from the target population in such a way that the characteristics of the population can be inferred from the corresponding charac- teristics of the sample. The way this is done and the implications for the accuracy of the resulting estimates of the characteristics of the population are discussed in more detail later. A list of all

34 Guidebook for Conducting Airport User Surveys and Other Customer Research the members of a population from which a sample is drawn is termed the “sampling frame.” This can be an actual list (such as a list of all airport employees), or it may simply be a definition of the population (such as all air passengers originating their air trip at a given airport during a particular time period). 4.3 Statistical Accuracy and Confidence Intervals The characteristics of interest of the population being surveyed, such as the mode of travel to the airport, will vary across the members of the population. Aggregate (or overall) measures of the population, such as the proportion of air passengers accessing the airport by taxi, can be estimated from the corresponding values for the sample. However, when drawing a sample from a population, the distribution of the characteristics of interest (such as the ground transportation mode used to access the airport or household income) across the members of the sample will generally be different from the corresponding distribution across the population, and thus measures of these two distributions, such as the average value, will also be different. This difference between the sample average and population mean is referred to as the error of the estimate. With very small samples relative to the size of the population, it is unlikely that the distribu- tion of a given characteristic across the sample will correspond exactly to its distribution across the population as a whole, since the opportunity for the sample to include the full range of values that exist in the population is limited by the small sample size. As the size of the sample increases, it becomes more likely that the distribution of any given characteristic in the sample will correspond to that in the population. The extent to which the distribution of a given characteristic in a sample corresponds to the distribution of the characteristic in the population as a whole depends on the sample size and how variable the characteristic is in the population. In statistical terminology, this variability is termed the “variance” of the characteristic. In the extreme case in which every member of the population has the same value for a given characteristic, a sample of only one respondent would provide a completely accurate estimate of that value. At the other extreme, if every member of the population has a different value for a given characteristic, a sample of the entire population would be required in order to include every possible value of the characteristic occurring in the population. If a sample of a given size is drawn randomly from a population multiple times, a slightly different distribution of any given characteristic would be expected in each sample. The greater the variance of the characteristic in the population, the more variation there would be in the distribution of the characteristic across the different samples. Therefore, the average value of a given characteristic in a sample of a given size, although it is a specific value for any particular sample, will vary across the different samples. This results in the following fundamental point: The average value of any given characteristic in a sample drawn randomly from a given population has an expected variance that depends on the variance of the characteristic in the population as well as the size of the sample relative to the size of the population. Because of this variance, there will be an expected error between the average value of a particular characteristic given by a single sample and the true average (or mean) value of the characteristic in the population. Therefore, an obvious question to ask in analyzing the results obtained from a given sample is how accurate the results are. Because the value of the popula- tion mean is not known (the survey is being performed to estimate this), the actual error is not known. However, the expected distribution of the error can be determined, as discussed in the following, and an estimate made of the likely range of the error.

Statistical Concepts 35 The standard deviation of the estimated average value of any particular characteristic deter- mined from a sample is termed the standard error of the estimate (SEE) and is a measure of the accuracy of an estimate. For large enough samples, the error will approximate a normal distribution with an expected (mean) error of zero, as illustrated in Figure 4-1. The figure shows the probability of a sample giving an error of any particular size, measured in terms of the number of standard deviations from the mean value. As the range being considered gets larger, expressed in terms of stan- dard deviations, so the probability of the actual value lying in the range approaches 100%. The greater the variance of the sample estimate of the mean (i.e., the larger the standard deviation of the sample estimate), the fewer standard deviations from the mean an error of any particular value will be. Figure 4-1 also illustrates an important related aspect of sample error. As the standard deviation of the sample estimate of the mean increases, so the range of values covered by any given number of standard deviations also increases. Because an error of any particular value (in the units of the variable) will be fewer standard deviations from the mean, this reduces the probability of getting an error no greater than that value, and hence increases the correspond- ing probability of getting an error greater than that particular value. Thus, as the variance (and hence the standard deviation) of the sample estimate increases, so the probability of getting an error greater than any particular value also increases. This leads to the second fundamental aspect of sampling accuracy: Although the actual error of a sample estimate of the mean value of any characteristic of the population is unknown, the probability of this error being less than any given value can be estimated. This second aspect has the important implication that any estimate of the expected error has two attributes: the magnitude of the error being considered and the probability that the actual error is less than this value (referred to as the “confidence level”). Because the error distribu- tion is symmetrical, as shown in Figure 4-1, it is common to express the expected error range, sometimes termed the “margin of error,” as plus or minus a specified amount (for a continuous variable) or as a number of percentage points (for a categorical variable, where the value is one of a defined set of values). (See the “Expressing the Accuracy of Variables Expressed as a Percentage” text box for an explanation of the difference between a value expressed as a per- centage and expressed in percentage points.) For example, the results of a survey question might be reported as being accurate to within plus or minus 3% with 95% confidence. In this case, Note: The probability of an error being between two values is given by the area under the probability density curve between those values. -3 -2 -1 0 1 2 3 Error (standard deviations) 95 % Pr ob ab ili ty D en si ty Figure 4-1. Example of the probability distribution of the expected error.

36 Guidebook for Conducting Airport User Surveys and Other Customer Research the probability of the estimate being within a margin of error of plus or minus 3 percentage points is 0.95, or 95%. The results could also be described as having a 95% confidence interval of plus or minus 3%, where the term “confidence interval” refers to the margin of error for a specified confidence level. Note that a confidence level of 95% means that there is a 1 in 20 (i.e., 5%) chance that the error would be larger than the stated confidence interval. As the confidence level increases (i.e., there is a greater probability that the actual error lies within the interval being considered), the size of the associated error range also increases. For a given confidence level, the size of the corresponding error range depends only on the vari- ance (and hence the standard deviation) of the expected error. As illustrated by Figure 4-1, a given confidence interval spans a fixed number of standard deviations on either side of the mean. For a 95% confidence interval, this range is plus or minus about 2 (strictly 1.96) stan- dard deviations. For a 90% confidence interval, this range is plus or minus about 1.65 standard deviations. Thus, if the variance of the estimated mean of some characteristic in a given sample is 0.0004 (i.e., the standard deviation is 0.02 or 2%), for a confidence interval of 95%, the margin of error would be plus or minus 4% (2 times the standard deviation). For a confidence interval of only 90%, the margin of error would be plus or minus 3.3% (1.65 times the standard deviation). Expressing the Accuracy of Variables Expressed as a Percentage For categorical variables, results are often expressed as a percentage of the sample—for example, the percentage of air passengers who use transit to access the airport. In such cases, expressing the accuracy of the estimate of the proportion of the sample in a given subgroup as a percentage can have two different meanings: • A percentage of the sample size (e.g., 5% of the sample), or • A percentage of the subgroup mean (e.g., 5% of the proportion in the subgroup). The first meaning is often referred to as “percentage points” to distinguish it from the latter. The two are very different. For example, if it is estimated that 10% of passengers take transit, then an accuracy of 5 percentage points at the 95% confidence level corresponds to the interval from 5% to 15%. This range corresponds to plus or minus 50% of the proportion of transit passengers in the survey (i.e., 5/10 or 50%). However, an accuracy of 5% of the estimated proportion of passengers taking transit corresponds to the interval from 9.5% to 10.5%, equivalent to an accuracy of 0.5 percentage points. To avoid confusion, care must be taken when expressing the accuracy of variables expressed as a percentage. When interpreting such values, care must be taken to be clear whether the accuracy is a percentage of the entire sample or of the subgroup in question. It may be helpful to make a distinction between percentage and percentage points in discussing accuracy. Failure to be clear in this distinction when reporting survey results can result in a situation where the reader cannot determine which way to interpret the stated accuracy.

Statistical Concepts 37 In practice, in addition to the sample size and variation in the values of the attributes being estimated, the accuracy of the estimated mean values of the attributes of the population also depends on the sampling method, the level of non-response to a survey, and the character- istics of those who did not respond (nonrespondents). Unfortunately, the characteristics of the nonrespondents are typically unknown. In the absence of any information about their characteristics (the usual case), they are generally assumed to be the same as those of the respondents. The appropriate level of confidence to be used in expressing the margin of error in the results of a sample depends on the costs associated with making an error. The higher the costs of an error, a greater level of confidence might be required that the true value is within the confidence interval. The width of the confidence interval can be reduced by increasing the sample size or, possibly, improving the sample design. Generally, 95% confidence intervals are used for most purposes, but 99% confidence intervals may be used for some critical variables, while in other cases 90% confidence intervals may be adequate. The issue of what level of confidence to use in establishing the size of a sample is often ignored. Accuracy is discussed further in Section 4.4. Information on the calculation of the SEE using the different sampling methods is provided in Appendix A. 4.4 Sampling Methods For a sample survey, the sample of respondents should be selected from the population in such a way that the probability of any individual respondent being selected can be estimated. This method allows generalizations to be made about the entire population from the characteristics of the sample and estimates to be made of the likely accuracy of the estimated characteristics of the population based on the size of the sample. The most straightforward approach to obtaining a representative sample from a population is to select the members of the sample randomly from the population. However, in practice this is often difficult to achieve, particularly in an airport environment. Furthermore, it has the disadvantage that the sample will include rela- tively few members of particular subgroups of the population that make up a small proportion of the total population. To address these concerns, other common sampling methods may be used. These methods are summarized in Table 4-1 and discussed in more detail in the follow- ing subsections. The choice of the appropriate sampling method is partly a question of how best to achieve the desired accuracy of the survey results and partly a consequence of the practicalities of performing the survey. For example, it is common to perform air passenger surveys in depar- ture lounges, because passengers are more willing to be interviewed or fill out a survey question- naire when they are no longer anxious about whether they will make their flight and they are sitting down. Furthermore, they have already experienced the entire airport departure process apart from boarding their flight. However, this locale constrains the sample to those passengers on a set of flights and does not provide a truly random sample of all passengers using the airport. For this reason, other sampling strategies are commonly used, as discussed later. Although not a truly random sample, their results may approximate a random sample if well designed. On the other hand, a mail survey sent to the home address of airport employees can sample employees randomly from a list of all employees at the airport. The sampling method selected will depend on the type of survey, data collection method, and characteristics of the population. Random and sequential sampling are the simplest methods to implement but require large sample sizes to obtain an adequate number of responses from small subgroups of the population, while stratified and cluster sampling can be used, with a

38 Guidebook for Conducting Airport User Surveys and Other Customer Research limited budget, to improve the accuracy of survey results for different subgroups. Often multi- stage sampling is appropriate. For example, cluster sampling may be used with flights selected using stratified sampling, and passengers on those flights can then selected using sequential sampling. A controlled sample attempts to design the sampling approach so that the composition of the sample is such that the distribution of any given characteristics in the sample corresponds to the distribution of those characteristics in the population. This objective is generally satisfied with a truly random sample, provided the sample size is large enough. In practice, achieving a truly random sample with airport user surveys is often difficult, as discussed in Section 4.4.1. In the case of other sampling methods, it is necessary to adjust the sampling rate or define the strata or clusters so that, where the characteristics of interest of the population vary across different subgroups or time periods, those subgroups or time periods are represented in the sample in proportion to their occurrence in the population. A controlled sample is thus an attribute of a particular sample design rather than a different type of sampling method. If the sample is not controlled so that its composition is such that the distributions of each of its characteristics reflect the distribution of the corresponding characteristics in the population, then the survey results need to be weighted to properly reflect the characteristics of the population. A cluster sample in which sampled flights are chosen to reflect the proportions of flights in markets that are believed to have different passenger char- acteristics (e.g., international, domestic short-haul, domestic long-haul) and passengers are sampled for each flight in proportion to the passengers on the flight would represent a controlled sample. Thus, a self-administered air passenger survey in which flights are selected in proportion to the number of flights in broadly defined markets and survey forms are given to every adult passenger on those sampled flights would qualify as a controlled sample. Type Method Comment Random Respondents are selected randomly from the target population. Often difficult to do in airport environment. Need to use some randomizing technique (e.g., use of random number tables). Selection by interviewers can lead to biases if not well trained. Sequential (systematic) Every nth individual is selected when potential respondents are arranged in some order. First respondent should be selected randomly from among the first n individuals. Good practical technique for airport surveys. Sample characteristics will be equivalent to a random sample if the order of potential respondents is not related to the variables of interest. Stratified Respondents are grouped into homogeneous groups (e.g., different categories of employee). Sampling occurs within each group separately. Used to obtain a more representative sample of different groups, particularly if the groups vary in size, or to obtain a specific accuracy in estimates for each group. Cluster Respondents are sampled from naturally occurring groups (e.g., flights). A sample of flights is selected, then all or a sample of passengers on those flights are selected. Suitable for large surveys where a wide range of flights can be sampled. Can use stratified sampling of flights to obtain a more representative sample. Non- probability Respondents are selected on the basis of some criterion that does not allow the probability of sampling any given member of the population to be determined. May be useful for gathering information on the range of possible responses, where the frequency with which those responses occur in a defined population is not required. Table 4-1. Summary of sampling methods.

Statistical Concepts 39 Because variation in the characteristics of the population over time or across different sub- groups will not in general be known until the survey results are obtained, designing a controlled sample means making assumptions about subsets of the population with different characteristics and ensuring that each subset is sampled in proportion to its occurrence in the population. If it turns out that two subsets of the population that were expected to have different characteristics in fact have similar characteristics, the results for the two subsets can be combined. However, if two subsets that in fact have different characteristics are assumed to be similar and are not sampled in proportion to their occurrence in the population, the results will be biased. Weighting of the subgroups will be required to remove this bias. Analysis of the results of previous surveys at the airport in question or of surveys conducted at other airports with similar traffic patterns can help identify subsets of the population that are likely to have different characteristics. The design of the controlled sample then attempts to ensure that those subsets are sampled in proportion to their occurrence in the population. 4.4.1 Random Sampling With random sampling, each individual must have an equal (or at least known) chance of being selected. An example of random sampling would be a tenant survey where a list of all airport tenants is assembled (the sampling frame), and a table of random numbers is used to select individual tenants from the list. The sampling approach will generally ensure that no individual can be in the sample more than once. For air passenger surveys, the sample size is typically small relative to the population and the methodology is such that there is very little like- lihood of surveying the same person twice. For most other airport user surveys, the methodology precludes sampling the same respondent twice. If an individual gets surveyed twice on two different trips, that is not the same thing as surveying the same traveler twice on the same trip. The former should be valid as the sample being drawn is really of passenger trips, not of passengers as individuals, and a single passenger may make more than one trip during the survey period. Obtaining a truly random sample is often difficult, particularly for airport surveys. For example, identifying each member of the population to include in the sampling process and then applying a method for randomly selecting them can be difficult, if not impossible, in an airport departure lounge. There are also problems associated with having interviewers select passengers to survey; this introduces a human element and invariably leads to biases. To avoid this, random numbers or sequential sampling, discussed in the following subsection, should be used for selecting individuals to survey. Interviews at groundside locations such as curb areas and parking lots, where the next available passenger is surveyed once an interview has been completed, are equivalent to random sample surveys as long as the ratio of interviews to passengers is fairly constant. However, such an approach will clearly change the sampling rate as the passenger flow changes. During periods of very low flow, every passenger might be interviewed, while during periods of high flow, only a small proportion of passengers would be interviewed, and this should be taken into account in analyzing the results. 4.4.2 Sequential Sampling Sequential sampling is generally a good form of sampling for use in airport surveys. With sequential sampling (also referred to as “systematic sampling”), the population is arranged in some logical order, and every nth individual is selected, starting with a randomly selected

40 Guidebook for Conducting Airport User Surveys and Other Customer Research individual from the first n individuals. An example of sequential sampling is to survey every fourth passenger in a check-in queue. Sequential sampling is usually easier to apply than random sampling and will yield a random sample if the order of individuals in the list is essentially random with respect to the characteristics being measured in the survey. For example, there is no reason to think that the order in which people sit in a departure lounge has any systematic relationship to the characteristics being measured (such as their trip purpose or how they got to the airport), and therefore selecting every nth person is in effect a random sample. Of course, depending on the layout of the lounge, early arriving passengers or those with difficulty walking may sit closer to the boarding point, while later arriving passengers may have to use seats further away. However, as long as all passengers in the lounge are included in the sampling strategy, where they sit should not affect their chance of being sampled. Where the population list is ordered by a relevant characteristic, the use of sequential sampling will often result in a sample with a more representative range of characteristics than would result from using random sampling. For example, in selecting flights to survey, if all flights during the survey period are listed in order of flight stage length, the resulting sample would likely better reflect passenger characteristics such as destination city or region than a random sample because sequential sampling ensures a more even spread of flights by stage length and thus over destinations and regions. With random sampling, some subgroups of the population (flights with a particular stage length in the previous example) may be missed completely, and others may be over-sampled. One common application of sequential sampling in air passenger surveys is to list flights by departure time (and destination to resolve flights with the same departure time) and select every nth flight to survey. A variation on this approach is to list the number of seats on each flight and calculate the cumulative total number of seats up to and including each flight (the total number of seats on previous flights on the list plus the number on the current flight). Flights are then selected by identifying the flight that corresponds to every mth seat on the cumulative list. This ensures that the probability of a given flight being sampled is proportional to the size of the aircraft, which approximates a random sample of air passengers if the same number of passengers is interviewed for each flight. The procedure to calculate n or m is described in Section 10.6.3. 4.4.3 Stratified Sampling In stratified sampling, the population is divided into mutually exclusive groups (strata), and individuals within each group are randomly sampled. Groups should be selected so that they are homogeneous with respect to the variables being studied (i.e., there is low variation within the groups), but the variation in the relevant variables is large between groups. For example, in a survey to determine passenger spending at airport concessions, passengers taking short- haul domestic flights are likely to spend much less than passengers taking long-haul inter- national flights. The variation in spending among short-haul domestic passengers and among long-haul international passengers is likely to be less than the variation in spending between the two groups. If the criterion for stratification is highly correlated to the variable being studied, such as in this example, the gain in accuracy can be significant. Examples of stratified sampling include dividing flights into groups such as international and domestic short and long haul, or by region, and dividing passengers to be sampled into groups based on day of the week, time period during the day, and airport terminal used. The variable used for stratifying the population must be known for all individuals in the population. Once the survey population has been stratified into groups, simple random or sequential sampling is used to select individuals from each group.

Statistical Concepts 41 With proportional stratified sampling, the proportions of individuals surveyed in each group are equal. This form of sampling is often used to ensure a more representative sample than simple random or sequential sampling. In nonproportional stratified sampling, different sampling fractions are used to improve the accuracy of estimates for a given overall sample size. Situations where nonproportional sampling is desirable include the following: • Where the variation in the variables being studied differs greatly between groups. The non- homogeneous groups (with a high variation in the variables of interest) should have a larger sample than the homogeneous groups. For example, consider a survey conducted to deter- mine the average number of check-in bags per passenger where a stratified sample is to be drawn with flights grouped into long- and short-haul domestic and international flights. If it is known that the variation in the number of check-in bags is greater for passengers on long-haul international flights than on short-haul domestic flights, the sampling fraction would be higher for the long-haul international flights. • Where comparisons of distinct subgroups of the population are required—for example, comparisons between domestic and international passengers. • Where the cost of collecting the data differs greatly between groups. Here, overall accuracy for a given cost can be improved by having a lower sampling fraction for the groups with high data-collection costs. However, while this may lead to a higher overall accuracy for the pooled data, when the characteristics of subgroups need to be considered, as is almost always the case in airport surveys, it can lead to very different accuracy for the various subgroups. Thus, the approach of reducing the sampling fraction for groups with higher data-collection costs is not generally recommended for airport surveys. Expanding the sample results of nonproportional stratified sampling to determine estimates for the population is not as straightforward as with proportional stratified sampling; this is discussed in Appendix A. If nonproportional stratified sampling is appropriate, it is suggested that the planning team either become knowledgeable on the subject (refer to the bibliography for appropriate guidance) or consider using external expertise. 4.4.4 Cluster Sampling With cluster sampling, the population is distributed in a large number of naturally occurring groups—for example, passengers on flights. The groups, or clusters, are sampled; thus not all clusters are included in the sample. This is the primary difference from stratified sampling, where individuals are sampled from every group. In the simplest form, all individuals within a cluster are sampled. When clusters are homogeneous, it is more efficient to sample only a fraction of the individuals within a cluster and to sample more clusters. Cluster sampling is used to make sampling easier and less costly by limiting the survey to well-defined groups such as passengers on specific flights, and it works well when the characteristics of interest have low variability between clusters and high variability within clusters. For example, although the household income of passengers on a given flight will span a wide range, the average household income of passengers on different flights will show much less variability. The accuracy of estimates made using cluster sampling will almost always be lower than if a random sample is used with the same sample size because the selected clusters may not be fully representative of the target population as a whole and can be significantly lower if vari- ability between clusters is high or a small number of clusters are selected. It is important that the consequences of the design of the cluster sample (often referred to as the design effect) be incorporated into the analysis when evaluating the accuracy of estimates and required sample sizes. Details of how to calculate sample sizes and confidence intervals for cluster samples are included in Appendix A.

42 Guidebook for Conducting Airport User Surveys and Other Customer Research A common example of cluster sampling in airport surveys is the use of individual flights as clusters, with the flights to be surveyed being selected using random, sequential, or stratified sampling. Then, either all passengers on each selected flight or a sample of passengers on those flights is surveyed. 4.4.5 Nonprobability Sampling Nonprobability (or uncontrolled) sampling is where the probability of an individual’s selection cannot be determined. Examples of nonprobability sampling include: • Surveys of passengers who ask for help at an airport information booth, where no record is kept of the number of passengers seeking help at the booth, and • Voluntary online surveys where all visitors to the airport website are invited to complete a survey. With nonprobability sampling, it is not possible to calculate the sample size required to achieve a given level of accuracy or to make generalizations about the population. These types of surveys are usually of limited value in ascertaining properties of the population but can be useful for obtaining ideas and user feedback. 4.5 Sample Size A critical issue in planning any survey is determining the appropriate sample size, which is influenced by such considerations as: • Survey purpose, • Analysis of subgroups of interest, • Required precision of the survey results, • Credibility of results among decision makers and data users, and • Available resources (including budget, personnel, and equipment). The survey purpose influences the required sample size in three ways: (1) by determining the key characteristics of the air travel party and the precision to which they need to be known, (2) by establishing the level of disaggregation to which the results need to be expressed, and (3) by identifying the value to be gained from improved precision. For any desired degree of precision in the survey results, the need to consider subgroups of interest—such as air passengers with ground origins in a particular area or visitors on business trips—will increase the required sample size of the overall survey in order to ensure a large enough number of respondents in the subgroup(s) of interest. The required precision and the credibility of results influence the size of confidence interval and the acceptable margin of error. Larger samples are required to reduce the margin of error or to increase the confidence level for a given margin of error. Although the target sample size for a survey should ideally be determined by the purpose and objectives of the survey and the uses to which the results will be put, in reality the budget available to fund the survey often constrains the sample size, particularly where the budget has been established before the detailed planning of the survey has begun. Time constraints can also influence the sample size if information is required on short notice. Other factors affecting the required sample size include the following: • The proportion of the population with the attributes being measured. An airport seeking pas- sengers’ opinions of the retail concessions (as distinct from food and beverage concessions),

Statistical Concepts 43 for example, must design a survey that takes into account the fact that only a small proportion of passengers will have actually visited the retail concessions. The passengers visiting the retail concessions are a subgroup of all passengers, and so the required sample size is found in a similar way to that for a survey with any subgroup of interest. • The variability of the respondent characteristics being measured. If the variability is high, a larger sample size will be required. As discussed in Section 4.1.1, the standard error of an estimated statistic (e.g., the average value of a given characteristic) is proportional to the standard deviation of that characteristic in the population and inversely proportional to the square root of the sample size. It therefore follows that, as the variability of that charac- teristic increases, a larger sample size is needed to achieve a desired level of accuracy. • The sample design used. For example, a good stratified sample can permit a smaller sample size than a random sample for a given level of accuracy, while cluster sampling will usually necessitate a larger sample size (as discussed in Sections 4.4.3 and 4.4.4). In the following discussion, sample size refers to the number of completed responses; the number of people approached to participate in the survey may be significantly higher depend- ing on the rates of refusal and incomplete responses. These refusals and incomplete responses will generally take some time to survey and process, which could be significant in some surveys (e.g., where follow-up phone calls are made) and should be allowed for in determining resource requirements. To estimate the total number of individuals to approach, divide the desired sample size of completed surveys by an estimate of the completed survey response rate, expressed as a pro- portion. For example, if a sample size of 1,000 is required and the response rate is expected to be about 70%, then 1,429 [1,000/0.7] individuals would need to be approached. If the targeted survey respondents must satisfy a criterion that cannot be visually identified (e.g., an originat- ing passenger), then the effective response rate of those approached will be lower, requiring a larger initial sample size. 4.5.1 Sample Size with Random Sampling The method for calculating the sample size required to obtain a specified accuracy differs if the required accuracy is for a question with categorical or numerical responses. With a cate- gorical response, the respondent must choose from a limited number of defined responses. For example, for a question on mode of travel to the airport, categories could be private vehicle, rented vehicle, taxi/limousine, train, bus, airplane, and walk/bicycle. Determination of the sample size for each type of question using random sampling is discussed in the follow- ing paragraphs. Categorical Response Questions When using categorical response questions and random sampling, the sample size required to give a specified level of accuracy is a function of the population size and the proportion of the population in the category of interest (e.g., proportion using a private vehicle as the mode of travel to the airport). This proportion is unknown and should be estimated in the survey plan- ning stage from experience, previous surveys, or values from other airports. The largest required sample size occurs when half of the population has the characteristic of interest. Table 4-2 provides approximate 95% confidence intervals for a range of population and sample sizes and two values of the proportion of the population in the category of interest (50% and 20%). Thus, for surveys with many questions with a range of mean proportions, it is appropriate to use the sample size based on the 50% proportion as this will provide at least the required accuracy for all cases. Table 4-3 gives the required sample size using random sampling, based on the 50% proportion, for various confidence intervals and a range of population sizes. Alternatively,

44 Guidebook for Conducting Airport User Surveys and Other Customer Research the sample size for an accuracy of ±a percentage points can be calculated for a 95% confidence level using the following expression: n p p a p p N ( ) ( ) ( ) = − + − 1.96 1 100 1.96 1 2 2 2 where n is the sample size, N is the population size, a is the half-width of the confidence interval, and p is the estimated proportion of the population in the category of interest. Population Size Sample Size Proportion of Population in Category 95% Confidence Interval for Proportion of Population in Category Range Mean ±a Percentage Points, where a equals: Lower Limit Upper Limit 100 80 50% 4.9 pts 45% 55% 20% 3.9 pts 16% 24% 60 50% 8.0 pts 42% 58% 20% 6.4 pts 14% 26% 40 50% 12.0 pts 38% 62% 20% 9.6 pts 10% 30% 50,000 1,000 50% 3.1 pts 47% 53% or higher 20% 2.5 pts 18% 22% 400 50% 4.9 pts 45% 55% 20% 3.9 pts 16% 24% 100 50% 9.8 pts 40% 60% 20% 7.8 pts 12% 28% Note: SEE estimated using binomial distribution and sampling without replacement; normal approximation used to determine confidence intervals. For further information, refer to statistical textbooks listed in the bibliography. Population Size Sample Size for 95% Confidence Interval: Sample Mean ±a Percentage Points, Where a Equals: 1 pt 2 pts 3 pts 4 pts 5 pts 6 pts 7 pts 8 pts 9 pts 10 pts 100 99 96 91 86 79 73 66 60 54 49 200 196 185 168 150 132 114 99 86 74 65 500 475 414 340 273 217 174 141 115 96 81 1,000 906 706 516 375 278 211 164 130 106 88 2,000 1,655 1,091 696 462 322 235 179 140 112 92 5,000 3,288 1,622 879 536 357 253 189 146 116 94 10,000 4,899 1,936 964 566 370 260 192 148 117 95 20,000 6,488 2,144 1,013 583 377 263 194 149 118 96 50,000 8,057 2,291 1,045 593 381 265 195 150 118 96 100,000 8,762 2,345 1,056 597 383 266 196 150 118 96 200,000 9,164 2,373 1,061 598 383 266 196 150 118 96 500,000 9,423 2,390 1,065 600 384 267 196 150 119 96 *Sample sizes where proportion of population in the category of interest is 50%. Table 4-2. Approximate 95% confidence intervals for a categorical variable for a range of population and sample sizes. Table 4-3. Required sample size using random sampling for various sized confidence intervals and a range of population sizes.*

Statistical Concepts 45 For large populations of over 50,000, the required sample size for a 95% confidence level is given approximately by: ( )= −40,000 1 2 n p p a p Thus, for a population in excess of 50,000, if the proportion of some characteristic of the population is 5% (p = 0.05), say, and the desired accuracy of the estimate of this proportion is ±1 percentage point at a 95% confidence level, the required sample size to achieve this accuracy is 1,900. It should be noted that an error of 1 percentage point on an estimated proportion of only 5% is an error of ±20% of the estimated proportion. If it is desired to reduce this error to only 5% of the estimated proportion (±0.25 percentage points), the required sample size would increase to about 30,000. If a subgroup composes S percent of the population, and the estimate of the proportion of some characteristic of the subgroup is required to the same accuracy as the estimate of the proportion for the population as a whole, the sample will need to be larger by a factor of 100/S. Thus, to achieve the same accuracy for a subgroup that composes 20% of the population, the sample would need to be five times larger (100/20 = 5). If this level of accuracy is required for multiple subgroups, the required total sample size is given by the largest of the estimated total sample sizes calculated using the factor for each subgroup. For very small populations such as with airport tenant surveys, a high proportion of the population must be sampled to obtain an accuracy of 5 percentage points for estimated proportions of the total population, but actual numbers of surveys required are small. For example, a sample size of 79 is required from a population of 100 to achieve an accuracy of 5 percentage points. For large populations, a sample size approaching 400 is required to achieve a similar level of accuracy using random sampling. Numerical Response Questions For questions with a numerical response, such as the number of travelers in a group, expen- ditures at the concessions, or time spent at the airport, the sample size required for a specified level of accuracy is dependent on the variability (as measured by the standard deviation) in the numerical response. With random sampling, the population mean and standard deviation are estimated by the average and standard deviation (weighted if appropriate) of the responses in the sample. The required sample size for an accuracy of ±w can be calculated for a 95% confi- dence level using the following expression: =1.962 2 2n s w where s is the standard deviation of the responses in the sample. The SEE can be estimated by dividing the standard deviation of the sample values by the square root of the number of completed responses, less one. The standard deviation of each variable of interest is unknown during the survey planning stage, and an initial estimate is required to calculate the required sample size. This initial estimate could be obtained from previous surveys at the airport or from other airports or may be estimated from knowledge of the typical range in values. Examples of the mean, standard deviation, SEE and accuracy of estimate (95% confidence interval), and required sample sizes for accuracy to within 10% of the mean for selected air passenger characteristics from some airport surveys are given in Table 4-4. As can be seen by these examples, the accuracy and required sample sizes vary greatly depending on the variable of interest. Expenditures at the airport vary greatly because many

46 Guidebook for Conducting Airport User Surveys and Other Customer Research people do not spend any money and some spend a lot. Thus, large sample sizes are required to produce estimates to within 10% of their expected value. In contrast, variability in the time passengers spend at the airport is much smaller, and small samples would give a similar accuracy (in percentage terms). 4.5.2 Sample Sizes with Stratified and Cluster Sampling The methods for determining the sample sizes with stratified and cluster sampling are more complex and are outlined in the following paragraphs (details provided in Appendix A). Stratified Sampling The objective of stratified sampling is to reduce the size of the required sample to achieve a desired level of accuracy in situations where it is possible to define population strata within which the variance of the population characteristic of interest differs between the strata. For example, if the characteristic of interest is the duration of passenger air trips (because trip duration affects the likely use of parking at the airport), the duration values are likely to differ considerably between international trips, long-haul domestic trips, and short-haul domestic trips. Because the variance of the air-trip duration within each of these three strata will be much smaller than the variance for the population as a whole, it may be possible to estimate the average trip duration for all air passengers to the desired level of accuracy with fewer total responses divided between the three strata than by randomly sampling the entire population. To achieve a similar level of accuracy in the results for each stratum, it will be necessary to use nonproportional stratified sampling, with the sample size in each stratum inversely proportional to the variance of the characteristic within that stratum. Because the actual variance of the characteristic for each stratum will not be known until the survey has been performed, it will be necessary to make an initial assumption of the differences in the vari- ance across the strata in order to determine the proportion of the survey responses to assign to each stratum. These assumptions can be based on the results of prior surveys or of surveys performed at similar airports. If σXi is the standard deviation of characteristic X in stratum i, then for a confidence interval for the sample mean of X across the population of 2w (i.e., ±w) at a 95% confidence level, w is given by: ∑[ ]( )= σ −1.96 12 2w W n N ni Xi i i i i Variable Mean With Sample Size = 400 Sample Size for Confidence Interval ±10% of Mean Standard Deviation 95% Confidence Interval* Number in travel group 1.4 1.6 ±0.16 or ±11% 503 Expenditure at all concessions Airport 1 $6.20 Airport 2 $8.00 ±$0.84 or ±14% ±$1.86 or ±23% 728 2,168 Time at airport (min) Large international 160 60 ±6 min or ±4% 55 Domestic 106 43 ±4 min or ±4% 64 *Confidence interval expressed as difference from sample mean, also given as a percentage of the sample mean. Source: Adapted from Table 3-4 of ACRP Report 26 (Biggs et al. 2009). $8.53 $19.00 Table 4-4. Examples of 95% confidence intervals and sample sizes for selected air passenger characteristics from some typical airport surveys.

Statistical Concepts 47 where Wi is the proportion of the total population in stratum i Ni is the population in stratum i ni is the sample size in stratum i A given confidence interval can be obtained for varying combinations of ni. However, if ni is selected to be inversely proportional to the variance of X within each stratum (i.e., ni = k/σ2Xi), then ni can be replaced by k/σ 2Xi in the equation, which can then be solved for k and, hence, ni calculated for each stratum. The expression for calculating the value of k and the sample sizes for each stratum is provided in Appendix A. The total sample size is obtained by summing ni across all the strata. Cluster Sampling Calculating an appropriate sample size with cluster sampling is considerably more compli- cated than with random or stratified sampling because the composition and size of the clusters affect the variance of the resulting estimates of the population characteristics. The accuracy of a cluster sample depends on the variance of the characteristic of interest within each cluster and the variance between clusters. If the variation in the sample mean between clusters is fairly small (i.e., the clusters are fairly homogeneous and have similar means) but the variance of the characteristic within each cluster is fairly large, then the cluster sample will give a similar accuracy to a random sample of the same overall sample size. One can think of this situation as a series of small random samples of the population as a whole. Conversely, if the variance between clusters is fairly high, then the overall variance of the population sample mean of the characteristic will be larger than for a random sample, and in consequence, a cluster sample will require a larger overall sample size to achieve the same level of accuracy. 4.5.3 Comparison of Sampling Methods An example of sample size calculations for different sampling methods is given in Appendix A. The example provides some insight into the efficiencies of each sampling method and is summa- rized in this section. In the example, a survey of passengers is to be undertaken to obtain information on airport access trips. A critical question to be answered may be: What is the percentage of departing passengers dropped off at the terminal curb? Random and stratified sampling of passengers—with stratification by flight sector (e.g., short-haul domestic, long-haul domestic, international) and day of the week—and one- and two-stage cluster sampling—with both random and stratified sampling of flights by sector—are examined. The flight schedule for the survey period includes 610 flights per week, and the number of originating passengers per week is estimated at 48,300. Some 42% of passengers are on short-haul domestic flights, 34% on long-haul domestic flights, and 24% on international flights. From past experience, initial estimates of the percentages of passengers dropped off at the curb are 40% of short-haul domestic passengers, 60% of long-haul domestic passengers, and 90% of international passengers. In the example, the percentage of passengers to be dropped off at the curb is quite strongly related to the flight sector but fairly weakly related to the day of the week. Table 4-5 summarizes the required sample sizes for an accuracy of ±2, 3, and 4 percentage points for a 95% confidence level using various sampling strategies. The following observations were made from this example: • Using random sampling, the required sample size approximately doubles as the accuracy improves from ±4 to ±3, and doubles again from ±3 to ±2 percentage points.

48 Guidebook for Conducting Airport User Surveys and Other Customer Research • Stratified sampling by flights (which has a strong relationship with the variable of interest) reduces the sample size required by 15%, but stratified sampling by day of the week (which has a weak relationship with the variable of interest) has a negligible effect on the required sample size. • Cluster sampling with random sampling of flights and surveying of all passengers on those flights was found to be very inefficient, increasing the sample size required by a factor of 9 or more compared to random sampling. • Cluster sampling with stratified sampling of flights by sector greatly improves the efficiency of cluster sampling. – With all of the passengers on the selected flights surveyed, the sample size required is reduced to approximately 3 times that of random sampling. – With a random sample of 50% of passengers on each flight surveyed, the sample size required is reduced by 30% to 2.1 times that required using random sampling. However, with only 50% of passengers surveyed on each flight, the number of flights surveyed increases. – Several other percentages of passengers to survey on each flight were examined, and both the 30% and 75% levels resulted in larger passenger sample sizes. The optimal balance between the number of flights and the proportion of passengers on those flights to survey depends on the variation in responses between and within flights and on the relative costs of surveying passengers and flights, which vary from survey to survey. The results of this example reflect the assumptions regarding variation used in the example, and results would vary in other situations. In comparing the required sample sizes for different sampling methods, it should be borne in mind that true random sampling of air passengers is almost impossible to achieve (as discussed in Section 10.6.3). 4.5.4 Determining Desired Accuracy While the mathematics of calculating required sample size are generally fairly straight- forward, deciding on the appropriate desired level of accuracy is anything but, because it depends on the consequences of being wrong. Although it is common in statistical analysis to use a target accuracy of ±5% at a 95% confidence level, this is an entirely arbitrary choice and is typically not achievable or not accurate enough for many issues addressed by air passenger surveys. Method Unit Sampled Mean ±a Percentage Points, a Equals: Comment 2 pts 3 pts 4 pts Random Passengers 2,218 1,012 574 Random sampling of passengers (pax) Stratified Passengers 1,879 853 484 Stratified by sector of flight Passengers 2,215 1,011 574 Stratified by day of the week Cluster 1. Flights Passengers 252 19,953 146 11,560 92 7,285 Random sampling of flights with all pax on each flight sampled 2. Flights Passengers 83 6,560 41 3,280 24 1,910 Stratified sampling of flights by sector with all pax on each flight sampled 3. Flights Passengers 117 4,615 47 1,860 26 1,040 Stratified sampling of flights by sector with 50% pax on each flight sampled Table 4-5. Sample sizes in example survey for an accuracy of ±2, 3, and 4 percentage points for a 95% confidence level using various sampling strategies.

Statistical Concepts 49 Consider the case where the characteristic of interest accounts for only a small proportion of respondents, say air passengers using transit to access the airport, which from past surveys is estimated to be approximately 5%. The proportion using transit is to be estimated for a subgroup that composes 20% of the population (e.g., air passengers from a particular part of the region). If the required accuracy for the estimated proportion of this subgroup is ±5% of the estimated proportion (i.e., ±0.25 percentage points) at a 95% confidence level, a random sample survey would require a sample size of 150,000 responses, a level of effort that is impractical. Even accepting an accuracy of ±20% of the estimated proportion (i.e., ±1 percentage point) at the same confidence level, the required sample size would still be 9,500—potentially achievable, but significantly larger than most air passenger surveys. Therefore, determination of the required sample size should proceed by asking the following questions: • What are the critical characteristics of the target survey population that will drive decision making? • For what subgroups of the target survey population will these characteristics be required for decision making, and what proportion of the survey population do these subgroups compose? • What is the expected proportion of respondents with the critical characteristics in each of the subgroups? • For a range of different possible sample sizes, what is the expected accuracy of the estimated proportion of respondents with the critical characteristics in each of the subgroups? • What are the potential consequences if decisions are made on the basis of the estimated propor- tions of respondents with the critical characteristics, and these estimates turn out to be wrong by the magnitude of the expected accuracy for each of the different possible sample sizes? The final decision on sample size will involve a trade-off between establishing a reasonable sample size (and associated budget) for the survey and the resulting accuracy that is achievable for the various critical characteristics for each of the subgroups of interest. This trade-off may involve accepting a significant reduction in the level of accuracy that will be achieved for many of the characteristics and subgroups, particularly those accounting for a small proportion of the target survey population. 4.6 Weighting Most survey designs attempt to select a representative sample of individuals from the target population. However, in practice, the resulting sample rarely corresponds exactly to the com- position of the population. Because of the sampling approach adopted or the inevitable vari- ability in executing the planned sampling approach, some groups are over-sampled and some are under-sampled. The objective of assigning weights to the individual survey responses is to correct for these differences and improve the accuracy of the results. For random, sequential, and proportional stratified sampling, the number of sampled indi- viduals with a particular characteristic can be expanded to an estimate for the population by simply dividing by the sampling fraction. Thus, if 1% of passengers are surveyed, population estimates can be obtained by multiplying the numbers found in the sample by 100. Each response is therefore given a weight of 100, and it is these weighted values that are used in the analysis and preparation of results. For nonproportional stratified sampling, numbers found in the sample within each stratum must be expanded separately by dividing by the sampling fraction for that stratum and then summed to obtain estimates for the population.

50 Guidebook for Conducting Airport User Surveys and Other Customer Research Similarly, for cluster sampling, the numbers found in the sample for each cluster must be expanded separately, dividing by the sampling fraction for that cluster (if not all individuals in the cluster were sampled), then the sample cluster numbers expanded to population estimates. If the clusters were selected using random, sequential, or proportional stratified sampling, the numbers found in the sample for each cluster are summed and divided by the fraction of clusters sampled. Weighting can also be used in surveys where the sampling proportion varies over the time of day. For example, if the same number of interviewers is used over the day, the proportion of passengers surveyed in the busy periods will be much less than during the quiet periods, and peak-period passengers will be under-represented in the sample. This issue can be addressed by applying higher weighting to surveys collected in the peak period. The method for determining the weights will vary depending on the survey type. For example, for surveys of passengers exiting the security checkpoint, weights for surveys collected in a particular hour could be set equal to the total number of passengers going through security in that hour divided by the number of surveys collected in that hour. Rather than applying equal weights to all passengers or to all passengers within a group, weights can be applied so that the sample is more representative of the population. For example, weights could be set so that the distribution of surveyed passengers by airline match the actual distribution of passengers by airline during the survey period. In some cases, different sets of weights may be required for analyzing different characteristics of the population. For example, in a survey of passengers, questions relating to their travel to the airport and air trip are relevant to all members of the travel party as well as the respondent, but personal questions such as gender and age apply only to the respondent. By including questions on the number of travelers in the travel party and number of questionnaires com- pleted by others in the travel party, it is possible to define two sets of weights, one for the airport access and air trip characteristics and the other for personal traveler characteristics. Although the air travel party and those traveling together to the airport are generally the same, this is not always the case, as discussed in Section 10.3. People who have been attending an event such as a business meeting or conference may travel together to the airport but then take different flights, while others may travel separately to the airport and meet there to travel together on the same flight. The latter situation is particularly common with large air travel parties such as school groups or sports teams. Therefore, it is desirable for passenger surveys to ask how many people are traveling together on the same flight as well as how many people traveled to the airport together with the respondent. This additional detail will allow separate weights to be calculated for the air travel party characteristics and the ground access travel characteristics. There is additional discussion of weighting air passenger survey results in Section 10.8. In analyzing weighted survey results, it is important to calculate percentages using the total summation of the relevant weights to ensure that the percentages sum correctly to 100%.

5.1 Introduction This chapter describes the process of designing and implementing a survey after the initial steps and planning decisions that were discussed in Chapter 3 have been completed. Each part of the survey design and implementation process needs to be given careful consideration because the way in which each is addressed will affect the quality of the results as well as the costs of performing the survey. Considerations that are specific to particular types of airport-user surveys are discussed in more detail in subsequent chapters, which should therefore be used in conjunction with this chapter in planning and implementing a survey of a particular type. 5.2 Survey Population The results of a survey represent a sample drawn from the larger population, although in some cases the sample may consist of the entire population (often referred to as a 100% sample or a census survey). This section addresses the need to clearly define the population of interest, including defining the characteristics of the population required for survey planning and where this information can be obtained. 5.2.1 Defining the Population of Interest The first step in designing any survey is to define the target population about which information is to be collected. For example, the target population could be origin–destination passengers at the airport, in which case connecting passengers would be excluded, or all airline or airport- based employees using the airport terminal, in which case flight crews not based at the airport would be included. While the target population may seem self-evident at first, the exact definition will influence the survey methodology and sampling strategy and requires careful thought. The population available to be surveyed may differ from the desired target population, depending on the survey period and method used to conduct the survey. For example, the survey sponsor may wish to obtain information on all air passengers using the airport throughout the year. Performing a survey over a relatively short period obviously limits the available population to travelers during that period, whose characteristics may differ from those at other times of the year. Of course, if a survey is being performed to obtain air passenger characteristics during peak travel conditions for airport capacity analysis, then it will only need to be conducted for a relatively short period during those conditions. The limitations that result from when or how a survey is performed must be taken into account when interpreting the results. When the characteristics of the target population change throughout the year, as they generally do for air passengers, there is no way to know whether C H A P T E R   5 Survey Design and Implementation 51  

52 Guidebook for Conducting Airport User Surveys and Other Customer Research a survey performed over a fairly short period will provide a reasonable representation of average annual conditions. If information on average annual conditions is desired, it will be necessary to perform the survey over a number of different periods throughout the year to account for seasonal variations. In some cases, the individual members of the target population can be identified prior to the survey. For example, it will generally be possible to obtain a list of all airport tenants, and it may be possible to obtain a list of all airport-based employees. However, it will not be possible to identify every air passenger using the airport during a specific period. In the case of area residents or businesses, while it may be possible to define the entire population, in practice it will be impossible (for residents) or not feasible (for businesses) to assemble a comprehensive list. In cases where it is not possible to identify individual members of the population prior to the survey, it is often possible to obtain some information about the relevant characteristics of the population, although this information will not necessarily be organized in a readily usable format and may require some additional research. Some of this information may be prospective, such as flight arrival and departure times and the aircraft types assigned to each flight that can be obtained from the Official Airline Guide (OAG; https://www.oag.com/) or other sources of flight-schedule information. Other information may be historical and require extrapolation to the period of the survey. For example, fairly detailed information on air passenger trips in the United States by airline and market is available from the Bureau of Transportation Statistics of the U.S. Department of Transportation (https://transtats.bts.gov/; see bibliography), and there are generally similar sources of information in other countries. Design of an appropriate survey sampling plan (discussed in Section 5.3) requires a well- defined target population as well as information on the size of the target population. In the case of air passenger surveys, the types of data that can be used to determine the size of the target population include the following: • Enplaned/deplaned air passengers for the period of the survey • Aircraft movements by time of day and seat capacity, which are typically available from airport flight information systems, gate assignment systems, or tower records • Vehicle counts on airport roadways, which can typically be collected using automatic traffic counters • Parking exit counts and duration data from parking-revenue control systems • Curb activity counts In the case of employee surveys, the types of data that can be used to determine the size of the target population include the following: • The number of employee security badges issued by employer • Reports of the number of on-airport employees by airport tenants • Aircraft movements by aircraft type and airline (for estimates of flight and cabin crew) • Counts of employees at parking-lot exits These lists are not exhaustive. The exact data required to determine the size of the target population for a survey will be a function of the goal of the survey, the target population, and the type of survey being conducted. Additional details on defining survey populations are included in the chapters devoted to each survey type. 5.2.2 Identifying Sources of Information In addition to information on the overall size of the survey population, assembly of infor- mation on the composition and characteristics of the population will generally be desirable.

Survey Design and Implementation 53 This information will enable development of a more detailed sampling plan, as well as provide an indication of the extent to which the survey results correspond to the distribution of char- acteristics within the population. This in turn will allow appropriate weights to be assigned to the individual survey responses to correct for any sampling bias or so that the survey results can be extrapolated to the population as a whole, where it is determined that weighting of the survey results is necessary, as discussed in more detail in Section 5.3.4. In some cases, detailed data on the size or characteristics of the population may not be readily available. For example, an airport authority may not have traffic counts on the terminal roadways or may have no information on the number of air parties arriving at the airport by shared- ride vans or using off-airport parking. It is increasingly common for airports to use automated vehicle identification (AVI) systems to track the number of trips made by different classes of commercial vehicles, such as shared-ride vans or hotel shuttles. However, even in these cases, information on the occupancy of those vehicles is generally not available. Smaller airports may have no data on how many of these trips are made. Similarly, where airport employers provide parking for their employees, the airport operator may have no information on the number of parking permits that have been issued, much less the number of vehicles parked at the airport by those employees on a given day. While an airport operator will generally know how many parking permits it has issued to its own employees, it may not know how often they are used. Therefore, in planning a survey, one of the first steps is to decide what characteristics of the population are desirable to know in order to develop appropriate weighting factors and what data on these characteristics are already available. Where information on desired characteristics is not readily available, what would be involved in obtaining it? Answering this question may require some consultation with operational staff or external agencies. As it becomes clearer how much work would be required, a decision can be made on whether the benefits of improving the survey weighting process justify the effort involved, as discussed further in Section 5.3.4. 5.2.3 Determining Population Characteristics Although the total number of air passengers per month is generally known from airline reports, and the total number of airport employees is known from employment records, the more relevant information for developing an appropriate sampling strategy is how the flow of passengers or employees varies by time of day or day of the week. Because the number of surveyed passengers in any hour or the work- shift patterns of surveyed employees may not correspond to the overall distribution of activity, in cases where the characteristics of the target population being surveyed are believed to vary over time, it is desirable to know how passenger flows or employee shifts vary over the day and week. Some airports require airlines to report air passenger statistics for each flight. In this case, it is fairly straightforward to assemble data on passenger flows by time of day, although adjustments will need to be made for connecting passengers unless these are reported separately. In the more common situation, where airlines do not report passenger traffic at this level of detail, one approach is to analyze the distribution Variation in Ground Access Mode Use In principle, detailed information on the number of parking exits by time, together with how long exiting vehicles have been parked, should be available from the parking-revenue control system. However, at many airports, this information cannot be readily extracted from the database, and some manual analysis of printouts or even a sample of parking tickets may be necessary. Information on the use of other modes, particularly drop-off and pick-up by private vehicle, is more difficult to obtain. Counts of vehicle trips may be available from AVI systems or trip fees paid by the operators, but these counts will not necessarily correspond to air passenger use because of variation in air party size, more than one air party in a vehicle, and vehicles traveling to or from the airport without passengers (deadheading). It may be necessary to perform an occupancy survey to determine appropriate assumptions for the variation in vehicle occupancy over the week. These counts can be supplemented by installing traffic counters at strategic locations on the airport roadway during the period of the survey.

54 Guidebook for Conducting Airport User Surveys and Other Customer Research of seats on departing and arriving flights and make assumptions about load factors and the percentage of through passengers. Where passenger data at the level of individual flights are not routinely reported, it may be possible to obtain this level of information for the period of the survey from airline staff or by counting boarding passengers for surveyed flights. Data may also be available from the TSA on the hourly variation in the flow of people through the security screening checkpoints; these data can be used to refine assumptions about the variation in airline load factors at different times during the week. Similarly, it may be possible to obtain information from airport employers on the number of their employees who will start their shift at different times throughout the week for the period of the survey. In the case of airport ground access modes used by air passengers and airport employees, it would be helpful to know how the use of the different modes varies over the week, although at many airports these data are not routinely collected, and thus some analysis is usually required to assemble the necessary data (see “Variation in Ground Access Mode Use” text box). In some cases, the survey sponsor may not even know the overall size of the target population. For example, if the target population is local area businesses, the total number may not be known to the airport operator. Some research may be necessary to obtain data from external sources, such as city and county business license records or local Chamber of Commerce. While the effort required to assemble statistical information on the size and characteristics of the airport user population can be considerable, the information has other uses apart from survey design. Combining information on airport user characteristics with the survey data will produce a more integrated profile of airport activity. 5.3 Sampling Strategy and Plan 5.3.1 Determining Strategy After identifying the population to be surveyed and determining whether this population can be counted, the next step in the survey design is to determine whether a census or sample survey should be used. Depending on the type of survey, the following strategies and plans must be determined: • If a census survey is to be used, determine how each individual in the population will be contacted. • If a sample survey is to be used, determine: – The sampling frame (e.g., all airlines and nonstop destinations served). – The sampling method (e.g., random, cluster, stratified), taking into account the overall budget and the feasibility and efficiency of collecting data using each method. – The sampling plan (e.g., multistage sampling, choice of strata or clusters, methods of sampling within clusters). The sampling plan to be used is dependent on the type of survey being conducted (see Chapters 10 through 15). – The sample size for the level of accuracy required. Sampling methods are explained and discussed in more detail in Chapter 4. Whether a census or sample survey is used, steps for minimizing non-response should be identified in the sampling strategy, as discussed in the following subsection. Consideration should also be given to whether further analysis of the number of individuals not responding is required and overall estimates adjusted. The sampling plan to be used is dependent on the survey type being conducted; this is covered in the chapter for each survey type (see Chapters 10 through 15 for discussions of spe- cific survey types).

Survey Design and Implementation 55 5.3.2 Minimizing Bias The two main sources of bias in airport user surveys (followed by measures to reduce their occurrence) are: • Use of non-random sampling procedures. Bias can be reduced by the following measures: – Selecting an appropriate sampling method for the type of survey population, location, and time constraints. – Training interviewers in sampling techniques (in the case of interview surveys). – Adequate level of supervision of interviewers. – Developing backup plans for unexpected events affecting the sampling process. These events include delayed or cancelled flights, gate changes, and extreme events such as a terminal evacuation in the case of air passenger surveys. The details of these plans will depend on the nature of the events being considered. • Non-response of sampled individuals, since nonrespondents can have significantly different characteristics from those responding. Although no data are collected for nonrespondents, by definition, potential bias from the level of non-response can be minimized by reducing non-response rates. Response rates (also discussed in Section 5.9) can be improved by the following measures: – Choosing the survey location and time so that respondents have time available to respond. – Keeping the questionnaire as short as practicable, given the objective of the survey, as well as easy to understand and complete in the case of self-administered surveys. – Using well-trained, experienced, and friendly staff to conduct interviews (in the case of interview surveys). – Using multilingual interviewer staff and questionnaires, if appropriate. – Providing incentives such as pens or coupons for free coffee. Incentives are generally not required for airport user surveys but can help if respondents are significantly inconvenienced. Sampling of air passengers to survey can be particularly problematic because of their transience. Passengers arriving at the gate after boarding has commenced are typically under-represented and have a high non-response rate, while connecting passengers are often over-represented since they typically arrive at the gate well before the flight departure time unless their inbound flight was delayed. Many frequent flyers spend time in airline club rooms until their flight begins to board and thus spend very little time in the gate area. Appropriate sampling methods are discussed in Sections 10.3 and 10.6. When selecting air passengers to survey, it is important that survey field staff follow a defined protocol and take care not to allow the selection to be influenced by the characteristics of the passengers, such as their apparent willingness to participate, age, or gender. Allowing such characteristics to influence the selection is termed “respondent selection bias” and will intro- duce bias into the survey results. 5.3.3 Determining the Required Sample Size The number of responses obtained in a survey determines the accuracy of the survey results, as discussed in more detail in Chapter 4. Therefore, some thought needs to be given to how accurate the results need to be, which will determine how many responses a survey will need to obtain. As discussed in Chapter 4, accuracy can be expressed as the largest percentage point difference that could be expected, with a given level of confidence, between the proportion of survey respondents who give a particular answer to a given question and the true proportion in the air passenger population in total. Commonly used values for the desired accuracy and level of confidence are an accuracy of plus or minus 3 percentage points with a 95 percent confidence. However, in deciding

56 Guidebook for Conducting Airport User Surveys and Other Customer Research whether to use these values or some others, those planning a survey should ask the following questions: • What accuracy is needed for the answers to each question in the survey? • How confident do those planning the survey need to be that this accuracy has been achieved? The important point to remember is that any specified level of accuracy is meaningless without also defining the level of confidence with which that accuracy is achieved. The sample size needed to achieve a given level of accuracy will vary with the level of confidence with which that accuracy is achieved. As discussed in Chapter 4, the sample size needed to achieve a desired level of accuracy at a given level of confidence varies with the sampling approach adopted. The required sample size will be smallest if the sampling approach is able to select survey respondents randomly from the target population. However, in practice, this is difficult to achieve, and thus other sampling approaches are used, as discussed in Section 4.4. These will require a larger sample size to achieve a desired level of accuracy at a given level of confidence. In deciding what level of accuracy is needed, consideration should be given to the likely percentage of survey respondents who will report some characteristic or interest, particu- larly if this percentage is likely to be fairly small. This commonly arises with airport ground- transportation mode use, where there are many different modes, some of which are used by a fairly small percentage of those responding to the survey. Thus, if a mode of interest, say public transit, is only used by 3% of the survey respondents, expressing this percentage to an accuracy of plus or minus 3 percentage points means that the true percentage used in the target popula- tion could lie between 0% and 6%. Depending on the use to which the survey results will be put, this may not be considered an adequate level of accuracy. Similarly, in considering what level of confidence is appropriate, it may be helpful to ask what decisions will be made based on the survey results and how much confidence the decision makers will expect to have in the findings of the survey. In deciding on the sample size needed to achieve a desired level of accuracy (at a given level of confidence), consideration should be given to whether some survey findings are needed for subsets of the target population, such as air passengers making business trips or airport employees who use public transit to travel to work. This will reduce the proportion of total survey respondents who fall within these subsets and give a particular answer to a given question, requiring a larger overall sample size than needed to achieve a given level of accuracy for the survey respondents overall in order to achieve the same level of accuracy for answers to the particular question by the respondents in the subset of interest. 5.3.4 Weighting and Expansion Ideally, the survey methodology will select a sample of respondents who provide an unbiased profile of the target population, and no further adjustments to the survey results are needed. However, in practice this is rarely achieved. Therefore, in such cases, weighting factors, often simply termed “weights,” can be calculated for each survey response, with the goal that the results of the survey correspond more closely to the characteristics of the target population. These weights can be thought of as numeric factors for each survey response that, when summed across all the survey responses for a given answer to a particular question (as distinct from simply counting the survey responses), provide an equivalent count of survey responses that is the same as the count of survey responses would have been had the survey been an unbiased random sample of the target population.

Survey Design and Implementation 57 Thus, for example, if the survey had only sampled half as many respondents with a particular set of characteristics as would have been expected from a truly random sample, the weight for those respondents would be two. Calculation of survey weights requires the development of control totals, such as the number of enplaning air passengers by day of the week and time of day, or the total number of airport employees by employment category. These control totals are then used to calculate the weights by comparing the proportion of survey responses in each category with the proportion given by the control totals. Since the control totals will generally cover a range of different characteristics for the target population, separate weights may need to be calculated for different questions in the survey. The appropriate weight would then be used in any analysis. In some cases, if the survey sampling gives results that are fairly close to those that would have been given by a true random sample, calculating and applying response weights may not significantly change the results of the survey. In such a situation, weighting the survey results may not be thought necessary, and indeed many airport user surveys do not use response weights. However, without calculating some control totals and comparing them to the cor- responding data from the unweighted survey results, it is not possible to determine whether the use of response weights would improve the survey results or, if so, by how much. A particular type of survey response weighting is intended to express the survey results in terms of some measure of overall airport activity, such as average daily air passengers or total airport employment. This type of weighting is sometimes referred to as “survey expansion” since the totals of the weighted survey responses are usually much larger than the actual number of unweighted survey responses. However, presenting survey results in this way can be misleading since it may appear that the survey sample was in fact much larger than in fact it was. Furthermore, it may not be apparent that the number of weighted survey responses for relatively infrequent characteristics, or combinations of characteristics, in fact only represents a very small number of actual respondents, perhaps only one. For this reason, when present- ing survey results using expansion weights in this way, it is good practice to also present the unweighted response data so that users of the survey results are aware of the number of actual survey responses that were obtained for any particular combination of characteristics. Of course, it is quite possible to calculate weights for both purposes: to correct for any bias in the survey responses and to expand the results to correspond to some level of airport activity. In this situation, it is good practice to calculate separate weights for each purpose so that the survey results can be presented both as the actual number of weighted survey responses and in terms of the chosen measure of airport activity. 5.4 Questionnaire Design and Structure The design and structure of the survey questionnaire, including the wording of individual questions, is crucial to the success of a survey. Issues to be considered include what informa- tion to request, the order in which the questions are to be asked, how much detail to try to obtain, and the amount of time that respondents can be expected to spend completing the survey. 5.4.1 Length As the amount of information to be obtained by a survey or the level of detail desired for the responses increases, so does the length of the questionnaire. Once the decision is made to incur the cost of performing a survey, there is often a strong desire to increase the amount of information it provides. However, increasing the length of the survey may increase the refusal

58 Guidebook for Conducting Airport User Surveys and Other Customer Research rate and the number of incomplete responses, and also reduce the number of interview surveys that the field staff can perform in a given time period, thereby increasing the cost of the survey to obtain the same number of responses. Another consideration with long surveys is that respondents may get fatigued, bored, or anxious to get on with some other activity and unintentionally give incorrect answers or give less care to their answers, leading to poorer-quality survey results. There are a number of practical limitations on survey length. The most obvious is the time that respondents are willing to spend answering the questions. This length of time will depend in part on the circumstances. Someone completing a survey questionnaire in the comfort of their office or home will generally be willing to answer more questions, and in greater detail, than some- one who is standing in a busy airport terminal and is anxious to catch a flight. The survey methodology may also impose limitations on survey length. If the survey questionnaire is a printed form that is to be completed by hand, it should take up no more than two sides of a single sheet of paper. The text has to be large enough for respondents to read, and the form has to provide enough space to write in the answers. Checkboxes should be large enough and with adequate space between them that it is clear which box the respondent is intended to check. 5.4.2 Response Options Survey questions fall into two broad categories: closed-ended questions and open-ended questions. Closed-ended questions are queries in which the respondent is offered predetermined answer choices. Open-ended questions are ones for which there are no predetermined answer choices, and the respondent is free to answer in his or her own words. Closed-ended questions have two subtypes based on the nature of the response options: • Numeric, in which respondents are asked to provide a number, such as a date, a time, or a number of people or bags • Categorical, in which respondents are asked to select one or more choices from a list, such as ground access mode used or concessions visited The results of open-ended questions are much more difficult to analyze, but they may provide richer information because the respondents are not forced to select from a limited number of categories. For many applications, it is common to use a hybrid form in which respondents are presented with a set of categorical responses, one of which is “other” with an option for an open-ended response that respondents are asked to provide (sometimes referred to as an “other–specify” option). This allows common responses that were not covered by the categorical options to be assigned their own category code after the fact. Also, “other” responses that really should have been one of the defined categories can be recoded. However, adding a category after the survey based on the “other” responses can result in an under-reporting of that category because some respondents who would have selected that option if it had been presented chose a defined category instead. This occurrence is less of a problem with an interview survey, where the respondents cannot see the defined categories when they answer the question, and the interviewer can code the response appropriately, than it is with a self-administered survey. Best Practices “[It is] important to keep questionnaire length down. [The] time sensitivity of potential respondents [will vary] depending on where they are: at the gate they’ll have more time versus arriving passengers trying to get to their car.” —Research participant

Survey Design and Implementation 59 In addition to being easier to analyze, categorical questions have the advantage that they are generally quicker to answer because they typically involve just checking one or more boxes or selecting the appropriate option(s) displayed on the screen. Also, because they present the respondent with a predefined set of possible responses, they encourage the use of standardized terminology, yet may also prompt a response that would not otherwise have been mentioned. For example, if the trip purpose options include “attend a conference or convention,” this might be selected by respondents who would otherwise simply give their trip purpose as “business.” While this is true for self-administered questionnaires, there is a potential disconnect with interview surveys, where the respondent does not see all the options, and the interviewer assigns the response provided to one of the defined categories. Different interviewers may handle a similar response in different ways. This phenomenon is called “inter-rater reliability,” and it can be a particular problem when asking about ground transportation modes, because different respondents may refer to the same mode in many different ways. One solution is to provide interviewers with printed cards that list the defined options; these can be shown to respondents to help them provide an appropriate response. Interviewers can also be instructed to read all the predefined options, although this can get unwieldy if there are a large number of such options. Categorical questions and the responses obtained from them may include the following types of problems: • The respondent checks multiple boxes when asked to check only one. The wording should make it clear whether one or multiple boxes should be checked. This wording should not be part of the question and should stand out. Online surveys and those using electronic data-collection devices (discussed in Section 5.10) eliminate this error. • A categorical response for “not applicable” or “no opinion” is not provided where some such form of non-response is appropriate. When using a rating scale for opinions such as 1–5 or 1–7, be careful to word all the questions in a consistent way so that the highest number corresponds to the most positive opinion and 1 corresponds to the most negative opinion. Consideration should be given to including a comment box after each group of questions to allow respondents to note any clarifications or other relevant information. 5.4.3 Question Wording The wording of questions is critical to the success of a survey. Respondents who misunder- stand a question are not going to provide the desired information. Worse, it may not even be clear that they have given an answer to a different question from the one intended. Similarly, interviewers who misunderstand a question may miscode the response. Therefore, considerable effort should be devoted to developing clear and unambiguous questions. Consider the question, “How far do you live from the nearest airport?” Is the question asking how far in terms of miles? Travel time? Travel time by car or by transit? And what kind of airport is meant: the local general aviation airport? The nearest airport with scheduled passenger service? Getting from the general and vague to the specific is both necessary and difficult. Airports with years of experience conducting user surveys still investigate possible refinements to their questions. Unfortunately, the easiest way to discover that a question is problematic is to look at the resulting data. Preventing this after-the-fact problem requires a serious commitment of time and effort to planning, thoughtful consideration of possible answers, and thorough testing of questions before the survey is deployed.

60 Guidebook for Conducting Airport User Surveys and Other Customer Research There are two broad categories of questions: • Factual questions • Opinion questions Factual questions ask for factual information that the respondent should be able to provide (such as how many bags they checked or how they got to the airport), while opinion questions seek the respondents’ views on an issue. Opinion questions present respondents with a range of options so that they can select the one that best describes their opinion. This type of question may take the form of a statement, with the respondents being asked how strongly they agree or disagree. Satisfaction questions that explore the respondents’ satisfaction with particular facilities or services are a subcategory of opinion questions. Wording concerns with factual questions largely revolve around ensuring that the intent of the question is clear to the respondents and that the descriptions used for categorical ques- tions are unambiguous. For example, difficulties can arise over local terminology that may not be familiar to visiting air passengers, such as the names of different ground transportation services (discussed further in Section 10.7). Question clarity is particularly important with self-administered questionnaires, where there is limited opportunity for respondents to clarify the intent of a question or ask how their response should be classified in terms of the response categories provided. The challenge with opinion questions is to ask the question in a way that allows for a mean- ingful answer. Such careful wording is particularly important with questions that ask respon- dents to indicate how likely they would be to use some proposed facility or service or their satisfaction with some existing facility or service. Because the likelihood of using a facility or service depends on the circumstances affecting the decision, such questions have to be framed in terms of a specific situation, such as the trip that an air passenger is currently taking, or the last time they arrived at or returned to the airport, if they would have used the facility or service on the current trip. Similarly, because satisfaction with a given facility or service is influenced by expectations and the respondent’s experience with the use of the facility or service, customer satisfaction questions need to be worded in a way that allows these influences to be identified. For example, a longer than usual time spent in the security screening queue may be considered as quite acceptable at peak times when it would be viewed as unsatisfactory at less busy times. Care is needed in providing response options to avoid unbalanced scales that may result in responses having a positive or negative bias. A balanced scale provides an equal number of positive and negative response options, for example: How likely is it that you will visit one of the restaurants in the airport on your trip today? Definitely will Probably will Probably will not Definitely will not Not sure There are a number of other issues that need careful thought in designing questions that seek respondents’ opinions or qualitative answers, as discussed in the following paragraphs.

Survey Design and Implementation 61 Biased questions or response options. Biased question wording invites a particular response; for example: What type of seating do you prefer in the boarding gate area? Comfortable individual chairs that you can arrange to suit your needs Rigid, fixed chairs in rows that you share with other passengers High stools at tables with power outlets for laptop computers Do you feel that the widely spaced tables in the food court provide adequate separation from other diners? Do you feel that the shops and restaurants at the airport offer enough variety? Please indicate your age: 25 or less 25 to 35 35 to 45 45 to 55 55 and over Biased response options favor a particular response, as in the following example: Double-barreled questions. A double-barreled question is one that asks effectively two different questions; for example: A respondent may want to answer “yes” for shops but “no” for restaurants. Answer choices that are not exhaustive or are mutually exclusive. It is not always possible to specify every possible answer, either because the options are too numerous (e.g., names of hotels in the local area) or it is not known what answers respondents will give, but where the specified response options are not exhaustive, the option “other” should always be given, with a request for the respondent to state their answer in their own words. These can then be coded or reclassified as appropriate. Specified response options should be mutually exclusive, particularly with numeric ranges. The following example illustrates response options that are not mutually exclusive: Order effects. The order in which response options are presented can influence the selection. In the following example, respondents may select “business” without reading far enough down the list to see that “conference or convention” would have been more appropriate.

62 Guidebook for Conducting Airport User Surveys and Other Customer Research Putting “conference or convention” first and following this with “other business purposes” would avoid this potential bias. This also avoids the ambiguity of whether “business” includes attendance at a conference or convention. Order effects must also be taken into account when presenting a list of attributes of an airport facility or service to be assigned a numeric rating (e.g., attributes of food/beverage concessions, such as variety, price, comfort, service, and value for money). Varying the list of attributes by rotating the sequence or presenting them in random order can help avoid: • Top-down bias: Early attributes in the list receive more careful evaluation than later attri- butes in the list. • Last-in-order bias: If the last item in the list is an “overall” evaluation, the attributes imme- diately preceding the “overall” evaluation in the list may have an undue impact on the “overall” evaluation. Appropriate scale lengths and scale labeling. Having too many response options on a numbered scale (e.g., 1 to 10) can make it difficult for respondents to decide which value to choose and can create ambiguity in interpreting the results. Having five levels on a numbered rating scale generally works well and has been widely used in airport customer satisfaction surveys (as shown in Chapter 10, Table 10-1). Providing labels for all options on a numbered response scale can help respondents decide which intermediate value to select. In the following example, respondents may have difficulty deciding where an assessment between “poor” and “excellent” fits on the scale: What is the purpose of your current trip? Business Vacation Visiting family or friends Traveling to/from school or college Conference or convention Other (please state): _____________________ 5 Excellent 4 3 2 1 Poor Rating and opinion scales raise a number of issues that need careful thought; these are dis- cussed in more detail in the next subsection. Rating versus ranking. Asking respondents’ preference for options or importance of different factors on a numeric rating scale (which should be specified) is usually much better than asking them to rank the options or factors. Ranking tends to force respondents to express a relative preference or importance between options or factors that they may feel are equal. Of more importance, numerical ranking values reflect order, not magnitude, and thus summary statistics, such as averages, have no meaning.

Survey Design and Implementation 63 Placement of non-substantive response options. Response options such as “don’t know,” “not sure,” “no opinion,” and “not applicable” should always be listed after the specified options. These should be spelled out and not shown as abbreviations (which respondents might not recognize). “Not applicable,” if included, should be the last option listed. Sufficient space for open-ended responses. When respondents are asked to provide answers in their own words or explain “other” responses, there should be enough space on a printed questionnaire or in the response field of an online survey or questionnaire (on a computer or electronic data-collection device) for a reasonably detailed answer. Rating and Opinion Questions It is common with airport customer satisfaction surveys to ask respondents to evaluate a set of attributes of facilities or services, such as the cleanliness of restrooms or the variety of food concessions. Such questions typically offer a numeric scale for responses (such as 1 to 5), where a higher number indicates a better rating or evaluation, or a one- or two-word description of each step on a scale of poor to excellent (or similar wording), often with a numeric value shown for each step for use in analysis. Numeric scales should have labels [such as 1 (poor) to 5 (excellent)] to indicate which way the scale goes. Practice varies as to whether every value on a numeric scale has a label or only the end points, with respondents left to interpolate their rating between the designated end points of the scale. This may simply reflect preference on the part of survey designers, a desire for simplicity, or a concern with the use of labels for intermediate points, for the reasons discussed later. A different type of question presents statements about airport facilities or services and asks respondents to indicate their agreement with the statements on a scale of “strongly disagree” to “strongly agree” (or similar wording). A related type of question may ask respondents how likely they would be to use a proposed service or facility if it were to be provided. In contrast to rating scales, each step on the scale should have a descriptive label because a numeric value alone is not meaningful. In general, such a scale will be bidirectional, with a neutral value (e.g., “neither agree nor disagree”) at the midpoint of the scale and an equal number of steps indicat- ing increasing disagreement or reducing likelihood. Such scales raise a number of issues that need careful thought, including: • How many values (steps) to use for the scale, • How to word the labels, especially the end-point labels (is “poor” the same as “terrible”?), and • Whether to use labels for interior values on the rating scale. Having more values in a scale allows finer discrimination, although this may make it more difficult for respondents to decide which value corresponds to their assessment, and any apparent improvement in discrimination may be illusory. One may need more values on a bidirectional scale (since the values on either side of the midpoint represent changes in different directions) than on a rating scale that represents a progressive improvement from the low end to the high end. A 5-point scale has been widely adopted for rating scales in airport customer satisfaction surveys. However, for agreement or likelihood scales, a 5-point scale only gives one interme- diate value between neutral and the extreme value in each direction, so a 7-point scale may be preferred to provide a little more discrimination. Using an even number of values for an agreement or likelihood scale eliminates the neutral midpoint value, forcing respondents who do not feel strongly either way to take either a positive or negative position. Whether this is helpful depends on the issue being addressed and whether it is useful to know the proportion of respondents who are indifferent to the issue or have no opinion either way—considerations that should be taken into account when choosing between scales with an odd or even number of values.

64 Guidebook for Conducting Airport User Surveys and Other Customer Research The choice of label wording (if used) for interior values of rating scales should reflect a steady progression in the evaluation from the lowest value to the highest, as would be the case with a numeric scale with labels only at the end values. There are two issues to consider. The first arises from the limited choice of words that can be used and be understood by the survey respondents. On a scale from 1 = poor to 5 = excellent, where should “good” go? Should this be 2, 3, or 4? This will affect what labels can be used for other points on the scale. The second issue is that the choice of label for each value will affect the ratings that are given. If a respondent feels that a given attribute of a facility is “good,” and “good” is the label used for the value 3 on a 5-point scale, then that respondent will give that attribute a rating of 3. On the other hand, if “good” is the label used for the value 4 on the scale, then the respondent will give that attribute a rating of 4. There are two important points that follow from this. The first is that the choice of wording of labels on rating and opinion scales matters and deserves careful thought. The second is that the numerical rating for a given attribute that is obtained in a survey will depend on the wording of the labels used for each point on the scale. Thus, care is needed when comparing numeric ratings obtained from surveys that use different labels for each point on the scale. For a more detailed discussion of these issues, a number of reference books address the nuances of question wording, such as Bradburn, Sudman, and Wansink (2015) and Schuman and Presser (1996), each cited in the bibliography. 5.4.4 Question Order and Interview Flow The following considerations affect the order in which the different questions should be asked: • The most obvious consideration is where the answer to one question affects subsequent questions or questions to be skipped. For example, it is important to determine whether an air passenger is starting a trip or connecting between flights before asking questions about the ground access trip to the airport. • A more subtle but equally important consideration is to introduce requests for information in a logical sequence. Asking survey respondents the type of place from which they began their trip to the airport gets them thinking about where they started their trip and leads natu- rally to questions about the location of that trip origin, such as the city or zip code. Earlier questions can also help clarify the intent of subsequent questions. Asking how many people are traveling together clarifies subsequent references to the travel party (however this is phrased), such as how many bags the travel party checked. • A third consideration is to obtain as much key information as possible if it is likely that the respondents may be unable to complete the survey. Asking these questions earlier in the survey makes it more likely that they will be answered. • A fourth consideration is that most surveys involve some branching that depends on the responses to earlier questions. These branches or skip patterns can request more detailed information for certain responses or omit questions that do not apply based on the responses given to previous questions. In the case of printed questionnaires, these skip patterns should not be too complex or respondents or interviewers will have difficulty deciding where to go next in the questionnaire and may miss key questions or attempt to answer questions that do not apply. The order of the questions is one way to simplify the skip patterns. Branching is less of a concern with programmable tablet computers or other electronic data-collection devices because the software handles the skips based on the responses to earlier questions. However, complexity can significantly increase the cost of the required programming. • In some cases, the wording of questions can be customized based on answers to previous questions. This can make questions clearer or easier to answer. Where respondents are being presented with alternative options to choose between (referred to as “stated preference” questions), the descriptions of the choices, such as the time or cost involved, can be customized so that the options are more realistic or plausible for the respondent’s circumstances.

Survey Design and Implementation 65 Another less technical suggestion is that the questionnaire should start with a clear intro- duction to the purpose of the survey and be followed by a question that is easy to answer and non-threatening, such as: “Did you travel to this airport by ground transportation to take this flight or are you connecting between flights?” This will help in getting the respondent’s cooperation for the survey. 5.4.5 Translations In the case of surveys where respondents may have limited facility with English (or with the primary language of the area where the survey is being performed, if this is not English), consid- eration should be given to providing the questionnaire in other languages. This situation arises at airports serving bilingual or multilingual areas and at international airports, as discussed further in Section 5.9.3. It is essential to have the translation performed by a native speaker of the language in question, because there may be subtle issues of usage that could affect how the questions are interpreted. It is also desirable to have the resulting translation reviewed by an aviation specialist with knowledge of the language to make sure that the translator has understood the intent of the questions. It should be recognized that there are often cultural differences in response to scales used for expressing opinions that go beyond simply translating the terms used. These cultural differences are not always well understood but should be taken into consideration when comparing or combining survey results obtained through the use of translated questionnaires. The decision as to whether translations are necessary or worth providing will depend on the proportion of the target population that may have difficulty completing a survey in English. If this falls within the anticipated confidence interval for the survey as a whole, as discussed in Section 4.3, it may not be worth incurring the cost of the translations to include these users in the survey. However, even in this situation there may be other reasons for including non-English- speaking airport users in the survey, such as gathering information about foreign tourists or demonstrating concern for non-English-speaking airport users. There are usually practical limits to the number of different languages that the question- naire can be translated into, and the choice of languages will depend on the composition of the target market. Aside from the cost of the translation, each version of the questionnaire should be field-tested, and consideration needs to be given to who would do the testing. Because the field staff may not be fluent enough in any of these languages to explain the nature of the survey, ask the questions, or understand the responses, the translated versions will most likely have to be self-administered. If so, a brief addition to the questionnaire may be necessary to explain its purpose, which in turn may change the layout of a printed questionnaire. It may be helpful to use colored paper to help the field staff distinguish between the different versions of a printed questionnaire. 5.5 Expected Data-Collection Rate Factors affecting the rate at which interviewers can collect responses include the following: • Questionnaire design—length, format, and types of questions. • Types of information collected—opinions usually take longer than current factual information. • Surveying method—intercept interview or self-administered. • Competency of interviewers—good interviewers can complete significantly more surveys. • Airport layout—time needed to move between gates or between terminals, considering likely congestion and the availability of moving walkways and inter-terminal transportation.

66 Guidebook for Conducting Airport User Surveys and Other Customer Research • Refusal rates—affect the number of potential respondents who must be approached to obtain a completed response. Collection rates differ significantly depending on these factors, but a rough guide to the expected average survey completion rate per hour per interviewer is as follows: • Intercept interviews: – 6 to 10 per hour for a 3- to 4-page survey with about 25 questions – 10 to 15 per hour for a short questionnaire (10 to 15 questions) – 15 to 20 per hour for a very short questionnaire (5 to 10 questions) and introduction • Self-administered: 25 to 40 per hour These rates are applicable to situations where there are passengers available to interview, and they include an allowance for typical gaps between interviews but exclude breaks and inactive periods such as between flights. Disruptions to the planned survey schedule due to events such as flight delays, cancellations, and gate changes will affect the number of completed interviews collected in a given shift due to the time needed to move to a different gate or the passengers on a cancelled flight not being available to interview. Flight cancellations can be addressed by designating backup flights to survey if a planned flight is cancelled. Flight departure delays present a different situation. On the one hand, passengers may be waiting in the gate lounge longer than expected, increasing the opportunity for completed surveys. On the other hand, if the delay is substantial, some passengers may leave the gate area (or not go to the gate) until closer to the revised departure time, and the survey field team may need to move to another assigned flight well before the delayed boarding time. Survey procedures need to provide survey field staff with clear instructions on what to do in the case of flight delays. Survey response rates affect the overall data-collection rate because refusals take up inter- viewer time, and passengers who do not complete survey questionnaires that are handed out reduce the number of survey responses obtained from a given flight. Where all passengers on a flight are approached to participate in the survey, experience indi- cates that responses are typically received from 40% to 60% of passengers. The non-responding passengers are mostly those who: • Arrive at the gate after the first boarding call. They either cannot be interviewed in the short time available or decline to participate as they are getting ready to board the flight. • Decline to participate because they are engaged in activities such as working, reading, or talking with others. In addition to passengers who decline to participate in a survey because they are engaged in activities they do not wish to interrupt or are getting ready to board a flight, some passengers simply choose not to participate in a survey for various reasons, including an aversion to being surveyed. For groundside surveys, refusal rates generally range from 5% to more than 30%, depending on the survey type and method and the time of day. 5.6 Survey Logistics 5.6.1 The Importance of Logistics The survey quality and the physical well-being and mental attitude of the survey field team will be directly affected, positively or negatively, by the amount of thought given to logistics. One of the principal challenges with air passenger interview surveys is that they are conducted in

Survey Design and Implementation 67 the physical environment of the airport terminal, which is often crowded and noisy. Surveys in airport terminals involve a lot of standing, and those conducted in airline gate lounges usually involve quite a bit of walking from gate to gate. Surveys that are conducted in the secure area of the terminal will require the survey field team members to be able to obtain security clearance. Careful attention to survey logistics is therefore critical to a successful passenger survey. 5.6.2 Survey Implementation Team Of first importance is the survey team. In addition to the strong project manager recommended earlier, the survey implementation team needs a technical expert, a field manager, field super- visors, and interviewers. This structure applies whether the survey is conducted in person or by handing out questionnaires. • The technical expert is charged with addressing any deviations from the survey design and making the most scientifically appropriate choices when challenges arise. • The field manager oversees operations and serves as a liaison to the project manager. • The supervisors oversee the daily conduct of the survey in the field. • The interviewers either ask questions and record answers or hand out questionnaires or tablets, answer questions when appropriate, and collect completed questionnaires or tablets after their use. Interviewers will need to be trained on what types of questions they can answer and what answers to give, as well as what types of questions they are not permitted to answer. Ideally, members of the survey team will be quick on their feet and effective problem-solvers. These characteristics will be helpful when things go wrong, which is likely to happen every day. The variety of things that can go wrong is essentially endless. Flights can be cancelled or delayed. Interviewers may not show up or may be late, or may show up in inappropriate attire. Lines may too long for interviewers to get where they need to be when they need to be there, or interview locations may be too crowded. Gate personnel may refuse access to survey staff. Passengers may get annoyed or complain. Security breaches or weather can shut down the entire operation. Labor disruptions may hinder airport operations. Every problem requires a solution as quickly as possible. It is therefore critical that the survey team develop contingency plans for everything from a late-arriving interviewer through weather problems to special events that either cause havoc at the airport or skew the passenger profile. Survey teams that do not do this in advance are likely to find themselves scrambling madly on a daily basis. Table 5-1 lists some common problems and possible plans for dealing with them. These plans are general in nature and should be tailored to the specific situation. 5.6.3 Other Logistical Considerations Field Office Space Survey teams that do not plan for adequate, appropriately equipped, and accessible field office space are likely to see the quality of their work and results suffer as a result. The first consid- eration is where the space should be located: before or after security screening. This decision should be driven by what type of access will be most convenient to field staff and least disrup- tive to the airport. The space should also: • Provide enough room for the field manager, field supervisors, and interviewers to check in, check out, and meet (perhaps for announcements or some refresher training). It need not accommodate the entire team, although a space for training the entire team will need to

68 Guidebook for Conducting Airport User Surveys and Other Customer Research be identified and reserved. Ideally, the field office would be a place where interviewers can rest during breaks away from the noise and crowds. This will make them happier and more productive. • Have the necessary technology and equipment, including Internet access, sufficient electrical outlets if a large number of electronics will need to be charged, an office laptop computer or tablet, a photocopier if needed by the team, and storage space for supplies and equipment. • Be private (not shared with others) and locked, both to prevent conflicts over use of the space and for security reasons. Ideally, there would be a place where survey field staff could safely leave personal effects such as handbags or purses. Communications The best and easiest way for survey staff to communicate with one another (interviewer to supervisor, for example) is by cell phone (or walkie-talkie in any cases where cell-phone coverage is not reliable). This eliminates wasted time walking around trying to find someone. Provisions need to be made for sufficient devices if survey staff members do not have their own cell phones or do not want to use them for work-related calls, as well as for charging cell phones as needed. Parking As airport personnel are well aware, parking is expensive. Arrangements for parking for members of the survey team who are not on the airport’s staff need to be made well in advance so they are in place when the survey starts. Provisions also must be made for parking tags, vouchers, or validations for those working early or late shifts when airport managers are not available. If arrangements are made for off-site parking, the team will need to consider the extra time staff will need to get to the airport, both in terms of scheduling and in wages. Weather Conditions If the survey is being conducted outdoors (such as a survey of meeters and greeters or one of drivers), plans should be made in case of inclement weather. In hot weather, it is important to Potential Problem Possible Contingency Plan Target flight is delayed Prepare a list of alternate flights to survey in case selected flights cannot be surveyed as planned; if departure delay is greater than 30 minutes, survey alternate flight; if departure delay is less than 30 minutes, continue interviews until team is scheduled to move to next flight and leave one interviewer behind to continue until boarding starts then rejoin team at next flight Target flight is cancelled Survey designated alternate flight Severe weather disrupts flight schedules Shift interview schedule by a designated amount of time to allow as many flights as possible to be surveyed in the planned sequence Interviewers fail to show up or arrive late Reallocate available interviewers among teams to ensure that each team has a balanced number of interviewers Interviewer runs out of printed survey questionnaires Record responses on an already-used questionnaire in a different color Airline suspends operation due to strike Survey designated alternate flight Table 5-1. Representative problems and contingency plans.

Survey Design and Implementation 69 provide water. In the cold or rain, provision needs to be made for shelter and possibly for extra breaks for team members to warm up. Material and Equipment Appendix D provides a checklist of supplies and equipment that will be required for the typical air passenger survey. This list can be adjusted to meet each survey’s needs and circumstances. Scheduling of Interviewers At least two shifts per day will usually be necessary because of the long hours that airports operate. The staffing schedule should avoid staff being assigned to a late-night shift followed by an early-morning shift. The schedule should also allow enough time for staff to move between the different survey locations. Depending on local employment regulations, a standard 8-hour shift will need to include paid breaks and a (usually unpaid) meal break. Time spent checking in, picking up equipment, returning equipment, checking out at the end of the shift, and moving to and from survey locations will all be part of the paid shift, so the actual time spent conducting the survey will be less than 8 hours. In some cases, it may be possible to have staff work overtime, although there are cost implications, and interviewer productivity and effectiveness are likely to decline with longer shifts. Consideration also needs to be given to local regulations on when time spent working is considered overtime that requires additional pay. In cases where flights depart late at night or only operate on certain days in the week, it may be more cost-effective to schedule an interview team for a partial shift to cover these flights than to include them in a regular shift. Partial or split shifts can also be used to help cover peaking of flight departures. 5.7 Selection and Training of Field Staff 5.7.1 Quality of Field Staff When hiring temporary staff, it is important to remember that generally you get what you pay for. The more the airport is willing to pay for interviewers, the higher the level of competence of the inter- viewers who will respond and, with careful selection of candidates, the higher the quality of data that will be collected. The length of the temporary commitment is also important. Trying to get competent and experienced interviewers for 1 or 2 weeks of work will be consid- erably more difficult than getting and keeping the same interviewers for 2 months. Dealing with a contractor for temporary staff can introduce a different dynamic. Some contractors will have a pool of experienced, quality interviewers, but others will have just as much difficulty finding and keeping interviewers as the survey sponsor would. There will also be a mark-up associated with the contract. The competence of the inter- viewers is a function of the final rate of pay, not how much is paid for their services to the contractor. It may seem that a high labor rate should attract good interviewers, but it is the final wage being paid to the interviewer that will attract quality interviewers, not the rate listed by the contractor. Obtaining Survey Interviewers Given the difficulty of attracting well-qualified interviewers, particularly for a fairly short duration, the best approach may be to retain a market research firm to provide the survey staff, preferably a firm with extensive airport survey experience. Such firms will generally be able to call on interviewers that they have used for other projects or that they employ on an ongoing basis. They can be made responsible for scheduling the field staff and providing field supervision. They may also have interviewers working on other projects who can be assigned to the survey as needed to handle peak periods or replace absent interviewers. Interviewers who have a long-term working relationship with a particular firm are also much less likely to quit unexpectedly.

70 Guidebook for Conducting Airport User Surveys and Other Customer Research Qualities to look for when selecting interviewers include: • Professional and educational background. • Experience in conducting airport surveys. (Experience in conducting surveys at other transportation facilities can be a substitute.) • Presentable dress and appearance. • Motivation, a drive to perform, and a willingness to take up a challenge. • Strong interpersonal skills, friendliness, outgoing nature, and a willingness to approach people. • Ability to understand what data are being collected and why. • Willingness to pay attention to detail to ensure good data are collected. • Ability to cope with long periods standing and lots of walking. • Comfort using electronic devices (if such devices are to be used). • Fluency in the desired language(s). • Ability to obtain a security clearance (when surveys will be conducted in the secure part of the airport). 5.7.2 Interviewer Training If interviewers are to be used to conduct the survey, they will need to be trained. Telephone surveys are likely to be conducted by a call center already staffed with trained interviewers. Training is not an area in which the survey team should be looking for cost savings. Although many factors contribute to the success or failure of an intercept survey, skilled interviewers are mission critical. In addition, there are so many unique aspects to airport surveys that the training session should be mandatory, even when interviewers have already been trained for other surveys by an outside vendor. The technical expert and field manager are likely to be the persons providing the training, and all field supervisors should attend. It is a good idea for the project manager to attend as well, because sometimes that person can answer questions no other team member can. In addition, the project manager can identify any areas where the training is insufficient and point these out to the trainer before it is too late. Training should always take place at the airport. This location will not only facilitate the airport tour (discussed in the following subsection) but will also expose interviewers to the conditions in which they will be working. It is better for interviewers who are taken aback by the distances involved in getting from gate to gate, or who are surprised by the prices of the food at the airport concessions, to find out sooner rather than later. 5.7.3 Content of the Training Session A sample agenda for a training session is provided in Appendix D. This outline can be expanded, shortened, or modified to meet a survey’s particular needs. After introductions of the survey team and the interviewers, the training should begin with a discussion of the purpose, goals, and objectives of the survey. Ideally, these should be considered in the context of actions to be taken and decisions to be made; enthusiasm for the project on the part of the project manager is also beneficial at this point. If people understand why they are doing what they are doing and how important it is, they are more likely to be motivated to do a good job. Having spent some time on “what’s in it for the survey sponsor,” the next part of the training should be devoted to “what’s in it for the interviewers.” People are always eager to learn about

Survey Design and Implementation 71 such basics as work hours and days, shifts, pay rates, and the like. Once they know what they are going to get out of participating in the survey data collection, they will be more likely to pay close attention to what follows. The next part of the training should deal with expectations, including everything from pro- ductivity and quality to behavior, attire, and grooming. People need to know what the rules are so that they can put what they are supposed to do into the appropriate context. It is important to stress that interviewers should not be checking their phones for emails or texts while they are conducting interviews. This is a major source of distraction as well as giving those being interviewed a poor impression of how seriously the interviewers are taking the interviews. The next part of the training is typically devoted to basic interviewing skills. Even if the project is being staffed with previously trained and experienced interviewers, it is wise to include this topic in the training by labeling it as a review. The quality of interviewer training varies considerably from firm to firm, and it is possible that the survey team will have higher standards than the firm from which the interviewers are drawn. After the interviewers understand the requirements of the job, it is time to go over the specifics. In this part of the training session, the questionnaire should be reviewed question by question. Any questions that could prove challenging for interviewers to ask or for respondents to answer should receive special attention. After the questionnaire has been reviewed, the operation of devices that will be used in the survey (such as tablets or other electronic data-collection devices) should be taught. A tour of any airport facilities or services that are referenced in the survey, whether passenger amenities or ground transportation services, should also be included. This tour will also orient staff to the layout of the airport and the ways to get from one place to another. By the end of this part of the training, interviewers should know exactly how to conduct the survey. The final step is to have them practice, first by interviewing one another and then by conducting practice interviews with passengers or other target respondents. The results from the latter should be reviewed by supervisors to ensure that they are accurate and complete before the interviewers are assigned to conduct actual interviews. 5.7.4 Duration of Training If the interviewing team is focused, it is possible to conduct an acceptable training session in a day, and a 1-day training session appears to be the general practice. There is, however, a large amount of material to master, even for experienced interviewers, and fatigue can be a problem. In an ideal world, training would probably be spread over 2 days, with perhaps 6 hours of instruction and practice per day. While 2 days may seem costly, the expense could well be offset by increased efficiency and accuracy of the interviewers during the actual survey, particu- larly during the first few days. Having interviewers familiarize themselves with the questions and survey procedures during the actual survey not only reduces the survey completion rate (increasing the cost per completed survey) but also runs the risk of generating poor-quality data until the interviewers gain experience. 5.7.5 Coaching and Retraining Even the best training session is not foolproof, and even a highly competent interviewer can fail to grasp an important point. It is therefore extremely important that everyone’s work be checked on a daily basis, particularly during the first few days of the project. Consideration should be given to who will do the checking and what issues they should be looking for.

72 Guidebook for Conducting Airport User Surveys and Other Customer Research It may then be necessary to take corrective action, either by coaching an interviewer who missed something during training or by retraining the entire group if the trainer failed to convey a point well enough. The survey planning team would be wise to budget for both of these occurrences, particularly if the questionnaires or procedures are complicated or difficult. It is also important either to train backup interviewers, who may or may not be deployed, or to provide for a second training session for new hires if people need to be replaced. As it is difficult to find people who are willing to be trained without any assurance that they will have the opportunity to participate in the survey, the latter approach is more typical. However, a second training session can present scheduling difficulties because attrition of the trained interviewers can occur throughout the survey. Another problem arises with recruiting and training additional interviewers during the course of the survey if they have to be issued with security badges, because this process can take quite some time. Therefore, it may be better to recruit and train enough interviewers at the start of the survey to allow for some attrition and adjust the length of the survey period and the hours worked by each interviewer to manage the attrition that actually occurs. Finally, it is important to note that when mistakes happen, the decision about whether to retain and tabulate work that is imperfect is both a research and a policy decision. The research decision has to do with the necessity of having enough interviews to achieve the desired statistical precision and power; the policy decision has to do with the importance of a given piece of information. It is also important to consider just how imperfect less-than-perfect work really is. 5.8 Pretests and Pilot Tests Pretests of questionnaires (or survey questions in the case of an online or telephone survey) and pilot tests of survey procedures provide an opportunity to identify any problems with question wording or other aspects of the questionnaire as well as the planned arrangements for performing the survey. Such tests allow corrections or adjustments to be made before the main data-collection effort. In the following discussion, the term “questionnaire” includes the survey questions in the case of online or telephone surveys. 5.8.1 Pretest of Questionnaires The goal of a pretest of the questionnaire is to make sure the survey questions can be asked easily by the interviewers (in the case of interview surveys), are clear to the respondents, and produce the desired information. Pretests also provide an opportunity to identify unanticipated responses or situations and to adjust the question response categories or the survey script as necessary. An initial pretest may be performed on a convenient group of people who were not involved in developing the questionnaire, such as staff of the survey sponsor. However, while this may help improve the question wording and identify unanticipated responses that require changes to the question response categories, it is not a truly representative test of the questionnaire and should be followed by a pretest involving a sample of the intended respondents of the survey, performed in the same way as planned for the survey itself. This pretest will increase the chances of identifying unanticipated responses or situations. The questionnaire designers should be involved in conducting the pretest because they will be able to identify unanticipated responses or any misunderstanding of the intent of a question better than someone who was not involved in the development of the questions. The results of the pretest should be subject to careful analysis to identify any apparent dif- ficulties with the question wording or survey flow, such as missing or incomplete answers,

Survey Design and Implementation 73 illogical responses, or incorrect skips. In particular, responses of “other” to categorical questions, where the respondent has provided an explanation, should be examined to identify any mis- understanding of the categories or commonly occurring responses that should be added to the designated categories. Those conducting the pretest should be debriefed following the test to find out how things went, what issues were encountered, whether things got awkward at any point, and so on. If electronic data-collection devices are to be used in the survey, the programming needs to be thoroughly tested and debugged before the pretest, or the validity of the pretest may be compromised by errors in the programming. A rigorous testing plan for the software should be included in the project schedule. Similarly, in the case of online or telephone surveys, the programming of the questions and question skip logic needs to be thoroughly tested, although generally the survey questions will be programmed using commercial software, so the software itself will not need to be tested. This program testing, done by people other than the programmers, will attempt to ensure that all possible response options and question branches are tested. This can be quite time-consuming and can be facilitated by preparing a set of response scripts that test all possible branches from each question. These test scripts ensure that certain response options are not overlooked and can also be used to check that changes made to fix any problems have not affected other parts of the program. The survey schedule should allow enough time to adequately test the program- ming and make any needed revisions before the first pretest. 5.8.2 Pilot Test of Survey Procedures The primary purposes of a pilot test of survey procedures are to identify any problems with the planned approach to conducting the survey and to refine the estimates of the number of field staff required. Because a pilot test typically involves performing the survey on a represen- tative sample of potential respondents, it presents an opportunity to conduct a pretest of the survey questionnaire, and the two tests are often combined. Other objectives of a survey pilot test include: • Testing the survey sampling strategy, • Validating the sampling plan, • Identifying problems with survey logistics, and • Performing checks (or additional checks) of data quality. The pilot test also presents an opportunity to make sure that any changes to the questionnaire as a result of a pretest have satisfactorily resolved any issues identified in the pretest. Feedback from the pilot test should include field observations by supervisory staff and issues reported by field staff in debriefing sessions, supplemented by an analysis of the survey responses collected. In the case of interview surveys, noting the start and end time of each response will enable this analysis to examine the average time to perform the survey as well as the interval between ending one interview and starting the next. This will provide information on the amount of time spent waiting for subjects to become available to be interviewed as well as the time required to move between locations. It can also indicate an issue with interviewers pausing unduly between interviews, which should be flagged for supervisor attention. Field staff should also note refusals to participate and record the reason where given or apparent, such as a language barrier or cell-phone conversation. The pilot test results should be compared to the expected number of responses from the sampling plan. This comparison may require collecting data (such as the number of passengers on flights surveyed in airline gate lounges) on the number of potential respondents.

74 Guidebook for Conducting Airport User Surveys and Other Customer Research The results should be examined for data quality, including missing or incomplete informa- tion, apparently illogical or inconsistent responses, and difficulty interpreting answers to open- ended questions. Where addresses or other location data are collected, these should be examined to ensure that the location can be identified. While this analysis addresses many of the same issues as the survey pretest, it focuses on the conduct of the survey rather than the design of the questionnaire. The results of the pilot-test analysis can be used to identify issues requiring particular attention in training field staff for the data collection phase. 5.8.3 Scheduling of Pretests and Pilot Tests Pretests and pilot tests should be performed far enough in advance of survey data collection that there is time to adequately analyze the results, resolve any issues, and perform any addi- tional testing required. This process is likely to require several weeks to do well. Any changes to the questionnaire have to be finalized before the forms are printed for the data collection phase and may involve reprogramming electronic data-collection devices (if these are used), which will also take time. Therefore, it is prudent to schedule the pretest and pilot test at least a month before the main data collection. The exact amount of lead time must be carefully considered in light of the time required for the intermediate steps, which will vary from survey to survey. 5.8.4 Quantity of Pretests and Pilot Tests In general, one questionnaire pretest and one pilot test of survey procedures will be suffi- cient, unless the two tests are combined. However, if a pilot test reveals problems that require significant changes to the questionnaire or procedures, a second pilot test should generally be performed to verify that the changes have successfully resolved the problems. When a second pilot test is not performed, the experience and results of the first day or two of the main data- collection period should be closely scrutinized to ensure that any changes have produced the intended effects. 5.9 Maximizing Response Rates The willingness of potential survey respondents to participate in an airport user survey varies with the survey method and type of survey, as well as the caliber of the survey field staff. If they have time, air passengers are generally cooperative. The response rates for other types of surveys, such as surveys of airport employees or tenants, can be improved by the way the initial contact is undertaken and the justification given for requesting the information, as discussed further in the following. 5.9.1 Techniques that Improve Response Rates Survey response rates, defined as the proportion of potential survey respondents asked to participate in a survey who agree to do so, can be improved through various measures, depending on the type of survey being performed. Intercept Interview Surveys Response rates for intercept interview surveys can be improved by the way the initial request is communicated. The quality, experience, and training of interviewers can significantly influ- ence response rates. Interviewers should remember to smile and offer a friendly greeting when approaching potential survey respondents and thank them at the end of the interview. Survey

Survey Design and Implementation 75 personnel should be clearly identified as performing an officially approved function by wearing identification badges (these will be necessary anyway if the survey is being performed in the secure part of the airport terminal) and professional attire. It is helpful for survey personnel to wear distinctive clothing, such as vests or aprons marked “airport survey” (or similar wording) and the name of the survey organization, that identifies them as performing a survey. Clear identification will: • Assure potential respondents that the survey is an officially sanctioned activity; • Help prevent airport, airline, or security staff from becoming suspicious about why the survey staff are approaching people; and • Simplify the initial explanation when the potential respondent is approached. High-visibility safety vests add to the safety of interviewers in groundside locations where vehicles and pedestrians mix. Vests and aprons can also be designed with large pockets to make it easy to carry questionnaires, pens, and so on. Survey staff should be trained to follow a standard introductory script that explains the purpose of the survey, which organization or organizations it is being performed for, and how the information will be used. If the survey involves sensitive or identifying information, such as the respondent’s address or income, assurances should be given that this information will only be used for statistical analysis and will not be divulged outside the survey team. As inter- viewers gain experience, they can adjust their introduction to respond to varying situations and the mood of the respondents. However, field supervisors should make sure that the intro- ductions retain the important points and do not become too casual. Passengers arriving at the gate after the first boarding announcement who are reluctant to complete an interview survey should ideally be provided a mail-back questionnaire, as discussed later. While response rates are typically low, they do provide some responses from this passenger segment. The questionnaire should include a serial number or other information identifying the day and flight number, and this information should be recorded so that the numbers of mail- back surveys handed out and returned can be tracked, and appropriate weights can be assigned. Telephone Surveys Telephone surveys are in many ways more difficult than intercept interview surveys, although they are less physically demanding on the interviewers. Because neither the interviewer nor the respondent can see each other, it is more difficult for the interviewers to tell whether they have reached an appropriate respondent. Also, many people are so tired of telephone solicitations masquerading as surveys that they are likely to be quite suspicious, if not hostile. Therefore, the initial introductory script is all the more critical. It may be helpful to send an advance letter, where addresses for target survey respondents are available, explaining the purpose of the survey and indicating that a follow-up call will be made. Obviously, this is not possible with telephone surveys conducted using random-digit dialing. Other fairly standard practices that can help improve response rates include starting each call with a friendly greeting, stating the expected length of the survey (if this is fairly short), identifying a specific time to call back if the current call is inconvenient or the desired respondent is not available, and making multiple follow-up calls at different times if no answer is received or participation is declined. Mail Surveys Mail surveys do not face the problem of contacting potential respondents as long as the mailing addresses are correct. However, response rates from mail-back surveys are generally much lower than for telephone surveys because there is no direct personal contact. The survey

76 Guidebook for Conducting Airport User Surveys and Other Customer Research should be accompanied by a cover letter explaining the purpose of the survey, preferably signed by an appropriate official of the sponsoring organization, such as the airport director, and indi- cating how the survey benefits the recipient and the requested response date. In the case of airport employee surveys, it may be preferable to have the cover letter issued on the letterhead of the employer and signed by an appropriate official, at least for large employers. A response date should be set to allow a reasonable time for completion of the survey and for follow-up reminders to be sent if a response is not received. There is a rapidly diminishing response rate to follow-up reminders, so it will generally not be worth sending more than two or three reminders. In cases where there is a relatively small target sample, such as airport employees, it may be worth making follow-up telephone calls and performing the survey by telephone if the recipient is available, rather than sending follow-up letters. Although the cost of making telephone calls, and in particular conducting the survey by telephone, is significant, this only has to be done for recipients who have not responded to the initial request (and perhaps one follow-up reminder). Mail-back surveys that are handed out to potential respondents, such as to air passengers in an airport departure lounge, do not generally permit any follow-up unless contact information has been obtained in the course of distributing the survey. In this case, response rates may be improved by offering some inducement to participate in the survey. One technique that might be considered is providing a pen with the survey questionnaire that is marked with the name of the survey. This inducement will not only facilitate completion of the survey but will also serve as a reminder to do so. Asking potential respondents whether they would be willing to complete a mail-back survey before handing them the questionnaire may also help improve response rates by creating an implied commitment on the part of the respondent. With all mail-back surveys, providing a prepaid return envelope—or designing the question- naire so that it folds to show the return address and prepaid postage—is essential to increasing response rates. Consideration may need to be given for questionnaires to be returned from inter- national destinations, such as using International Business Reply Service envelopes. Although they are more expensive than using regular mail, costs are incurred only for those returned. Online Surveys Respondents to online surveys are generally contacted by email to request their participation. Follow-up reminder emails can be sent at regular intervals to those who have not yet responded, although this produces the same diminishing returns as reminders to mail-back surveys. With a limited and well-defined target sample, it may be more effective to follow up by telephone, or even to call before sending the first request by email. Such a call will help ensure that the correct respondent has been identified and prepare the respondent for receiving the survey request. A telephone call will also avoid sending unnecessary reminders in cases where the respondent does not wish to participate in the survey. Although making telephone calls is much more expensive than sending email reminders, it may be worth incurring this cost to obtain a better response rate, particularly if the target sample is fairly small. The survey software should give the respondent the opportunity to save partly completed responses and return to complete the survey later, particularly for longer or more complex surveys. 5.9.2 Refusals and Incomplete Surveys It will generally be useful during the course of a survey to record refusals and monitor incomplete surveys. Monitoring incomplete surveys should distinguish between the following:

Survey Design and Implementation 77 • Those where the respondent terminated the survey before the end, for reasons such as the need to board a flight or undertake some other activity, but the survey was complete up to that point. • Those where the survey was largely completed but some questions that should have been answered were not. Comparison of refusal rates and incomplete surveys across interviewers in the case of an interview survey may identify field staff members who have a higher refusal or incomplete rate than average and could benefit from closer supervision or additional training. Incomplete surveys may still provide useful information for those questions that were answered, and the data analysis phase of the survey should consider how best to make use of this information. For example, responses for completed questions in an incomplete survey can be included in frequency tabulations of those questions, although it will not be possible to include them in cross-tabulations with questions that were not answered. Where the contract for a survey requires the contractor to meet a certain target number of completed surveys, a decision should be made on which questions must be answered before a survey is considered complete. 5.9.3 Other Languages The two language issues to be considered regarding survey populations that do not speak English are the languages spoken by the target airport users to be interviewed and the local languages of the area. This primarily affects air passenger surveys and surveys of local area resi- dents, since airport employees, concessionaires, tenants, and representatives of local area businesses will generally have good enough English to respond to a survey. If the airport is serving a bilingual or even multilingual area, at least some interviewers will need to speak those languages, particularly if the area is officially bilingual, as is the case in Canada. However, a requirement for bilingual or multilingual interviewers will increase the wage rate that will need to be paid. If the survey results are to be truly representative of the population, the multilingual nature of the area must be reflected in the survey process. Although there are no officially bilingual areas in the United States, there are certainly areas where a significant fraction of the population has limited or no English-speaking ability. As well as multilingual interviewers, this process calls for multilingual questionnaires and other handouts. Translations must be of excellent quality to maintain the meaning of each question and response option, as discussed in Section 5.4.5. Where only a small proportion of the local population cannot communicate effectively in English, the need for multilingual interviewers becomes less clear and a decision is needed on whether obtaining responses from non-English-speaking potential respondents justifies the cost involved. In the case of a flight-based survey where it is decided to provide questionnaires in other languages, ideally at least one bilingual or multilingual interviewer should be available for each flight surveyed. The second language issue concerns international passengers. As the number of people flying internationally continues to grow, more and more languages are being spoken at airports, particularly at major-gateway international airports. This prevalence of multiple languages is a real problem for surveys and one for which there is no easy solution. The survey planning team must determine whether failing to include non-English-speaking potential respondents in the survey will be a concern and attempt to arrive at a workable solution. Where inter- national passengers are primarily from a small number of countries, such as those on a flight to a non-English-speaking destination, bilingual or multilingual interviewers could be scheduled to survey the appropriate flights. It will not usually be feasible to translate questionnaires into

78 Guidebook for Conducting Airport User Surveys and Other Customer Research many languages, but if most passengers who cannot speak English speak a common language, a translation in this language could be developed. One important consideration with international passengers is what proportion of foreign visitors from a particular country speaks enough English to easily understand the survey questions and give appropriate answers. It would be helpful to undertake some analysis of the number of refusals due to language problems in previous surveys (if this information is available). If the survey team is conducting a strict sampling process, the language barrier may interfere with this process and lead to some bias in the results. A related issue is that travelers whose first language is not English may appear to understand the questions, but in fact do not fully under- stand them and give incorrect answers. It is important that interviewers be trained to identify such situations and that they devote time to ensuring that the respondents understand the questions and that they understand the responses. A benefit of using a multilingual questionnaire is that it will allow the survey team to examine whether the characteristics of non-English-speaking respondents are significantly different from those of English-speaking respondents, and thus how important it is to make special arrange- ments to include these respondents in future surveys. 5.10 Use of Electronic Data-Collection Devices There has been a steady trend toward using tablets or other handheld electronic data- collection devices (EDCDs) for performing interview surveys and observation studies. These devices have a number of distinct advantages over printed questionnaires but also raise several issues that need to be carefully considered. 5.10.1 Advantages and Disadvantages One of the most obvious advantages of using EDCDs is the elimination of the data-entry task and the associated potential for errors to be introduced in the data. EDCDs store the recorded data directly in a database that can typically be downloaded and combined with the corresponding data from other devices. Questions are displayed and responses are entered using a program running on the device. This can reduce interviewer recording error and allow question skip logic to be programmed based on respondent answers. While the capabilities of commercial software for data capture vary, most if not all programs provide the capability of exporting the data in a format that can be read by or imported into commercial statistical analysis software packages and spreadsheet software. This capability can represent a significant cost and time savings as well as eliminate a potential source of error. It also allows survey data to be available more rapidly. Some survey sponsors have raised the concern that the absence of a printed questionnaire means that there is no opportunity to go back and check the validity of the data in the data file. It is true that there is no opportunity to go back to check the validity of data, but this is no different from using printed questionnaires. There is no way with printed questionnaires to check whether the response that was recorded on the questionnaire was in fact what the respondent said or intended. The use of printed questionnaires does allow the data-entry task to be checked for errors, but this is an irrelevant consideration with EDCDs because the task is eliminated. The electronic data files should of course be backed up regularly so that they can be restored if they become corrupted. There is also the possibility that failure or loss of an EDCD could result in the loss of the data stored on the device. This possibility is no different from the possibility of loss or accidental

Survey Design and Implementation 79 destruction of paper questionnaires before the data have been extracted from them. Modern EDCDs are quite robust and preserve their data if they lose power. Survey procedures should provide for periodic downloading of the data on EDCDs. Some devices can upload and back up data as soon as they are captured when a cell-phone network or Wi-Fi connection can be maintained. In the final analysis, if a survey sponsor wants a paper backup to the electronic data file, one can always be printed out. One caveat is that with printed questionnaires, well-trained interviewers sometimes make notes on the questionnaires to clarify responses or suggest cautions in interpreting the response. Depending on the software, programming EDCDs to provide the same capability may be possible, but entering these comments typically takes longer on an EDCD than on a printed questionnaire. A difficulty with using EDCDs is that they cannot be used to survey passengers who are hurrying to catch a flight or anxious to board a flight that had started boarding, because (unlike printed questionnaires) they cannot be handed out to respondents to take with them. Inability to survey such passengers can result in biased information if these passengers have different characteristics from those interviewed, which of course will not be known if they are not included in the survey. This issue can be addressed by providing interviewers with mail-back questionnaires to hand to passengers in this situation. Because the response rate from mail-back questionnaires is typically quite low, the number of such questionnaires handed out should be recorded on the EDCD (including the serial number of the questionnaire and where distributed) so that appro- priate weights will be assigned to those questionnaires that are returned. There are obvious cost considerations in having printed questionnaires available for respondents who cannot be interviewed in the usual way, so consideration needs to be given to how important it is to include these potential respondents in the survey. Use of EDCDs introduces additional costs for purchasing or renting the devices and the soft- ware and programming the devices. There will also be logistical issues regarding availability and recharging of devices. To some extent, the requirement to be able to use EDCDs efficiently can limit the applicants when selecting interviewer staff and can introduce the need for additional training. These issues are discussed in the following subsections. An additional advantage of using EDCDs is that the results of the survey can be available more quickly since they can be uploaded to a database as the survey proceeds. Indeed, this allows rapid analysis of early responses that can identify any unforeseen issues with survey design or question wording. One potential disadvantage of using EDCDs is that respondents cannot see how long the survey is and therefore may be more likely to decline to participate. As noted in the following subsection, this can be partly offset by stating how long the survey will take at the outset. Survey Design and Data Quality An important advantage of EDCDs is that they allow more complex branching based on responses to prior questions and can tailor subsequent questions to information already pro- vided. While branching can also be done with printed questionnaires, it can quickly become unwieldy and confusing if there are more than a few such branches. There is no such limitation with an EDCD program, and the resulting logic is transparent to the user, which helps minimize interviewer error. Furthermore, survey questions can be tailored to individual situations, and the questionnaire can include questions that apply to only a few respondents without affecting the questions asked of others.

80 Guidebook for Conducting Airport User Surveys and Other Customer Research The ability to tailor questions on the basis of prior answers can also allow the introduction of questions that are only applicable to some respondents by skipping those questions that are not applicable without increasing the chance of interviewer error with the skip logic. Also, with printed questionnaires, the questionnaire may appear very long to the respondent and adversely affect the response rate, but the length of the survey is not evident to the respondent when using EDCDs. However, for this reason, it is generally advisable to inform respondents of the likely time required to complete the survey. Another advantage is the ability to do real-time data checking and ask follow-up questions to clarify apparently inconsistent responses. This ability allows the interviewer to correct mis- understandings or mistakes while the respondent is still available. In other cases, it may provide an explanation for what appears to be inconsistent information but is in fact an unusual or unanticipated combination of circumstances, such as different members of the same travel party returning on different days and some members returning home by transit although they came to the airport by a private vehicle that was parked at the airport. This is a powerful feature. For example, as more detailed information becomes available about air passenger travel patterns, it has become clear that the range of traveler behavior is much wider than is often assumed. Therefore, trying to fit the full range of behavior into only a few simplified categories can lead to a misunderstanding of the situation. Cost and Technical Support Considerations Although EDCDs provide powerful capabilities, the units themselves can be fairly expensive, and a large number may be required for a given survey, although this is becoming less of a concern with the availability of relatively inexpensive tablet computers. Survey firms and airports conducting frequent surveys will generally acquire their own equipment and allocate the cost over many different surveys. It may be cost-effective for sponsors conducting surveys infrequently to rent units. In this situation, it may be less costly for the survey planning team to purchase a few units for questionnaire development and post-survey use but rent additional units for the period of the survey, depending on the relative cost of purchase versus rental. There is also the cost of the survey software to be considered. Depending on the software used, license fees can cost as much as the units themselves. However, the cost of the equipment and software has to be balanced against the savings in data-entry costs, both for the current survey and for future surveys that will use the equipment. While the technology involved in using EDCDs will be familiar to anyone with a smart- phone, some technical support will be necessary, particularly for programming the units and downloading the data. Survey firms with experience using EDCDs should be able to provide this level of technical support, either with their own staff or through a subcontract with a technical specialist who has appropriate programming experience. Some airports may have in-house information technology staff with the necessary expertise, while others may need to contract for this support. 5.10.2 Choice of Equipment The choice of equipment for EDCDs involves trade-offs between cost, capability, and ease of use. With increasing capabilities in terms of memory, screen size, and battery life, tablets are becoming widely used for survey data collection. Custom devices that have been designed specifically for use in survey data collection are also available. These generally have larger screens and keyboards but are typically bulkier and heavier and are thus harder to use. The ability to hold the unit in one hand while entering data with the other is an attractive feature of tablets. Consideration should also be given to the need for staff to hold the units for extended periods

Survey Design and Implementation 81 of time while standing; therefore, compactness and light weight are distinct advantages. On the other hand, a small screen limits the font size and the number of response options that can be displayed on one screen, while a compact keypad is more awkward to use and more likely to lead to mistyping. Tablets typically display a touch-sensitive virtual keyboard on the screen that can be used for text and data entry. This usually has a standard keyboard layout that facilitates entering text responses. However, the virtual keyboard takes up screen space, which reduces the space available to display the questions. Smartphones are another option and have become widely available and are easier to hold than tablets. However, they typically have a smaller screen, and their virtual keyboards are more difficult to use and more likely to result in mistyping, although some smartphones (sometimes referred to as “phablets”) now have a larger format. While smaller devices may preclude their use for interview surveys, they may be satisfactory for online surveys, including self-administered surveys where respondents use their own smartphones to complete the survey online, since survey respondents will generally be comfortable using their own smartphones. These are the principal factors to consider in selecting equipment: • Screen size and clarity • Data storage capacity • Software that can be used • Touch-sensitive screen for data entry • Physical keypad versus virtual keyboard • Key size and keypad or keyboard layout • Battery charge life and recharge time • Weight and handling • Data export capability • Wi-Fi or cell phone capability • Suitability for use outdoors in rain or cold weather (where this is required) • Cost Laptop computers have been used for some survey applications where complex, graphically intensive screen displays need to be shown to respondents. However, these are awkward to hold for extended periods and would require either that respondents and interviewers be seated during the interviews or that interviewers be provided with a mobile cart to support the unit. Use of Respondents’ Own Devices The widespread use of smartphones, tablets, and laptop computers presents the opportunity of having survey respondents complete the survey online using their own devices. Respondents will need to be provided with a link to the survey website, which can be handed out on a printed card, and the survey will need to be conducted in a location where there is an adequate Wi-Fi connection with enough bandwidth to support the number of survey respondents anticipated at any one time. The card with the link to the survey website can include a QR code to allow respondents to simply scan the code to connect to the survey. Of course, this approach also implies that a survey website will need to be established and programmed, although this is also true if EDCDs are programmed to use an online survey rather than have the survey programmed on the device itself. Since most respondents using their own devices will typically be using a smartphone, which can have a fairly small screen, the survey website will need to be programmed with this in mind. The obvious advantage of allowing respondents to use their own devices is that this approach allows a much larger number of surveys to be conducted by a given number of survey field staff, since they only have to ask the respondents to participate in the survey and hand out a card

82 Guidebook for Conducting Airport User Surveys and Other Customer Research with the link to the survey website, and not spend time conducting the survey itself. It also significantly reduces the number of EDCDs required compared to handing out EDCDs to survey respondents, although it may be appropriate to have a few available for respondents who do not have, or do not wish to use, their own devices. Such an approach can be conducted in conjunction with the use of a printed self-administered questionnaire, which can include the link to the survey website for those who wish to use their own devices. This allows respondents to complete the survey using whichever approach they feel most comfortable with. One potential disadvantage of having respondents use their own devices compared to an interview survey by field staff is that there is much less opportunity for respondents to ask clarifying questions to the field staff or for interviewers to ask clarifying questions about apparently questionable responses. However, this is also true with surveys using EDCDs that are handed out, or indeed with any self-administered survey. This disadvantage can be partly offset with online surveys or surveys programmed into EDCDs by incorporating response checks into the survey programming that can trigger clarifying questions. 5.10.3 Programming Considerations Selection of software to program EDCDs involves the choice between purchasing com- mercial software and developing a custom program using a suitable programming language. Commercial software is available that has been designed for programming surveys on EDCDs, particularly tablets and smartphones. The more advanced software generally includes the capability to present the usual types of questions and response formats, including categorical checkbox questions and free text input, as well as to program skips to different questions based on the responses to prior questions. However, this software often has limited ability to display sophisticated graphics or custom screen layouts. Thus, for applications that require this capa- bility, it may be necessary to develop a custom program. In this situation, most survey sponsors will retain a survey firm that already has this capability. Selecting an appropriate commercial software package involves deciding which program features and capabilities are required and which packages support the EDCDs being considered. The choice of EDCD and software should be approached as an integrated decision because such issues as screen size and keypad design interact with how the software displays the questions and allows the responses to be entered. As with any other software and hardware, capabilities are continually evolving and hardware costs are dropping, so the survey planning team or contractor will need to assess the currently available technology before making a purchase. Some questions to consider when reviewing commercial software are: • What question formats are supported? • Can the software combine different types of question format (e.g., checkbox responses and text answers to open-ended questions) in the same screen display? • How does the software handle questions that have more response options than will fit on a single screen? • What are the limits to the number of response options a given question can have? • Can the user customize the response code that is used for each categorical response for a given question? • Can the user customize the order that categorical responses appear on the screen? • Can the software rotate or permute (vary randomly) the order in which categorical responses or attributes being rated appear on the screen?

Survey Design and Implementation 83 • Are there limits to the number of questions? • Does the software work appropriately on a tablet or other EDCD whether the device is held vertically or horizontally? • Does the software allow the respondent or interviewer to back up to a prior question and change the response? • Does the software indicate to the respondent or interviewer how far along the survey is and how much remains until the survey is completed? • Can the software auto-record the date and time of the start of any screening questions and the completion of a given survey response? • Can the software use an existing data file to control what response options for a question are displayed based on prior responses (e.g., listing hotels in a given city)? • Can the software display response options based on the first few characters entered (e.g., suggesting city names based on the first two or three letters)? Do the options get refined as more letters are entered? • Can the user restrict the format used for numeric entries (e.g., require that a zip code have five digits and no decimal point)? • Can the user specify a template for numeric entries (e.g., telephone numbers as xxx-xxx-xxxx)? • Can the user specify range and consistency checks to be made on numeric responses? • Does the software provide specific capabilities to enter dates and times? • Does the software support touch-screen responses? • Can the software display a virtual keyboard on screen for entering text answers to open-ended questions? • Can the software display graphical features? • Can the software allow the user to select a location on a graphical feature (e.g., a location on a map)? • Does the software allow logic branches, are there constraints on how complex logic branches can be, and how easy is it to program logic branches? • How much memory or storage space does the program occupy? • How much memory or storage space is required to store each survey response, and how does this vary with the number of questions? • What data download/upload formats does the software support? • Can the software upload response data via Wi-Fi or a cell-phone connection to an online server as it is being collected? One option that should be considered is to have the EDCDs link to an online survey, either by Wi-Fi or cell-phone connection. While this does not eliminate the need to program the online survey, there is commercial software available to do this, and this avoids the need to write software for the EDCDs themselves. 5.10.4 Survey Logistics Other considerations also need to be taken into account in using EDCDs. There will need to be a secure room where the equipment can be left when not in use and where the data can be downloaded. It will usually be necessary to have spare batteries that can be charged while the units are in use or to have spare EDCDs to use while others are being recharged. Charging a large number of units can be a significant logistical issue that will require a secure location where they can be left while being charged. There are commercially available solutions such as lockable cases that can accommodate multiple units and recharge them. Downloading the data from the units may require a desktop or laptop computer unless the data can be uploaded to an online account. Some software allows response data to be uploaded via cell phone or Wi-Fi as responses are obtained. Access to a telephone line or Wi-Fi link will

84 Guidebook for Conducting Airport User Surveys and Other Customer Research allow data downloaded from the units to be transferred to a central database on a regular basis. Because the units themselves are not inexpensive (several hundred dollars each), appropriate steps should be taken to ensure their security while in use. Units should be checked in and out to survey staff at the start and end of each shift, and staff should be trained not to let them out of their possession while they are in use. It may be prudent to provide each unit with a short tether that can be clipped to the interviewer’s apron or vest to help prevent loss and damage. Apron pockets are also helpful as somewhere to put the unit when not in use. If tablets or other EDCDs are handed to survey respondents to complete a self-administered survey, it will be necessary to establish appropriate device security procedures to prevent respon- dents walking off with the device, either unintentionally or deliberately. These could include: • Placing the devices in a brightly colored case, and • Attaching sensors to the devices that will emit a loud tone if they are moved more than a limited distance from the field staff member responsible for handing them to respondents and collecting them afterwards. 5.10.5 Staffing Considerations Use of EDCDs will generally require staff training unless the interviewers already have experience with the particular model being used. Some older interviewers may have less experience handling small electronic devices, may have difficulty using the small keypads or seeing the screen display fonts, and may require additional practice to become comfortable using the devices. The other staffing consideration relates to programming and technical support. The program- ming can be subcontracted if no one on the survey planning team has the necessary skill and experience, but this will incur additional cost and administrative effort. The team will need to coordinate closely with the programmer to ensure that the finished program performs satisfac- torily. Although technical support in the field does not require the same level of specialized skill as programming and will generally be within the capabilities of survey supervisory staff, some training may be required beforehand if the field supervisors have not performed this function previously. 5.10.6 Summary Although EDCDs offer a number of distinct advantages, experience in their use for airport user surveys has been mixed. Problems can arise if their use has not been carefully planned and thought through. As with any evolving technology, some of the difficulties that arose with the early hardware and software have been overcome. Even so, the effective use of EDCDs requires some technical skill and experience. Survey sponsors should consider case by case whether the advantages justify the effort involved. In general, handheld EDCDs have typically been used for interview surveys. Although printed questionnaires have generally been used in the past for self-administered surveys, this is changing, and increasingly, EDCDs are handed to respondents for self-administered surveys. Surveys with the following characteristics tend to favor the use of EDCDs: • A long or complex set of questions with multiple branches • The need to tailor questions to the respondent’s answers to earlier questions • Questions with a large number of categorical response options • A large sample size (resulting in high data-entry costs) • The desire to include data consistency checks during the survey interviews

Survey Design and Implementation 85 • Availability of survey staff with experience in the use of EDCDs • Surveys that are likely to be repeated frequently (spreading the cost of programming over several surveys) On the other hand, surveys with the following characteristics may be more appropriately performed using printed questionnaires: • A fairly simple questionnaire with limited branching • A relatively small sample size • Straightforward questions that do not require data consistency checks • Respondents with limited time to answer questions and who may need to be given a printed survey questionnaire to complete and mail back later • Respondents needing to enter information or complete the questionnaire themselves • Lack of staff with appropriate experience in performing interviews using EDCDs • No plans to repeat the survey in the near future The widespread use of smartphones offers the possibility of survey respondents using their own phones for a self-administered survey if Wi-Fi service is available to allow them to access an online survey. Respondents will need to be given the link to the survey, such as on a preprinted card that can be handed out. Consideration will need to be given to respondents who do not have (or do not wish to use) their own phones to complete the survey, such as having printed questionnaires available or a few tablets that can be handed out. 5.11 Data Entry and Quality Control 5.11.1 Data Coding and Data Entry from Printed Questionnaires Data coding refers to the assignment of numeric codes to the various response options. Typically, this is done at the time the questionnaire is developed, and the codes are typically shown on the printed survey questionnaire to assist in data entry or embedded in the EDCD program. In some cases, responses to open-ended questions are assigned numeric codes in a separate field, particularly where the same response recurs frequently in the data, such as for hotel names. Although response codes can be omitted from printed questionnaires and provided to the data-entry staff as a separate document, this procedure may require them to refer to two different documents during data entry (unless they are using data-entry software that auto- matically inserts the codes) and may slow down the data entry or introduce data-entry errors. It will of course be necessary to define any response codes for commonly occurring text responses to open-ended questions after the survey has been performed. Data entry involves transferring the survey response data from the survey questionnaires to a computer file using the numeric codes shown on the survey questionnaire or developed later and defined in a survey codebook (a document that lists each code and its meaning). Two methods may be considered: • Manual data entry, using a data-entry contractor or survey staff retained after (or during) the survey data collection for this purpose. This method is labor intensive and subject to data-entry errors, although it can be facilitated by using data-entry software that presents the survey questions to the data-entry staff and allows the responses to be selected from the available options or text to be entered in appropriate fields. Manually entered data should be verified as discussed in Section 5.11.3. • Mass-scanning techniques, using high-speed scanning hardware and software that is capable of automatically coding responses into a database.

86 Guidebook for Conducting Airport User Surveys and Other Customer Research 5.11.2 Timing of Data Entry When the survey takes place over a relatively long period, such as several weeks, it is highly desirable to start the data-entry and verification process while the survey is still in progress. This can help identify any problems with the survey questions or procedures while there is still some opportunity to correct them during the remainder of the survey. In general, data-entry staff will not have enough local knowledge about the airport and ground transportation system to perform effective verification and error correction. Therefore, verification and error correction will require the active involvement of the survey planning team and possibly the assistance of other operations or planning staff with the necessary local knowledge. 5.11.3 Data Verification and Editing Once the survey responses have been entered into a computer file, it is important that the data are subjected to a careful and thorough verification process and that any errors are corrected. There are two aspects to data verification: • Checking that the data entry was done correctly if the survey response data have been entered from printed questionnaires. Ideally this involves repeating the data-entry task with different staff and comparing the two files. Data-entry software typically provides a verification func- tion that compares the second data entry to the original file and flags any differences. Any discrepancies are then resolved with reference to the survey questionnaires. However, this technique doubles the data-entry cost. A less expensive but less reliable approach that is commonly adopted involves verifying a random sample of survey questionnaires. The size of the sample depends on how much data-entry error is considered acceptable as well as the level of confidence in the data entry-staff based on past performance. This technique will establish whether the required accuracy for data entry has been achieved, but of course it cannot identify and correct any errors on the survey questionnaires that are not included in the verification. • Analyzing the data to identify any obvious errors, inconsistencies, or apparently illogical responses. This can address such issues as whether a trip origin zip (postal) code is in the reported city and whether the street name of a reported address exists in the city indicated. A common problem is misspelling of text responses to open-ended questions, such as city or street names, or switching digits in zip and postal codes. Checks can be run to make sure that respondents reporting the use of ground transportation services reported trip origins in locations where use of the service would be plausible. Numeric responses can be checked to ensure that they are within a reasonable range. Text responses in the “other” category of categorical questions should be reviewed to determine whether the response should have been given as one of the defined categories or new categories defined. In some cases, an error will be fairly clear, such as misspelled names or transposed digits in a zip or postal code. In other cases, it may be less obvious what the correct answer should have been or even whether there is an error at all. Data-entry errors are clearly not an issue if the survey responses have been collected using an EDCD or have been obtained from an online survey. However, survey data obtained in these ways still need to be verified for any obvious errors, inconsistencies, or illogical responses. Survey interviewers or survey respondents in self-administered surveys using tablets or other EDCDs, as well as respondents to online surveys, may have mistyped or misspelled the response to a question, or respondents may have misunderstood a question or simply made a mistake in answering a question.

Survey Design and Implementation 87 Data verification can be time-consuming if done thoroughly, but the overall quality of the survey data is greatly improved by devoting adequate resources to this task. The majority of the required effort lies not so much in identifying apparent errors in the data, which is fairly straightforward, but in the research necessary to determine what the correct response should have been. For example, it may be fairly easy to determine that a zip code is not in the reported city, but figuring out what the error is in the zip code, or even whether the zip code is correct but the city name wrong, is much more time-consuming. The more redundant information that is obtained in the survey, such as asking for both the zip code and state (or province) of a respon- dent’s home, the easier it is to identify and correct any errors. The Self-Administered Surveys subsection of Section 10.3.2 includes a discussion on the verification of air party size that may be helpful in understanding what can be involved in the data verification and cleaning process. Data editing is the final step in the process and involves making the necessary changes to the data file to correct any known errors. It is good practice to preserve the originally reported data in one set of data file variables (these may be referred to as fields or columns) and copy the data to a different set of variables before editing it. This procedure allows the data that have been changed to be identified later. Then changes can be revised—or even reversed—if subsequent information comes to light. 5.12 Analysis and Reporting of Survey Results At this stage of the project, the survey data collection is complete, the data have been entered into appropriate databases, and the data have been checked for the internal integrity of each response and cleaned accordingly. It is now time to undertake the analysis, as determined by the goals of the survey project. When the analysis will be completed by in-house staff, the staff time involved will generally be budgeted through a separate process than when having the analysis and reporting done as part of an external contract for the planning and conduct of the survey, which could result in some cost savings. 5.12.1 Considerations for Doing the Analysis In-House It is appropriate to use in-house resources if the following considerations make it feasible: • There is sufficient expertise in-house to complete the tasks listed in this section, and this expertise is available in a timely fashion after the data collection period. • The appropriate software and hardware exist to complete the analysis. Modern spreadsheet applications have many data-analysis capabilities. The requirements may extend beyond these capabilities, in which case the survey sponsor should consider using the more powerful sta- tistical features of applications such as SAS or SPSS. There are also freeware applications with extensive statistical analysis capabilities, such as R (https://www.r-project.org). 5.12.2 Weighting and Expansion Once the data have been cleaned and made available to the analyst, one of the first priorities (if weighted data are to be used in the analysis) is the assignment of one or more weighting factors to each interview response. The motivation for the use of weighting factors and the general approach to calculating response weights is discussed in Section 5.3.4. In the case of sample surveys that have been conducted according to a strict sampling plan, the response weights will be determined as part of this plan, as discussed in the case of air passenger surveys in Section 10.8. These weights are usually added to each interview record.

88 Guidebook for Conducting Airport User Surveys and Other Customer Research In the case of groundside surveys, discussed in Section 10.12, there will be one weight assigned to the interview record based on the vehicle count at the facility and a second weight assigned based on the passenger volumes. The analysis may use either of these weight factors depending on the focus of the results—passenger versus vehicle. Calculating weights for each survey record requires control totals (census counts) for relevant operational statistics, against which the survey response totals can be compared. Depending on the available data, these census counts might include the following: • Air passenger counts, possibly by hour or by day • Parking-lot exit counts, by parking duration • Automated vehicle identification system counts for modes other than private vehicles • Roadway traffic counts • Control count observation surveys—such as at the curb area, security clearance area, or other suitable location—for determining the effective sampling rate and weighting of responses to be used in the analysis 5.12.3 Tabulation and Interpretation Tabulation and interpretation are the core of the analytical process. Typically, this process begins with basic frequency counts of the key variables and cross-tabulations. This step can often reveal unexpected relationships as well as unexpected problems in the data. It is recommended that frequency counts be performed for all variables to get an initial look at the numbers of valid responses for each question. The subsequent process of data analysis is determined by the goals of the survey and is often a personally defined process that is unique to the analyst. It is quite likely that the analysis will reveal unexpected or interesting results that appear to warrant further analytical effort. However, such additional analysis can only be pursued as time and resources permit. Since it is in these further analyses that the full value of the data will be obtained, the survey budget should provide sufficient provision for an adequate analysis of the results. Software tools such as Tableau and the Power Pivot add-in of Microsoft Excel are available that support data exploration and allow easy filtering and cross-tabulation so that non-specialist analysts can explore the data for patterns or relationships. Although these capabilities can be a powerful addition to the analysis, care is needed to ensure that any apparent patterns are statisti- cally significant and not simply due to the inherent variability in the data. 5.12.4 Survey Accuracy and Limitations An important, but sometimes overlooked, purpose of the data analysis is to identify the accuracy of the results; another is to identify the limitations within the survey data. Where results are provided for subgroups of the population, the accuracy of the estimates for these sub- groups should also be identified. For some desired analyses: • The sample size may not be sufficient, and consequently, it may not be feasible to drill down to increasing levels of detail. • The conduct of the survey or the response rates will produce limitations in how the data can be analyzed or the level of detail that can be achieved in the analysis. Minimizing such limitations is a key reason to include the expertise of a data analyst from the outset of the survey project. • The limitations may be a function of the questions, how they are coded, and what responses were obtained. These limitations may not be revealed using data-analysis techniques alone.

Survey Design and Implementation 89 5.12.5 Report Preparation and Presentations The primary reports and presentations must be directed toward the goals and purpose of the survey. A formal report is generally required and will be a substantive document that fulfills the requirements of the survey project. A presentation on the results of the survey, which addresses the purpose of the survey and explains what was obtained for the funds approved, will close the loop with senior management. Some reports take the format of a series of presentation slides that can be used for both purposes, although with such an approach, the details of the methodology and the discussion of the results are typically fairly brief. Such reports generally do not include detailed tabulations, and they present the results for each question graphically on a separate slide, although each such slide may include the numeric results. More traditional reports will typically include a chapter on methodology and one or more chapters discussing the survey findings. Survey results for each question will often be presented graphically and with a table showing the number of responses for each question response option and the percent of respondents selecting each option. More detailed cross-tabulations are sometimes included as an appendix or appendices. A fairly common practice is to define a set of banner column headings that show the response options for a limited set of questions, and then tabulate the responses for the other questions against the questions in the banner heading. Although this provides consistency in the presentation of the cross-tabulation results, it can also result in cross-tabulations not being shown that might be of particular interest to users of the survey results. Some survey reports contain appendices with more details on the survey methodology, including details of the sampling procedure and training materials for field interviewers (in the case of interview surveys). All reports should include a copy of the survey questionnaire or a listing of the questions and response options, together with the question skip logic in the case of a survey using EDCDs or an online survey. 5.12.6 Publication of Survey Products and Findings The entire survey project, while unique to the agency and airport involved, may include aspects that are of potential use to a wider audience. In this regard, consideration should be given by the survey sponsor to making the survey results available to interested parties beyond the survey planning team and internal stakeholders. There are many forums where such publications would be welcome. Because there are many airports conducting surveys at any time across the country and around the world, other planners may just be beginning the process and would benefit from the knowledge gained from the survey. Perhaps the single most effective way to distribute information about airport user surveys is for survey sponsors to post the survey reports on their websites. As noted in the preceding subsection, survey reports should include a copy of the questionnaire or list of questions, since understanding the wording of the questions and response options is essential to the inter- pretation of the survey results. As public agencies or departments of public agencies, most U.S. airport operators and many others throughout the world are subject to public disclosure laws or regulations. However, the critical issue is not whether the survey sponsor would provide a copy of the survey report if asked, but how interested parties would know that the survey report exists in the first place. Placing a report on the agency website allows web search engines to catalog it and greatly simplifies access to the report.

90 Guidebook for Conducting Airport User Surveys and Other Customer Research In some cases, the survey contractor or (less commonly) the survey sponsor may decide to copyright the questionnaire. This generally occurs when a survey contractor uses essentially the same questionnaire to perform similar surveys at multiple airports and does not want competing firms to be able to benefit from the question wording and flow sequence. As long as the ques- tionnaire is included in the survey report with this restriction, this does not limit readers of the report from understanding the question wording and skip logic. The situation becomes more difficult if the survey contractor does not want the questionnaire included in the project report or attempts to limit disclosure of the survey methodology, questionnaire, and survey findings by the survey sponsor. Faced with such a situation, the survey sponsor can try to negotiate a compromise that allows the report to be made publicly available. Failing this, a survey sponsor that plans to post the survey report on its website may have no choice but to find a different survey contractor. A few survey sponsors have gone so far as to post the survey response data on their websites or in some cases the website of the local jurisdiction of which they are part. This allows interested stakeholders to undertake their own analysis, which can lead to greater insight into some of the issues addressed in the survey, provided that the stakeholders share their findings with the survey sponsor. Survey sponsors can encourage this by restricting access to the survey response data to those with a login code, which the survey sponsor will provide in return for a commit- ment by the user of the data to share any findings with the survey sponsor. 5.13 Post-Survey Analysis: Lessons Learned The periods immediately following the survey and after the analysis are important times to reflect on how the survey went. Chances are that there are a number of things that went really well. It is also likely that some things did not go as well as planned. Documenting this infor- mation at this point will prove worthwhile when it is time to do another survey. For the things that went right and the things that went wrong, it is important to document why this is the case and what the thinking was underlying the various decisions. Especially important is documenting what went wrong, why it went wrong, and how it could be addressed in the future. At this closing stage of the project, there are probably some ideas and thoughts that could be included in this document that would help in the future. During the survey, the project team likely received comments—from supervisors, inter- viewers, and even the airport users interviewed—about aspects that could have been improved or that went well. Documenting these comments will ensure that they are considered before the next survey, when improvements can be made at the design stage. With interview surveys, it may be helpful to ask a few questions at the end of a small sample of interviews to formalize the process of obtaining comments from survey respondents and ensure that these get recorded. (This is fairly easily done with the use of EDCDs but somewhat more difficult with printed questionnaires.) Alternatively, field supervisors could ask a few questions of a sample of survey respondents as part of thanking them for their participation. This would have the advantage of allowing the interviewers to continue to survey additional respondents or move on to other locations. Questions could include: • Was there anything about the survey that you found confusing or hard to understand? • Was there anything that you expected we would ask about but that we did not? • Is there anything that you feel we should change with the survey or its procedures? • Do you have any comments on the survey staff member who interviewed you (or handed you the survey)?

Survey Design and Implementation 91 It would not be necessary to ask each question in every case, but the questions could be varied across the sample of respondents. 5.14 Documenting the Survey It is important to record as much as possible about the survey. The importance of recording what went right and what went wrong has already been emphasized. Throughout the survey project, a number of other documents will have been created for various purposes. All this documentation should be preserved for possible re-use in later surveys. Documents that are likely to have been created and that should be maintained for future use include the following: • Survey design reports (including sampling procedure, survey plan, and logistics documentation) • Requests for proposals for any contractors • Training materials • Survey questionnaire • Codebook(s) with definitions of codes used for question responses • Description of any data-entry and analysis software used • Survey analysis reports and presentations • Lessons learned • The data and data description file (variable definitions, type, and field lengths) Of all the components of the survey, the data are the most valuable. This information, after all, is what the money was spent to collect. Even after the analysis is completed, these data will be invaluable in the future. 5.15 Level of Effort Required for Survey Planning It should be clear from the discussion in the preceding sections that planning an effective airport user survey requires careful consideration of a large number of different factors. It follows that adequate resources and time need to be allocated to the planning stage. The appropriate level of resources and lead time will vary with the circumstances of each survey— in particular, the survey method being used and whether the survey is being performed for the first time or is repeating a former survey. For an air passenger interview survey with several thousand respondents and being performed for the first time, experience suggests that the planning stage requires as much as 20% of the overall budget and should commence at least 6 months before the planned data collection. The time required is considerable, but the quality of the resulting information will depend on the effort devoted to sound planning.

92 Qualitative Methods 6.1 Introduction Most of the material in this guidebook has focused on quantitative research methods. These are methods that yield primarily numeric results and are based on the principles of large numbers. Most of these principles are mathematical in nature and are referred to as “statistics,” a topic that is discussed in Chapter 4. In contrast, qualitative research methods yield predominantly narra- tive information. This information is either obtained by an individual research team member or generated by interaction among research participants. The principles involved here are those of human inter- action and small numbers and derive from such fields as psychology and sociology. These methods include open-ended questioning, small- group management, interpersonal engagement, and the analysis of narrative commentary. As a general rule, quantitative studies are best suited to asking and answering “what” or “how many” questions, while qualitative studies are best suited to “why” or “how” questions. Seeking to answer “why” or “how” questions in any depth in a quantitative study rarely yields satisfactory answers, while endeavoring to answer “how many” questions in qualitative studies leads to imprecise measurements. At times, of course, an area of concern will raise both “what” and “why” questions. If this is the case, it may be appropriate to consider which type of query predominates and proceed accordingly. If the issue is of sufficient importance, on the other hand, consideration could be given to including both quantitative and qualitative components in the study. 6.2 Advantages of Qualitative Research In addition to answering questions that cannot readily be addressed by surveys and other quantitative methods, qualitative studies can be attractive to researchers because they are generally less expensive, although not on a cost-per-participant basis. They also facilitate an in-depth understanding of what motivates customers to do what they do. Finally, focus groups, by far the most popular form of qualitative research, permit managers actually to see and hear their customers talk about their product or service. C H A P T E R   6 Best Practice “We also meet with traveler advisory groups to gather feedback quarterly. . . . Fifty percent of the group are frequent travelers, and the other 50% are staff members. One group gathers feedback from frequent travelers of varying demographics, and the other gathers feedback from travelers with disabilities and disability advocates. . . . There’s always an open forum time during the meetings. . . . Most airports don’t have these types of committees, but they should.” —Research participant

Qualitative Methods 93 6.3 Concerns About Qualitative Research While qualitative research is popular for the insights it yields, its relatively low cost, and its ability to showcase the customer first-hand, there are notable concerns about these methods. Qualitative sample sizes are small, and participants are often purposefully rather than randomly selected, both of which qualities violate key statistical principles. In addition, regardless of how much effort is devoted to the control of study design, implementation, and interpretation, all of the qualitative methods are fundamentally subjective in nature and depend to a heightened degree on the skills of the researchers. Finally, becoming a qualitative researcher appears to be a deceptively simple undertaking, with the result that there are a number of practitioners who are undertrained and unskilled. The fact that there are only two formal training programs for professional focus group moderators in the United States (RIVA and the Burke Institute) is a further indicator that this profession is almost certainly not taken seriously enough. Those seeking to conduct qualitative research would therefore be well-advised to make particularly careful assessments of the skills and experience of the providers. 6.4 Useful Qualitative Techniques Four qualitative research techniques appear to be particularly useful for commercial airports: focus groups, in-depth interviews, observation, and mystery shopping. Standard terminology typically associated with these research methods is italicized in the following subsections. 6.4.1 Focus Groups As noted previously, focus groups are by far the most popular qualita- tive method. These are small groups, typically of eight to 10 people, who meet for a period of time under the guidance of a professional focus group moderator. It is also possible (albeit rarely ideal) for a truly skilled moderator to interact with as many as 12 participants face-to-face. The apparent consensus recommendation for online groups is between three and six participants. This is noteworthy because in-person groups this small will tend to feel more like a series of individual interviews rather than generating the interaction that is one of the hallmarks of focus group research. As a rule, there is also considerably less inter- action among group members in online groups than there is in in-person groups. The duration of a focus group can vary with the number and types of discussion topics as well as the characteristics of the participants, but it is usually somewhere between an hour-and-a-half and 2 hours. Groups with people who are especially important or otherwise committed, such as business executives or community leaders, can be as short as an hour. These groups may also be smaller than eight to 10 participants to compensate for the shortened time frame, but it is not advisable to have fewer than six participants unless the group is held online. The number of focus groups conducted on a particular topic will vary depending on client preferences. It is always advisable, however, to hold three (or more) groups if possible, given that the method is Recommendation for Online Focus Group Size This assessment derives from an informal but extensive conversation on the American Association for Public Opinion Research listserv that was initiated by the principal investigator on this guidebook project when it became apparent that the COVID-19 pandemic would lead to the cessation of in-person focus groups and a transition to online sessions. Although participants were by no means unanimous in their opinions, the general consensus was that online groups should contain at least three people and should be limited to no more than six. Involving fewer participants would limit the information derived from the groups too severely, while including more than six would make the groups too difficult to manage online. This conversation took place during the spring of 2020. Subsequent conversations with representatives of facilities that were contacted about the research project’s groups confirmed these sentiments.

94 Guidebook for Conducting Airport User Surveys and Other Customer Research qualitative. A single group will have little meaning by itself because it represents only a few opinions. With two groups, confidence increases if the two generally agree with one another, but there is always the possibility that they can diverge. A third group will then serve as a tie breaker. Although the statistically inclined would be hesitant about drawing conclusions about an issue from a sample of some 30 people from large populations across three sessions, decisions based on focus group research with three groups have had widespread support among research practitioners and users. Typically, in-person groups are held at a formal focus group facility that has a conference room setup, a viewing room for observers with a one-way mirror, and high-level recording capabilities. In addition to hosting the groups, the facilities recruit the participants and confirm their attendance, both the day prior to the session and, if necessary, the day of the session. Recruiting usually relies on extensive databases maintained by focus group facilities, with potential participants often numbering in the thousands. For some specialized groups, such as those with members of an organization or a company’s customers, client databases are cus- tomarily used. Recruitment is typically undertaken by telephone staff following a standardized script. Online recruiting is also used if it is suited to the target audience, can be appropriately scripted, and does not require probing of possible answers. Online focus groups are typically recruited and confirmed in the same manner as in-person groups. They also function much the same way. The key differences between the two approaches are the numbers of participants it is possible to work with in terms of effective group management and the nature of the dialogue. Participant recruitment is guided by a document called the screening questionnaire (or screener) that details the types of participants that are desired in the group. Typically, this screener is designed by the moderator in consultation with the client. The moderator also designs the moderator’s outline or discussion guide that will be used to guide the conversations, again in consultation with the client. Examples of these documents from a variety of actual airport projects can be found in Appendices B and C. Finally, it is important to know that almost all focus group projects provide participants with a cooperation fee (often simply referred to as the co-op) for their participation. At the date of this writing, these fees for members of the general public ranged from between about $125 to $150 per person. For professionals, they can be considerably higher. It is always helpful to ask the facility what co-op they would recommend since the amount is part of what drives participation; if the fee is too low, recruitment becomes more time-consuming and expensive. Most focus groups are held with people from the local area and do not involve significant travel time or expenses. In these cases, any travel costs are typically not reimbursed. In the relatively rare instances when substantial travel is involved, compensation is provided for appropriate expenditures. Generally, focus group recruiting should begin about 2 weeks before the group is to be held. With populations that are difficult to locate and recruit, such as frequent business travelers or people who use a particular ground access mode, more time may need to be allowed. Mondays, Fridays, weekends, holidays, the week of Thanksgiving, and the period from December 15 through the first week in January should be avoided as group dates. As indicated earlier, focus groups are best employed to answer “why” questions. A key advantage of groups over individual interviews is the dialogue that emerges, with people ques- tioning one another, trying to persuade their fellow members, raising “aha” points, and even changing their own minds. In a sense, groups are microcosms of how people think about things, how they interact with others about them, and how opinions form and evolve.

Qualitative Methods 95 One potential disadvantage of this interactive approach to understanding the customer is the possibility that individuals with strong personalities or opinions will influence the manner in which others respond. Taken to extremes, this can lead to a group in which there are essen- tially leaders and followers, with the actual perspectives of the latter being represented inaccu- rately or not at all. Experienced and astute moderators will usually see inappropriate influence occurring and take steps to prevent or at least moderate it. If only the latter is possible given the nature of the group, this will need to be considered in the final report. If it is not clear in what direction the group is trending, or if it appears that a few people are dominating a part of the discussion, the moderator may ask for a show of hands to establish a better understanding of the group’s sentiments. It is important to note in this regard, however, that these data are not to be interpreted as numeric or reported as such. Focus group findings are qualitative and should always be viewed as indicative rather than definitive or mathematically projectable to the population as a whole. Best Practice “We used a professional research facility . . . to conduct focus groups that examined guest perceptions of and expectations for parking services. This study showed us how to tailor parking offerings to increase revenues. The focus groups identified amenities and elements that customers wanted. We were able to implement these items in the parking facilities, which substantially increased our revenue. We also developed brand names for our parking products based on information gathered in the focus groups. The focus group research enabled us to create a brand new parking product that competes directly with our off-site competitor.” —Research participant Best Practice “We did focus groups to gather information for the development of the open areas in the terminals. We conducted six focus groups, each with 12 participants. We asked about types of seating they preferred, what would make them want to spend time in the area, what local restaurant concepts they would prefer for the spaces, etc.” —Research participant Success “I’ve done one focus group with members of our aircraft spotting community. Those are people who like to come out to the airport . . . to take photos of airplanes. There historically has been a very adversarial relationship between our law enforcement and operations and those folks, which is unfortunate. . . . I said why don’t we just get you guys into a room. . . . And that was just a win–win scenario for us, listening to their concerns. We’ve got some changes in place that we’ll roll out next week based on that discussion so everything we love about doing market research is to see actual results that make people happy, that was an early win I guess.” —Research participant

96 Guidebook for Conducting Airport User Surveys and Other Customer Research Also, as a general rule, focus group observers should always be cautious about drawing con- clusions from what they have heard before reading the full report. Professional moderators will generally recognize what is solid enough to be reliable in a group’s discussions and be able to advise end-users about what should be regarded seriously in terms of potential action. Focus groups can consider an extremely wide range of topics—virtually anything about which sponsors want to better understand the “how” or “why” of. Focus group subjects about which the authors of this guidebook have inquired include branding, ground access mode choice, perceptions of several individual modes, various aspects of parking, and curbside operations. Participants in the project’s telephone survey, site visits, and focus groups reported that they had held groups about master planning, understanding their markets, airport choice, the design of a new facility, the construction of a new terminal, how to develop the airport’s open areas, awareness of and attitudes toward airport amenities, and relationships with airport spotters, whose hobby it is to visit the airport to photograph airplanes. Reporting on the results of focus groups typically involves three steps. First, a professional focus group transcriptionist uses the audio recordings to prepare transcripts that are as close to verbatim as possible. Although this is never 100% successful given challenges with such things as diction and accents as well as occasional cross-talk that the moderator was unable to halt in time, an experienced transcriptionist will almost always prepare a document that captures most of what people actually said. Second, the moderator reviews the transcripts in their entirety, taking notes or actually drafting report text in the process. Transcript review is an essential step in order to prevent the intrusion of perceptual bias into the resulting report, although it is unfortunately sometimes overlooked in practice. What people remember from a group can differ significantly from what was actually said. Airports would therefore be well advised to make sure their focus group moderators rely on transcript review for report preparation. Finally, after the text of the report has been written, the moderator will often insert actual quotes from participants to illustrate the points that group members were making. These quotes both reinforce the key findings from the research and bring the experience to life for someone who was not in attendance as an observer. 6.4.2 In-Depth Interviews In-depth interviews, also known as “individual depth interviews” (IDIs), are largely open- ended interviews that are guided by a script similar to a focus group moderator’s outline. Such interviews can be conducted with customers or stakeholders, although their cost tends to limit their use with members of the general public. Typically, IDIs are conducted by executive interviewers who have either excelled in general interviewing; displayed particular talent in probing for clarity, depth, and specifics in quantita- tive studies with open-ended questions; or come from fields such as psychology or journalism. Successful executive interviewers are able to build rapport, function as active listeners, probe superficial or terse answers, and be sensitive to verbal and nonverbal cues. IDIs tend to be relatively long, often having durations of 1 to 2 hours. Many, if not most, are conducted in person, although thorough-depth interviews can be conducted by telephone if the interviewer is sufficiently skilled and personable. During the research conducted for this guidebook, only one airport mentioned an example of IDI work, which involved personal interviews at the gates. Outreach to stakeholders, however, sometimes includes individual executive interviews, although the extent to which they are

Qualitative Methods 97 structured is not clear. In all probability, this varies from airport to airport and perhaps even from project to project. Stakeholder outreach is certainly an arena in which in-depth inter- viewing would be appropriate. Finally, the research preliminary to the preparation of this guidebook itself included 46 in-depth interviews with airport representatives about their research efforts and their sense of what the guidebook should address. All of these interviews were conducted by tele- phone; they ranged in length from 30 minutes to two-and-a-half hours. Results of the inter- views were used to guide the design of subsequent project research activities (site visits and focus groups) and to inform the development of the guidebook itself. 6.4.3 Observation Observation research quite literally means watching what people do without communicating with them. It can be largely qualitative or exploratory in nature, as when researchers are trying to figure out what the possible behaviors are in a given situation, or it can be quantitative, when behaviors are being tallied and recorded. Many observation studies use a hybrid of these two methods. Observations can be made by people or electronically, and both of these strategies are used in airports. Electronic observation is purely quantitative in nature and is discussed in Chapter 9. One prominent example is various types of customer monitoring at checkpoints. The topic of discussion here is human observation. A key advantage of observation research is that it does not reflect self- reported behavior and is thus reliable given a well-trained and attentive observer. Observation also prevents issues with respect to cooperation; because people are not being asked to participate, they are not offered the option to refuse. The major disadvantage of observation research is that it is limited to behavior and does not capture motivations or attitudes. Thus, if researchers need to know why people do what they do or how they feel about it, this technique will not be appropriate. Often, primarily observational studies also include an interactive component. The most com- mon use of observation at airports is for assessing wayfinding. Wayfinding is important both because it is a key concern of and driver of satisfaction for passengers and because the more time passengers have at their in-airport destination, the more they are likely to spend at concessions. Thus, wayfinding is also important to concessionaires. In wayfinding studies, observers either follow passengers on their trips from, for example, security screening to a particular concourse or gate, or observe them at strategic decision points along the path. Passengers may also be asked, by interviewers walking along with them, about decision points that are identified as being potentially problematic. Photographs can be used to enhance the reporting, such as by documenting unclear signage. Another use of observation that airports reported is evaluating the presentation and behaviors of employees. These studies involve recording actual appearance and actions during transac- tions with customers using a checklist of expectations such as uniform cleanliness, smiling, eye contact, and responses to information requests. Appropriate and inappropriate behaviors can both be illustrated with photographs. Best Practice “We also use the technology from Avius . . . to collect data on staff observations. So these are KPIs [key performance indicators] that our customer service staff [look for when they] observe. Every day they are supposed to observe 10 employees. . . . They go and they stand and they watch the employee, they have an iPad in their hand, and they watch the employee engage with the customers and they are looking for specific things. A lot of times it is to observe for a clean shirt and uniform, shoes, name tag, smiling, eye contact. Those are things that the employees need to know that our customers are looking for. . . .” —Research participant

98 Guidebook for Conducting Airport User Surveys and Other Customer Research One possibly unique observational study was undertaken to evaluate an airport’s pilot food-scrap recycling program. The airport’s overall goal was to increase recycling of waste from 23% to 50%. Analysis of waste stream components revealed that compostables were the largest single source of material that was recoverable but was going unrecovered. The airport then set up five sets of three bins in the central terminal food court. Vinyl signage that explained and promoted recycling at the airport was also placed on tables. Because self-reports of socially desirable behavior such as recycling are unreliable, the airport used unobtrusive observation to determine how passengers disposed of waste. The observers, who were stationed in an employee area located above the food court, recorded approximately 1,400 customers’ waste disposal behavior. In addition, almost 500 trav- elers participated in a 2-minute interview that included questions about familiarity with food scrap recycling. The study found that although the pilot program was successful, the recycling guidelines with which passengers had been provided were complex and confusing. Recommendations for improvement of the information were then developed for management to consider. 6.4.4 Mystery Shopping Mystery shoppers patronize a retail establishment as if they were customers, using predetermined behavioral and verbal scripts. In the case of airports, mystery shopping is usually conducted with conces- sions such as stores and food service providers, but it can also be used at check-in areas, security checkpoints, and gate areas. The overall goal of mystery shopping is to evaluate the establishment in question from the perspective of customers. Specific objectives might include assessing overall ambience, considering cleanliness and order- liness, appraising service personnel, identifying employees who might be recommended for awards, finding out if there are personnel who require further training, and determining the extent to which the venue conforms to the vision the airport has for its concessions. Mystery shopping’s level and depth can vary substantially, depending on the objectives of the study. A basic shopping study might be limited to observations of such things as cleanli- ness, tidiness, and displays or to verbal inquiries about information such as product availability and pricing. A more comprehensive shopping study could include interactions with concession personnel during an entire transaction, from atmosphere and greeting through product infor- mation, selection, pricing, delivery or packaging, product quality, and overall customer service. In some cases, experiences with returns are also evaluated. Given that mystery shoppers are operating incognito, they cannot very well take notes as the visit progresses, although some will carry hidden recording devices. Traditionally, mystery shoppers have been expected to have a high level of knowledge of the kinds of establishments they are patronizing as well as excellent memories. Observations are then recorded immediately following the visit. Forms for this purpose typically include questions that are both closed-ended (“Were you welcomed within X TIME of arrival?”) and open-ended (“How would you describe the atmosphere of the establishment?”). Reasons for ratings and other assessments are often Best Practice “Our partner research company . . . identifies guests who are scheduled for travel at the airport and willing to participate in the [mystery shopper] program; the participants are kept anonymous from the airport. The participants rate their experiences with food, retail, security checkpoints, ground transportation, parking facilities and cashiers, traffic-control officers, guest services, staff in terminals, all restrooms, our proprietary taxi service, and rental car services. The participants are provided with questions that allow them to offer insight from the passengers’ perspectives as they travel through the airport. All survey questions are based on our service standards, so we can measure, from the guests’ perspectives, if standards are being met with each experience within the airport. The surveys are completed every month, and we conduct about 1,500 shops/ surveys per year. They are completed on a smartphone or tablet while the participant is still in the airport but away from the area they observed.” —Research participant

Qualitative Methods 99 added to ensure that opportunities for positive feedback and areas needing improvement are clearly articulated. Mystery shopping can be conducted with a single shopper for purposes of consistency or with two shoppers (over time) to obtain a range of perspectives rather than the assessments of a single individual. When there are noticeable disagreements among shoppers, it may be appropriate to add a visit or visits. A key advantage of mystery shopping is that it provides in-depth assessments of a variety of attributes of concessions from a customer perspective. Respondents to air passenger or resident surveys often will not be able to recall these attributes in sufficient detail to make the results actionable. Mystery shopping can also be used in conjunction with customer satisfaction surveys to analyze areas meriting commendation, areas with deficiencies, and specific improvements that may be warranted. A major disadvantage of mystery shopping is that a typical study includes a limited number of visits to a concession. In addition, many of the assessments rely on judgment, and the resulting ratings are dependent both on the person making the assessment and the person serving the shopper. Thus, if there is significant variation in service quality over time or in how the mystery shoppers rate the same attribute, the results may not be accurate. Variation between mystery shoppers can be reduced with good training and careful atten- tion to the content of the shopper reports. Other strategies for addressing this situation include enhanced training or retraining of the shoppers, group consultation with the shoppers involving their results, follow-up by yet another shopper, observation of shops followed by consultation with the shopper, and management shopping to validate shoppers’ work. There may still be some variation due to the nature of the method, but it should be reduced to a manageable level.

100 Monitoring and Enhancing Customer Service 7.1 Introduction Many organizations, including airports, emphasize customer service as a central objective. Accordingly, a variety of programs designed to enhance customer service have been developed and adopted across many industries. These include customer relationship management (CRM), experience management (XM), and customer experience (CX) programs. All of these programs incorporate measurement (i.e., monitoring) of customer service. However, such measurement is only one of the elements necessary for a program to effectively enhance customer service. Although it is beyond the scope of this chapter to fully compare and contrast CRM, XM, and CX programs (hereafter referred to collectively as CX programs), it is useful to note that they have many common elements: • An emphasis on action: – A direct link between customer feedback and organizational responses • Strategies formalized as mission statements, core organizational values, or brand promises • An emphasis on understanding customer motivations, journeys, and pain points • Intentional shaping of organizational culture: – Linking key performance indicators (KPIs) to customer service, enabling customer service champions, and emphasizing customer service in employee evaluation and hiring • Closing the loop by following up on customer feedback with concrete actions (discussed in more detail in the following) 7.2 Customer Feedback and Closed-Loop Processes Within CX programs, feedback from customers is used in two distinct ways, which are com- monly described as the “inner-loop” and “outer-loop” processes. The grouping of these two processes under a single closed-loop label can be a source of confusion regarding the type of feedback needed for monitoring customer satisfaction. Because the inner-loop process depends on responses from individual customers, the ideal form of data collection would be a census that collects feedback from every customer. In the real world, a complete census is attempted only with small customer subgroups such as airport tenants, and even then, it is virtually never achieved. This leaves researchers and managers to choose among alternative ways of collecting feedback that have different strengths. 7.2.1 Closing the Outer Loop The outer loop functions at a macro level in which feedback is aggregated across large groups of customers (for example, passengers using Concourse A and passengers using Concourse B) C H A P T E R   7

Monitoring and Enhancing Customer Service 101 and is used to drive large-scale management actions. For the purpose of closing the outer loop, it is critical that the feedback be an unbiased, representative description of how those large groups of customers evaluate the quality of their service. The best practice for obtaining such a description is to collect feedback from a random sample of customers. With random sampling, more feedback is not necessarily better. A relatively small number of responses (less than 500) from customers who have been carefully selected and invited to provide feedback is more likely to represent the customer population than a set of thousands, or even tens of thousands, of responses from customers who are selected in a potentially biased way. Moreover, doubling the size of a random sample only makes the confidence interval 25% narrower. Accordingly, programs intended to drive large-scale management actions (such as the ASQ, DKMA, and JD Power surveys) focus significant attention and resources on the careful selection of a rep- resentative sample by incorporating random selection. The success of such data collection is measured by how well it adheres to best sampling practice, not by the number of responses collected. 7.2.2 Closing the Inner Loop The inner loop functions at a micro level in which the organization responds to individual customers based on the feedback they provide. Usually, this means that automated alerts or other processes are triggered by negative feedback, sent to customer service staff, and followed up by contact with individual customers to address their dissatisfaction. Inner-loop processes are critical when organizations have a limited number of customers, each of which has a significant effect on organizational success. For example, most airports have formal or informal inner-loop processes for their tenants (whether or not they call those tenants “customers”). The justification for airports to use an inner-loop process to respond to individual passengers is not as direct. Many airports do so in order to build a customer-oriented culture, show public responsiveness to pas- senger concerns, and limit multiplier effects associated with negative word of mouth. Monitoring the topics of customer concern can also help airport managers identify emerging trends in customer experi- ence, particularly between the sampling periods of benchmarking or tracking surveys. In contrast with the outer loop, more feedback is virtually always better for inner loop purposes. Even if airport staff cannot respond to every dissatisfied customer, they can prioritize the most dissatisfied customers and respond to as many as possible. Furthermore, the presence of potential bias in the feedback is of little concern. It is not a problem when the feedback overrepresents dissatisfied customers because the inner-loop process is successful when it identifies and responds to as many dissatisfied customers as possible. In airports, the best way to get a lot of feedback is to invite as many passengers as possible to comment using whatever channel is most convenient. Social media and text messaging programs are particularly well-adapted to this purpose. 7.2.3 The Feedback Dilemma One weakness of many CX programs is that they focus on what to do with feedback and pay limited attention to how feedback is collected. In fact, airport managers seem to face a dilemma based on the relative strengths and weaknesses of different methods of collecting feedback. Best Practice “Kipsu is a text messaging program. We have signs throughout the airport that say if you have a question, text us your question, and the Kipsu software program phone number is basically a location designator, so they say, ‘OK, where is the closest coffee shop?’ That message goes right to our information office and they get a real-time response between 8 a.m. and 8 p.m. on any questions they have.” —Research participant

102 Guidebook for Conducting Airport User Surveys and Other Customer Research Random sampling supports outer-loop processes well but is prob- lematic for inner-loop processes because it identifies only a small proportion of all dissatisfied customers. In contrast, collecting lots of self-selected voluntary feedback from multiple channels (for example, social media, comment cards, and online feedback forms) supports inner-loop processes well but is problematic for outer-loop processes because the feedback may be biased and provides little confidence that the aggregated results represent the satisfaction of all customers. The answer to this perceived dilemma is that airports can collect feedback using multiple methods. The inner and outer closed-loop processes can and (whenever possible) should be supported by inde- pendent forms of data collection. 7.3 Monitoring Customer Service Many airports conduct surveys to monitor the quality of their customer service. Whether conducted annually, quarterly, or on some other frequent or routine schedule, such research is critical for enhanc- ing customer service. Typically, such surveys include a measure of overall satisfaction with the airport and a series of more specific expe- rience factors (for example, waiting time at security or comfort of the gate/waiting area) that are also rated on a satisfaction scale. Whether the survey is custom designed for a particular airport or is a standardized airport benchmarking survey (such as the ASQ, DKMA, and JD Power Surveys), such research not only plays a critical role in the outer-loop process, but it also tracks satisfaction over time to assess the effectiveness of management actions and identify changes in priority for the different experience factors. 7.3.1 Prioritizing Experience Factors Overall satisfaction is a general measure that is affected by many experience factors. To effectively increase overall satisfaction, managers need to prioritize actions that are focused on the specific experience factors that affect overall satisfaction most strongly. The standard technique for identifying high-priority experience factors is called “key driver analysis.” In key-driver analysis, researchers use an X/Y grid to plot and compare the various experience factors. The X-axis commonly represents the importance of each factor as a determinant of overall satisfaction (measurement of importance is discussed in more detail later), and the Y-axis represents customers’ satisfaction with each factor. The grid is used to identify experience factors that fall in the quadrant where importance is high and satisfaction is low. Those experience factors are then considered the highest-priority ways to improve overall satisfaction (Figure 7-1). One of the advantages of benchmarking surveys such as the ASQ, DKMA, and JD Power surveys is that the key-driver analyses can be enhanced by representing relative performance. The standard measure- ment of performance is simply the average satisfaction rating for each experience factor. However, with benchmarking data, the analysis can instead plot the difference between the satisfaction ratings at the target airport and the average ratings across a selected panel of peer air- ports (example in Figure 7-2). Representing performance relative to a Common Sources of Customer Feedback • Custom-designed tracking surveys • Airport benchmarking surveys • Pop-up surveys when signing on to free Wi-Fi • Online feedback forms accessed via posted URLs or QR codes • Comment cards distributed and collected by airport staff • Feedback forms on the airport website • Programs to collect SMS [short message service]/text feedback • Analysis of messages posted to social media • Instant feedback devices, such as happy buttons Best Practice “ASQ gives us a benchmark comparison to other airports. We use the results for presentations to the board and the leadership teams. We use the information to stay updated on tenants and partners as well as any issues that need to be addressed.” —Research participant

Monitoring and Enhancing Customer Service 103 baseline is useful because some experience factors are consistently rated as more satisfactory (for example, helpfulness of airport staff) than other factors (such as time required to clear security). Managers can use both the standard analysis and the relative analysis to improve customer service and stand out from peer airports. In the standard key-driver analysis, the importance of each experience factor is based on the correlation between the ratings of that experience factor and the ratings of overall satisfaction. This approach has a number of potential problems. First, it is based on the questionable assump- tion that a higher correlation indicates a stronger causal relationship between the experience factor and overall satisfaction. Instead, the correlation may be the result of any number of non- causal factors. Second, the correlation is based only on the respondents who rated both the experience factor and overall satisfaction. A varying proportion of respondents commonly fail to rate every experience factor, particularly when the possible responses include a “not applicable” option. It is not clear that an experience factor that has a high correlation with overall satisfac- tion but is based on a minority of respondents should be considered more important than a factor that has a lower correlation that is based on most respondents. Advanced correlation and regression analysis can be used to more fully understand the relation- ships between experience factors and overall satisfaction. However, such analysis is complex, difficult to standardize across sampling periods, and only addresses some of the problems just discussed. A simpler, alternative form of importance analysis is to ask respondents to explicitly indicate the experience factors that were most important to their overall satisfaction. Each factor can then be assigned points based on the number of times it was selected as important, and the point totals for each factor can be plotted on the Y-axis of the key-driver grid. As with the alternative metrics of performance, managers can use both the correlation-based and explicit importance analyses to better understand potential drivers of overall satisfaction. Figure 7-1. Example of key-driver analysis grid.

104 Guidebook for Conducting Airport User Surveys and Other Customer Research 7.3.2 Dashboards to Communicate Monitoring Results Dashboards are a particularly effective way of communicating the results of monitoring studies. They provide a condensed summary of the most important results, usually in the form of multiple data visual- izations such as bar charts, trend lines, and gauges. For example, a dash- board might include the grid representing the results of a key-driver analysis. Effective dashboard design is a large topic beyond the scope of this report, but in general, dashboards are most effective when they (Few 2013): • Link explicitly with established airport goals. • Serve as a basis for specific insights told in a separate, story-based form. • Are used regularly and broadly within the airport administration. (Online dashboards often record who uses them and how often they are used.) Figure 7-2. Example of average satisfaction ratings versus satisfaction ratings relative to peer airports. Dashboards “I think dashboards are a hot thing right now and have been for a few years, and so that’s been the solution that I’ve employed, to the extent possible if you’ve got one singular area of the business you’re trying to understand. If it’s restrooms, just piece together all the data points onto a single dashboard that can give you a holistic summary. It’s not tremendous for insights, but it seems to feed that [information], and at least enabling people to access the data.” —Airport focus group participant

Monitoring and Enhancing Customer Service 105 7.3.3 Monitoring Surveys and Actionable Results Monitoring surveys drill down from overall satisfaction to a range of more specific experience factors. From there, key-driver analysis is an important tool to identify the factors that should have high priority for management action. However, the experience factors do not always provide sufficient detail to show managers what actions are most likely to improve customer service. Airport experiences are complex, and a questionnaire (or set of questionnaires) that included questions about every aspect of airport experience would be impractically long or too costly to administer regularly. Accordingly, monitoring surveys are not always the ultimate tool to help managers choose specific actions. Instead, monitoring surveys should be thought of as a means of raising warning flags that serve a critical function in improving customer experiences. Managers may some- times need additional analysis or research to drill down to the level of actionable insights (see Chapter 8), but whether they provide general or specific warnings, monitoring surveys are an essential tool for improving customer service.

106 Targeted Studies of Specific Issues 8.1 Introduction Airport user studies can be classified into two general categories: (1) monitoring research that is conducted on a regular schedule, and (2) targeted studies that are designed and conducted ad hoc to address specific issues. The primary function of monitoring research is to help managers track customer experience over time using research methods that stay generally consistent. One might think of them as periodic medical checkups for the airport that include metrics analogous to body temperature, blood analyses, and assessment of heart and lungs. In contrast, targeted studies are like medical tests focused on the diagnosis and measurement of specific health issues. The methods are designed for the specific situation in order to better understand an issue or measure the magnitude of a problem. 8.2 Situations That Call for Targeted Studies The issues or problems that merit targeted studies are identified in a variety of ways. Several of the primary situations that call for targeted studies are described in the following paragraphs. 8.2.1 Research Questions from Managers Some issues that arise in the course of normal airport business translate directly to research questions. For example, an airline might reject gate assignments because their passengers experience signifi- cant inconvenience when the assigned gates are located in a concourse accessed by train. A study approaching representative samples of passengers in that concourse and a concourse that does not require a train ride could ask passengers about the ease of getting to their gates and convert this to an empirical discussion. It can be difficult for airports to recognize such opportunities where relatively simple studies produce immediate benefits. As research programs mature, the usefulness of targeted studies becomes more evident, and managers tend to recognize more opportunities to conduct them. 8.2.2 Monitoring Surveys Indicate Customer Dissatisfaction The results of monitoring surveys may also prompt targeted studies. By their nature, monitoring surveys often produce general indications that an issue or problem is present (see Chapter 7 for more discussion). C H A P T E R   8 Success “ASQ has told us that our checkpoints were the #1 driver to satisfaction. We decided to [redirect] a significant amount of funding from the annual survey to checkpoint surveys, Terminal A Checkpoint 1 and 2 surveys, to ask customers about different aspects of the checkpoint experience—entry staff, screening, recompose area, music level, everything that we could change if we were told we should. At [Checkpoints] A and B, we looked at where we were deficient and learned that we could get 5-star ratings despite the fact that customers said wait times are too long.” —Research participant

Targeted Studies of Specific Issues 107 Such indications may be insufficient to support an actionable diagnosis of the issue’s root cause. Imagine a customer satisfaction survey that shows low satisfaction with the “comfort of the gate areas.” Such a result is important but incomplete. What should managers do to increase satisfaction? Is the problem temperature, noise, entertainment, or seating? Targeted research can be used to drill down from this general finding and identify root causes that translate directly to management action. 8.2.3 Complaints in Unstructured Customer Feedback Many airports obtain and analyze customer feedback from a variety of unstructured sources, including comment cards, hotlines, and social media (see Chapter 7 for more discussion). Commonly, targeted studies are used to better understand and quantify the results of such analyses. For example, an airport might observe an increase in the number of comments about waiting at baggage claim. A targeted study might interview passengers waiting for their bags to better understand that experience, or the study might measure the prevalence of dissatisfaction by asking a representative sample of passengers leaving baggage claim to rate their satisfaction with the wait. 8.3 When to Avoid Targeted Studies Targeted research is often extremely valuable, but there are also situations where a particular study should not be conducted, some of which are described in the following subsections. 8.3.1 When Research Results Will Not Prompt Management Action Airports have too few resources to conduct unnecessary research. Before designing a targeted study, managers should first consider all the possible research outcomes and ask themselves if any of those outcomes will prompt them to act. If not, then the research is not needed. Likewise, if the course of action is already determined regardless of the research results, then research efforts should focus elsewhere. 8.3.2 When Research Is Likely to Reveal Undesirable Results or Force Action Research projects can seem attractive as a way to justify a preselected course of action or to demonstrate action on a difficult issue. In such situations, it is important to imagine the impli- cations of all the possible research outcomes. What if the results suggest that the preselected action is wrong? Also, asking about a difficult issue means that you assume a responsibility to act on the results. Airport research is in the public domain, and some questions may be better left unasked. 8.3.3 When Research Results Are Unlikely to Be Accurate In some situations, bad data are worse than no data at all. Targeted studies may be so subject to response biases that the results are more likely to be misleading than to provide accurate insight. Self-reports of socially desirable behavior are a classic example. For example, it would be foolish to estimate concession employee income from tips based on the incidence and amount of tipping reported by passengers. Response bias and research-demand characteristics (when respondents alter their responses to fit their interpretation of the study’s purpose) are common and should be considered before researching any potentially sensitive issue.

108 Guidebook for Conducting Airport User Surveys and Other Customer Research Targeted studies may also produce inaccurate results if insufficient survey staff resources are available for the study, an insufficient number of potential study subjects meet the desired criteria for participation, or too few respond to a survey conducted as part of the study. 8.4 Preparing to Conduct a Targeted Study The success of a research study depends on defining a good research question. A good ques- tion addresses the highest-priority information need, will be answered clearly by the research results, and when answered, will translate directly to management actions. There is no single convention for developing good research questions, but one activity that can prove useful is the “5 Whys.” In a 5 Whys exercise, participants repeatedly ask “why” about a problem or issue until a root cause is identified. The 5 in the 5 Whys exercise is not always necessary. Sometimes a root cause is identified more quickly. For example, managers might ask: 1. Why is an airline requesting new gate assignments? – Because they believe the train ride to their assigned gates inconveniences passengers. 2. Why does the airline have this belief? – Because of complaints from some of their passengers. 3. Why does the airline believe that these complaints represent the views of most passengers? – Because there has never been a study to systematically measure the effect of the train ride on passenger experiences. The 5 Whys method is useful because it drives the identification of a specific information need that can be translated into a good research question. In this relatively simple airport example, the research question could be, “How does the ease of accessing gates in the concourse with the train compare to a concourse without a train?” In more complex situations, the 5 Whys The 5 Whys The 5 Whys technique was developed by Sakichi Toyoda and used by the Toyota Motor Corporation as a critical component of problem solving in the Toyota production system (Ohno 1988). The key benefit of the technique is to encourage teams to avoid assumptions and logic traps while tracing the chain of causality from the effect down to a root cause or causes. The process starts with writing down the problem and making sure everyone on the team understands it. It is important that the team make answers precise and distinguish causes from symptoms by paying close attention to cause and effect. The root causes can be reviewed by reversing the sentences created by the analysis using the expression “and therefore.” The greatest potential weakness of the 5 Whys process is overconfidence in the initial answers. In answering each why, it is crucial to ask, “How might this answer be wrong?” This may seem to be an obvious and simple thing to do, but it runs counter to the normal pattern of human reasoning. By nature, we look for the reasons why our answers are correct. Looking instead for the reasons we might be incorrect is an important practice for improving critical thinking and an essential part of designing good, targeted studies.

Targeted Studies of Specific Issues 109 exercise may identify multiple information needs that require research. The best course of action can then be determined by assessing which combination of research questions must be answered to direct management actions. Perhaps the ideal outcome arises when the 5 Whys process shows that the problem is understood well enough that the needed management action is obvious, and research is not necessary at all. 8.5 The Type of Study That Should Be Conducted Customer research is generally classified as: (1) qualitative studies in which open-ended comments collected orally or in text are assessed for themes and concepts, or (2) quantitative studies in which counts of behavior or survey scales are used to create numeric measures. Depending on the research question, targeted studies might use qualitative, quantitative, or a mix of both research methods. 8.5.1 Targeted Qualitative Studies Common sayings such as “You can’t manage what you don’t measure” suggest that quanti- tative methods are the default form of applied research. However, before measuring anything, managers and researchers should first ask themselves whether they know enough about the situation to be confident that they are measuring the right things. Recall the earlier example in which a monitoring survey showed low satisfaction with the comfort of the gate areas. Researchers might design a questionnaire measuring satisfaction with the factors that they think are affecting comfort, but it is quite possible that they will guess incorrectly. Ideally, the researchers would instead conduct qualitative research, talking to passengers and hearing them describe what makes them comfortable and uncomfortable. Such qualitative work explores the conceptual space related to comfort at the gate, increasing confidence in the airport’s overall understanding. (See Chapter 6 for a broader discussion of qualitative research methods.) The following discussion will show that this understanding can complement and enhance other types of research. 8.5.2 Targeted Quantitative Studies When those involved in planning the research are confident that they will be measuring the right thing, quantitative studies offer powerful advantages. Many research questions concerning psychological constructs such as satisfaction are inherently quantitative—how many, how much, how satisfied, how long? And in the case of most airports, they are best answered using surveys. (See Chapter 2 for a broader discussion of quantitative research methods.) Targeted surveys are generally conducted as one-off studies that focus on fully understanding a specific research question. In contrast, monitoring surveys (see Chapters 7 and 10) are repeated to track one or more general indicators over time. 8.5.3 Combining Qualitative and Quantitative Measures in Single Studies It is extremely difficult to combine qualitative and quantitative investigation in a single study. In the most common studies combining the methods, coding of open-ended comments is used to produce and compare frequency counts for different themes or topics. Such analyses have several significant weaknesses. First, the frequency with which respondents spontaneously mention a given topic is not the same as the frequency that respondents will report that topic when asked about it directly. For example, an issue that is of great importance to relatively few

110 Guidebook for Conducting Airport User Surveys and Other Customer Research passengers, such as crowding in the VIP lounges, may be much more common in some open- ended feedback channels than issues that are annoying for virtually all passengers, such as how noisy the gate areas are. Prioritizing management action based on the relative prevalence of those issues may not produce the optimal improvement in passenger satisfaction. Even if the validity of topic counts from qualitative sources is assumed, qualitative studies often have too few respondents to support confidence in their numeric conclusions. Quali- tative studies tend to be time-consuming and costly, so they rarely include large numbers of respondents. At the other extreme, large numbers of qualitative responses can be collected via social media or as comments from other feedback channels. The problem with quantifying those responses is that no matter how large the source of the data, there is no certainty that respon- dents are an unbiased representation of all passengers. Insights that require answers to both qualitative and quantitative questions generally require a sequence of studies. 8.5.4 Qualitative and Quantitative Studies in Sequence Qualitative and quantitative research methods are ideally used so that their strengths are complementary. Recall the example of low satisfaction with the comfort of the gate areas. The earlier discussion pointed out how qualitative research was initially the best approach— to deepen the airport’s understanding of the factors that influence comfort. Based on that deeper understanding, researchers could confidently design a survey question- naire that would numerically measure satisfaction with the factors so they could be compared to assess which of them were the key drivers. The example focused on gate comfort is not hypothetical. A large U.S. airport identified “comfort of the gate areas” as a high-priority factor in their customer experience based on an ASQ monitoring survey. It then used qualitative interviews to identify 15 different factors affect- ing gate comfort. Finally, quantitative surveys were used to measure the importance of each factor and the degree to which customers were satisfied with each factor. The survey results showed that the key factor in gate comfort was a lack of seating in the areas near customers’ gates. Airport managers cited these findings when deciding to spend more than $200,000 on new seats and the reconfiguration of seating around multiple gate areas. 8.6 Practical Considerations Conducting targeted studies, particularly paired qualitative and quantitative studies, may seem impractical at particular airports. How- ever, the process of identifying specific information needs described in this chapter should prove useful for understanding issues at any airport, and the combination of multiple targeted studies is an extremely powerful tool that could help many airports make better decisions and choose more effective management actions. Considerable value comes from recognizing the situations when targeted studies would be useful and working through processes to identify the specific information gaps and research questions that should be addressed. In some situations, that examination may reveal that the problem is understood well enough that the needed management Success “The wayfinding one [is an example of a successful application of research] because of where they were in the process. Research was done early on, the way we did it and turnaround time, they had already developed sign design and things were in physical production, we were willing to test with mockups, live test it, and then have the wherewithal to say we’re willing to make changes. It’s not an easy thing to do, and a lot of agencies would say let’s just go with what’s been done before rather than listening to their users about what they like best. Some of the feedback was helpful to refine the signage. They didn’t go back to the drawing board. It was costly to make changes to their designs, but not as costly as if they had to change all of their designs at a later point.” —Research participant

Targeted Studies of Specific Issues 111 action is obvious. Even when research is called for but cannot be conducted due to a shortage of time or resources, the extent to which the airport is making a decision based on expert judgment will be clarified. In the end, there are many situations in which conducting targeted research is money well spent. It is not uncommon for airport managers to make decisions involving millions of dollars. Targeted research can help them make informed decisions that focus improvement programs on the issues that will most effectively address airport problems. When the full cost of making uninformed decisions and taking ineffective actions is considered, research that at first appears complex and costly is often an excellent value.

112 New and Developing Data-Collection Techniques 9.1 Introduction One of the key objectives of updating ACRP Report 26: Guidebook for Conducting Airport User Surveys was to include information on new and emerging data-collection methods being used by airports and provide examples illustrating the use of highlighted practices. To achieve this goal, the project team conducted telephone surveys, case studies, and focus groups of airport representatives. Individuals from firms that specialize in processing data from cell phones and Wi-Fi were also interviewed. Cell-phone data include data that are derived from the cell-phone signal itself and are tracked by the cell-phone service provider as well as location data obtained from the global positioning systems (GPS) receiver in the smartphone that are reported through an app running on the phone. Finally, the team conducted an interview with a marketing expert from a major hotel chain. This chapter draws heavily on examples gathered through these efforts. 9.2 Overview of New and Developing Data-Collection Methods Overall, 80% of respondents contacted through the telephone survey stated that their air- port had conducted customer research in the previous 5 years. For the purposes of the survey, “customer research” was defined as follows: By customer research is meant research among air passengers; meeters, greeters, and well-wishers; and people who use related services operated or hosted by the airport, such as parking facilities, car rental services, and hotels. Content analysis of social media posts was mentioned by 57% of respondents, and 28% noted conducting micro-surveys that pop up when someone connects with Wi-Fi. Among those stating that they used social media, all had used Facebook and Twitter, and 92% had used Instagram. Other social media platforms mentioned more than once included Google Reviews, LinkedIn, and YouTube. The platforms mentioned are generally consistent with those reported in ACRP Synthesis 56: Understanding the Value of Social Media at Airports for Customer Engagement (Perry, Damian, and Lagu 2014), but there have been some changes. For example, in ACRP Synthesis 56, representatives from 17 airports were interviewed, all noted using Facebook and Twitter, and 71% had used Instagram (Perry, Damian, and Lagu 2014). In the 2020 telephone survey, airport representatives were also asked about other sources of information they used to conduct research studies. Airports could indicate more than one data source. A total of 13% of respondents indicated that they had used financial transaction data, 9% had used cell-phone tracking data, and 4% had used airport beacon data. All of these new and developing data-collection methods for customer-based airport research are covered in this chapter. C H A P T E R   9

New and Developing Data-Collection Techniques 113 9.3 Content Analysis of Social Media Posts Multiple airports are using social media as sources of research information. Typical ways include: • Monitoring what customers are saying about the airport to identify potential issues, • Interacting with customers to address issues and identify employees for recognition, • Communicating with customers to promote the airport, and • Conducting surveys. While airports are using social media in various ways, the population of airport customers is not the same as the population of airport customers on social media. That is, given that qualita- tive and quantitative research methods are aimed at describing the full population of airport customers, the use of social media alone to interact with customers will not be reflective of the entire population of customers. Multiple airport representatives emphasized the need to monitor and respond to posts on social media to help address issues. As noted by two airport representatives, typical customer experiences that are monitored on social media include “issues regarding airlines, TSA, police, parking, Wi-Fi, restaurants, wait times, terminal cleanliness, etc.” and “retail, concessions, and food and beverage offerings we have in the terminal.” This is one way of closing the inner loop with customers that was discussed in Chapter 7. Sentiment analysis is one common technique that airports use to analyze media posts. The number of times a particular issue is being talked about in positive, negative, and neutral terms is tallied across different social media platforms, and potential problem statements to be addressed are generated. The ability to compare what is being said across multiple digital sources can be helpful for prioritizing responses; as one airport representative noted, “if all of the sources—like Facebook, Twitter, Yelp, Google, and others—are saying the same thing, then there is a problem.” Airport representatives mentioned a variety of software programs that they have used to analyze social media posts, including Hootsuite, Meltwater, Optimus, Salesforce, Semantra, Tatvam, and WordCloud. Among these platforms, only Hootsuite and Meltwater were men- tioned in ACRP Synthesis 56. Other software mentioned in the telephone interviews included Kipsu, a program that analyzes text messages, and Tableau, a data-analysis software program. The diversity in software programs noted by airport representatives is not surprising, given the number of companies that have emerged over the past few years focused on monitoring and analyzing social media posts. What is more interesting though is the frequency with which airports are generating sentiment reports and monitoring what customers are saying. The fre- quency varied across airports, and no pattern was observed by airport size. For example, among those airport representatives that mentioned how often they were generating reports, one small- hub and one medium-hub airport generated monthly reports, one large-hub airport generated weekly reports, and one small-hub airport generated daily reports. Many airports use social media to interact directly with customers, and several airport repre- sentatives noted that social media is another tool for customers to contact them, along with phone calls and email. Kipsu is used at one large-hub airport to route texts to an information office, from which people receive a real-time response (from 8:00 a.m. to 8:00 p.m.). One small airport has taken this philosophy of using social media to interact with its customers a step further, actually reaching out to frequent social media users, particularly those who have negative things to say about the airport: Sentiment analysis identifies what is being talked about and whether the statements being made are positive, negative, or neutral.

114 Guidebook for Conducting Airport User Surveys and Other Customer Research We watch our reviews, our ratings, we flag those who are frequent travelers and people who comment frequently, and we’ll do things like reach out. We had one frequent business traveler who was really down on the airport [and] was saying all kinds of negative things about the airport on social media. So, we reached out to him and offered to meet him in our restaurant and buy him a cup of coffee and have a one-on-one conversation. We’re doing more of that and developing true personal relationships with people who are here all the time, and it’s amazing what you learn and how they turn around and become your biggest fan on social media as well. I don’t know if that’s a small airport thing or just a wave of what the world is like with social media now, but it’s very worthwhile. In addition to monitoring customer sentiment and interacting with customers, numerous airport representatives noted using social media to, as one stated, “promote the airport’s services and amenities as well as share news and updates.” Multiple airports use social media for market- ing campaigns and monitor the success of these campaigns via metrics like click-through rates and engagement with posts. One airport representative mentioned using social media to provide coupons for their parking sites in exchange for the recipients providing their email information (to use for future customer surveys). Another airport representative noted working with a marketing firm to do paid digital advertising through search engines to promote parking and concessions. Another example of using social media to promote the airport that an airport representative noted was this one: We use social media to engage with people. We take quick polls, ask people to post favorite photos, and offer contests. We also use it to engage with people regarding local events and celebrations, such as March Madness. As noted earlier, many airports use social media posts to identify potential issues, but it should not be forgotten that social media inter- actions can also be used to recognize employees and identify opportu- nities for airports. For example, one airport representative mentioned using social media posts to determine which employees were provid- ing great service and to subsequently reach out to those employees to congratulate them. Another airport posts QR codes throughout the airport. By scanning the QR code with their phones, individuals are directed to a website where they can provide feedback. One airport representative discovered a merchandising opportunity: We track and react to passenger feedback through social media. We look at passenger complaints and suggestions. For example, we had passengers requesting that certain team merchandise be sold in our stores; we shared the requests with the vendors so they could stock what the customers were looking for. Finally, social media is often used in conjunction with other data-gathering efforts to conduct customer research. For example, one airport representative stated that as part of rebranding research: We reached out to the public to gather information regarding their views of the airport. We posted the survey on our website, sent email links to our database and social media, and offered a tablet survey option at the airport. It should be borne in mind that the population of airport customers on social media is not the same as the population of airport customers, and while surveys conducted through social media can be informative, they should not replace methods designed to reach the broader airport customer population. When using social media data to inform decisions, it is often useful to know how large the population of social media is at a particular airport and how their concession sales compare to the general population of customers. Also, it is important to note that there are potential trade-offs between instant feedback devices and scannable QR codes. Instant feedback tends to provide larger sample sizes, whereas QR codes tend to provide more detailed insights. Airports have used social media posts to identify opportunities to sell new merchandise.

New and Developing Data-Collection Techniques 115 9.4 Wi-Fi Micro-Surveys At many airports, it is common to ask customers to answer a few short questions when they access Wi-Fi. Multiple airport representatives reported success with using micro-surveys that pop up on Wi-Fi, although four airports noted that they had discontinued requiring Wi-Fi surveys to log on due to negative reactions from passengers. Interestingly, among those airports that had discontinued requiring Wi-Fi surveys to log on, three were large-hub airports and one was a medium-hub airport. It may be that Wi-Fi micro-surveys are better received among customers using smaller airports. Most pop-up micro-surveys that airports use contain three to five questions. If airports have more than five questions to ask on a specific topic, these questions are divided across multiple micro-surveys. Topics covered are broad and include ambience, cleanliness of the terminal, wayfinding, signage, staffing, concessions, and the availability of electrical outlets. Airports also commonly use them to gather information on customer demographics, the type of travel customers are doing, where they are coming from, how they arrived at the airport, and the airline they are flying. Micro-surveys can also be used to home in on specific topics such as ongoing construction at the airport. Several airport representatives noted that they liked the Wi-Fi surveys because they offer more real-time feedback, “allow us to get a very large sample set in a short period of time,” and “give us the ability to instan- taneously change questions as needed.” One large-hub airport reported receiving 2,500 Wi-Fi responses a day, and two medium-hub airports reported receiving “thousands of responses a day” (pre-COVID). One airport noted, “Wi-Fi survey is critical to the marketing and commu- nication teams.” Several airports were using Wi-Fi data in creative ways. One airport asked for the airline the customer was flying to determine which part of the concourse they were in. Another airport used a beacon sensor to anonymously pick up on Bluetooth and Wi-Fi from passengers’ phones to help it track the average security wait times so that it could inform TSA on their performance. Another strategy that one airport representative noted for increasing survey participation was to offer basic Wi-Fi for free to all customers, but to require an email address and basic information for customers who wanted to access higher-speed internet. This allowed the airport to keep track of unique travelers and how many times they had traveled through the airport so that it could create lists of frequent travelers. This list was subsequently used in market research studies of what it termed “road warriors.” While asking those who log on to Wi-Fi to complete surveys is convenient, the results may be biased and may not be reflective of the entire population of airport customers. If the airport is asking questions about Wi-Fi service, there is no bias because it is in contact with the sub- group for whom the questions are most relevant. However, if the airport is asking about other things, the reported rates in the survey may differ considerably from the population rates. This limitation may be of lesser concern when interpreting differences rather than estimating the true population rates. For example, if the survey shows that Concourse A has fewer people than Concourse B who reported finding the shops they wanted, the airport can have more confidence in the difference than in the accuracy of the actual percentage of people who found the shops they wanted in either concourse. Online surveys that pop up when airport users access the airport Wi-Fi are also discussed in Chapter 10. Wi-Fi surveys provide airports the ability to obtain real-time feedback.

116 Guidebook for Conducting Airport User Surveys and Other Customer Research 9.5 Real-Time Customer Monitoring Given that airport checkpoints and restrooms are key drivers of customer satisfaction, airports use a variety of tools to monitor the status of these areas of the airport in real time. These tools include devices placed in restrooms, baggage claim areas, at the end of TSA checkpoints, and other areas of interest. These devices gather customer sentiments on the condition of the area and enable real-time monitoring. Information obtained from these devices can be used to improve the customer experience and engage with other stakeholders such as TSA and custodial staff. Feedback Now, Happy or Not, and Avius are companies that airport representatives mentioned using for real-time customer monitoring. Some of these companies provide the ability to bench- mark against other airports. At the time of writing, COVID-related concerns were prompting at least one of these companies to move toward touchless technologies that would enable gath- ering feedback via facial expressions or hand gestures instead of having customers touch the screen. During the pandemic, one airport found that its restroom ratings “took a dive” and that customers were placing a higher importance on cleanliness. Whether customers will continue to place a higher emphasis on restroom cleanliness post-COVID remains to be seen, but if they do, then these real-time monitoring programs will likely become more important. 9.6 Other Data Sources As noted in the overview, airports reported a wide variety of other data sources they used for customer research. This section addresses two of the most common examples noted by airports: airport beacon data and user location data. 9.6.1 Airport Beacon Data Airport beacons are small devices that broadcast a short-range Bluetooth signal. Mobile device apps can detect the Bluetooth signal when they are in close proximity to the beacon. Tracking data with airport beacons is not common across airports but has been used to passively and anonymously pick up on Bluetooth and Wi-Fi connections from passengers’ phones in TSA lines. This allows the airport to track average security wait times, post this information on the airport’s website, and work with TSA as necessary to adjust its staffing. Several airlines also reported using cell-phone tracking data for this purpose. 9.6.2 User Location Data Multiple data sources, including cell phones, GPS, and Wi-Fi, can be used to locate users. Data from these three types of sources all provide the ability to locate individuals but differ in their costs, coverage, and precision. Table 9-1 provides a summary of the key aspects of these data types. Cell-phone geo-location works from the signals transmitted between cell-phone towers and handsets and uses time differences between signals received at different towers. GPS geo-location uses a GPS receiver in a smartphone handset to determine the phone’s loca- tion from signals received from GPS satellites and then reports its location to a provider of an app running on the smartphone when that app is used. Wi-Fi geo-location works when a smart- phone user accesses a Wi-Fi system with multiple Wi-Fi hubs in an area (such as an airport), and the Wi-Fi hubs determine the location of the smartphone by measuring time differences in signals from the smartphone received at different hubs. Cell-Phone Data Some airports have used cell-phone data to better understand passengers using the airport or to target potential customers. As part of the interviews the research team conducted, one

New and Developing Data-Collection Techniques 117 of the more interesting applications of cell-phone data within a travel context was related to a study that was conducted for Destination Canada in British Columbia based on cell-phone data from Rogers, a Canadian phone company. Destination Canada is an organization that promotes tourism in Canada. In Canada (but not in the United States) telecommunication companies like Rogers have the ability, as part of studies that are made public, to identify individuals who are roaming internationally. As described by a representative from the firm that conducted this study: Destination Canada wanted to understand where the tourists were coming in, how long they stayed, what areas of British Columbia they went to, which places they visited if they were in Vancouver, etc. We did a 3-month study over the summer, and what it revealed to Destination Canada was that they had been making a lot of assumptions based on passenger manifests from airlines. What they discovered was that there were enough people coming across the border from Seattle and other places that they had been making assumptions that were badly skewed. They had far more tourists coming from Japan than they ever recognized. As a result, they were able to redirect their marketing dollars to focus more on Japan and other destinations they had overlooked in the past. One airport representative noted that by establishing a geofence around its largest competitor, it was able to target electronic ads to individuals who may not have been aware of the airport or may not have considered it: We established a geofence around our largest competitor, and we are able to target electronic ads when [individuals] within our catchment area enter the competitive airport. Ads are pushed directly to them through different sources and websites they may go to and [inform them] that “hey, you have a local airport that you can use.” The click-thru rate has been just astoundingly good, like 80%. So even though it’s a small number of people, it’s having the full effect of what we’re trying to do. We have a lot of inter- national students and a couple of universities in our area, and a lot of them just don’t realize that there’s an airport [here]. They just only think of the international airport. An industry expert from a major hotel chain noted that cell-phone data have been used for retail path research. The goal of retail path analysis is to follow the path that potential customers take through a store and understand how much time they spend in the store, which aisles they go down, and so forth. Historically, cameras have been used for detailed path analyses and to examine questions such as how customers react to a storefront display, which items they pick up in a store, and so on. Another industry expert, who conducts analysis on cell-phone data, described how this process would work at an airport. In an airport, as phone owners move, their phones go through different microcells. As the name suggests, microcells are small cell-phone towers that provide Cell Phone GPS Wi-Fi Cost High Low Low Coverage Location points about every 2 minutes About 100 location points per day (depending on the extent of app use) Multiple location points when Wi-Fi is turned on and individual is in range of transmitters Location accuracy Moderate High Low Ability to identify home and work locations Reliably Dependent on app use Cannot be used to determine home and work locations Typical airport applications Catchment analysis, tourist destination analysis, potential for retail path research Catchment analysis, targeted marketing, travel patterns Limited retail path research Table 9-1. Comparison of cell-phone, GPS, and Wi-Fi data. The strength of cell-phone data is the ability to track individuals about every 2 minutes.

118 Guidebook for Conducting Airport User Surveys and Other Customer Research connectivity to the cell-phone network over a limited area. He noted that the ability to track pas- senger movements along a concourse depends on how far apart those microcells are and how accurate the geo-positioning is along the airport concourse. Many large airports have microcells that allow for accurate tracking of passengers—for example, some airports have used cell-phone data to track passenger movements through the airport. The location accuracy of cell phones is not of a quality that allows the researcher to see what specific areas within a store are visited, but cell-phone data can provide information on the stores or general shopping area that the customer visited. The real strength of cell-phone data, compared to other sources, is its ability to track individuals every 2 minutes or so. This is why the use of cell-phone data is particularly helpful for understanding specific components of an individual’s journey and why it is being envisioned for real-time or near–real-time applications. For example, one of the industry experts noted that these data can “be used to determine if people who are flying into Atlanta [for example] are renting a car and dispersing around the southeast or returning home.” Another industry expert noted that his firm had worked with several regional airports to help them understand who parked in the airport parking lot versus in off-site lots, where they came from, and how long their vehicles stayed in the lots. One airport representative noted that she would like to use cell phones to track passenger movements through the airport, provide push notifications for discounts at concessions, and identify wayfinding issues. A push advertisement or discount code for a specific store would be sent by the cell-phone provider to an individual’s phone (for example, via a text message) when the passenger was approaching that store. GPS Data In contrast to cell-phone data, GPS data tend to be less expensive to purchase and provide more accurate positioning; however, they are not as comprehensive because location-based data are collected passively from apps on a smartphone and only about 200 location points are collected per day. That is, compared to cell-phone data, GPS data have greater location precision and are less expensive, but do not provide as much coverage (in terms of location data points in a day). GPS data are also likely to be more biased than cell-phone data. In part, this is due to how GPS data are collected. For example, individuals who are constantly using apps on their phones will have more location points than individuals who use apps less frequently. In addition, many phone carriers are providing the ability for customers to turn off geo-location services, and those who turn off location-based services may have different travel patterns from those who use the location-based services. GPS-based data are primarily used for catchment analysis and real- time targeted marketing. The Mobile Ad ID is a unique identifier assigned to a mobile device. However, as one of the industry experts explained, “the Mobile Ad IDs are often tagged to the cell-phone carrier’s main office, so there is a lot of noise in that data. The Mobile Ad IDs are instant data that provide a sighting, but they can be misleading. They can help with real-time targeted marketing [but for other real-time applications, cell-phone data would be preferred].” In summary, the interviews with industry experts identified several uses and applications of GPS-based data that could be used for airport applications such as catchment analysis and to do targeted marketing for concessions (for example, by delivering coupons for particular stores) via the Mobil Ad ID capability. The strength of GPS data is their ability to identify individuals with a high degree of precision.

New and Developing Data-Collection Techniques 119 Wi-Fi Location Data Similar to GPS-based data, Wi-Fi data provide a less-expensive alternative than cell-phone data for locating and tracking the move- ments of individuals within a building. Steve Hornyak, an entrepreneur who previously worked for a company that processed Wi-Fi data, notes that in the past he worked with airports that were “trying to figure out how to use Wi-Fi data to study traffic flows and how much time people spent in different terminal locations. If devices have their Wi-Fi turned on, you can figure out where their users are, where they are stopping, which stores they are passing. So, it was being used more for traffic flow management than retail purposes.” However, Hornyak noted that using Wi-Fi for retail path analysis “didn’t work very well and wasn’t very accurate” and that the best applications were focused on understanding general traffic-flow patterns and dwell times in airports. Hornyak also noted that the sample obtained from Wi-Fi tends to be biased. “With privacy concerns, Apple and Google are [making changes to the operating systems that] allow people to turn off the ability to track them—that will change the sample size and the consistency in the type of sample you get as well.” In summary, several airports have explored using Wi-Fi data for retail path analysis and other applications, but these applications have not been as effective as more-expensive applications that use cameras to study to movements and reactions of individuals within a store. Wi-Fi data are also limited in that they do not capture the full journey of the individual, so home and work locations (and associated socioeconomic and sociodemographic characteristics of the home location) are unknown. Combining Location-Based Data and Smartphones One airport representative provided details on how location-based data are being combined with smartphone technologies to survey passengers directly via their smartphones without using Wi-Fi or requiring the use of terminal-based signal technology. First, a GPS perimeter or geofencing is established around an airport terminal by a GPS aggregator using specified GPS coordinates. The GPS aggregator typically has access to 75 or more mobile apps and uses pop-up text notifications or other advertisements to invite passengers to take a survey on their smartphones. The advantage of using this approach is that the potential survey population is larger than the population that just accesses the airport Wi-Fi. Further, since the passenger is tagged by the geofencing, survey notifications can be sent out anywhere in the United States within 72 hours or more of visiting the terminal—that is, the survey does not need to be com- pleted at the airport where the passenger was tagged. One airport that used this approach noted that it recommends using a cash incentive or sweepstakes entry to increase response rates, and that, depending on the survey topic, it saw participation rates of 30% or higher before COVID and 10%–15% during COVID. 9.7 Privacy, Bias, and Other Electronic Data Concerns Two primary cautions regarding the use of electronic data collection are worth discussing, both of which merit consideration when studies are planned and executed. The first is the issue of privacy and confidentiality in the use of such research methods as analysis of social media data and airport user online surveys. Care needs to be taken that studies are designed, executed, and reported in such a way that the identities of airport users are safeguarded. Second, there is the potential bias in responses in online surveys toward younger genera- tions. If alternative methods are not employed to capture the attitudes and behaviors of older The strength of Wi-Fi location data is their lower cost relative to cell-phone and GPS data.

120 Guidebook for Conducting Airport User Surveys and Other Customer Research customers, use of the findings for decision making and action run the risk of underrepresenting these travelers. A major caveat with the use of many of the data sources covered in this chapter is the extent to which the data are de-identified. In some cases, such as parking transaction data, the airport may know the general residential location of the customer from vehicle license plate data or from payment transactions (if the transaction is made by credit card, and zip-code information is recorded and stored). In other cases, the residential location of the customer will have to be inferred from the data. 9.8 Summary This chapter provided an overview of new and developing data-collection methods and techniques that have been used for airport customer research. Several of these data-collection methods, such as analysis of social media posts, are not new but are now easier for airports to use due to the number of software vendors available. Pop-up Wi-Fi surveys are common across airports—particularly small-hub and medium-hub airports—and are helpful for collecting quick feedback on targeted issues. Several strategies have successfully been used for encouraging individuals to take a survey when logging into Wi-Fi, most notably by offering a basic Internet service that can be accessed for free and an enhanced Internet service that can be accessed only after providing an email address and other information. Through the information provided, some airports have been able to access their Wi-Fi service to create databases of air travelers and identify frequent travelers. These databases have been helpful for conducting other consumer research. However, other airports have discontinued requiring a survey in order to access Wi-Fi due to negative customer feedback. Some airports have used logins from Wi-Fi to generate a contact list of customers who can serve on a research panel. Maintenance of research panels is not a trivial task, but it can be extremely valuable to have contact information for airport customers available as a recruit- ment source for focus groups or open-ended interviews, pretesting of questionnaires, or other situations where a representative sample of the full customer population is not essential. Another frequently mentioned data source used for customer research is user location data. User tracking data has been used in a variety of ways, including for estimating TSA queue lengths and wait times and for catchment analysis. User tracking data can provide insights into both who is using the airport and how they are using it—for example, whether they are parking in off-airport lots or visiting specific concessions. User tracking data have also been used to identify residents within a smaller airport’s catchment area when they entered a larger airport. Customized ads were sent to these individuals, helping to raise awareness that the smaller airport was an option they might have been able use for their journey. These are some of the many examples of how airports are using new and developing data sources for customer research. Looking ahead, it is expected that the use of user location-based data sources will become more common and that advanced methods will be developed to help ensure customer privacy and anonymity.