Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
55 The purpose of this synthesis was to summarize the current state of practice in transit rider surveys, most notably how emerging technologies and data sources are affecting the ways these surveys are conducted. While the synthesis includes examples from other types of surveys, the focus is on originâdestination (OD) surveys since they represent the most complex and cost liest type of survey regularly performed by a transit provider. Over the last decade, new technologiesâsuch as mobile devices, high-speed cellular Inter- net, and automated data collection toolsâhave changed how organizations conduct surveys. Moreover, the emergence of big data from fare cards, mobile phones, and GPS systems provides new alternatives to traditional rider surveys. This report follows up on prior TCRP research on the topic to document the changing state of survey practice. The synthesis was developed to provide an overview of survey practice methods and tech- nologies to transit professionals. The information presented is derived from: â¢ A broad literature review of existing TCRP reports, white papers, academic research, docu- mentation from prior survey efforts, and federal guidance. â¢ A survey distributed to 67 organizations responsible for overseeing such studies, and covering a variety of topics including the frequency of and justification for OD studies and how new technologies and data sources are integrated into survey efforts (see Appendix A for a copy of the questionnaire, and Appendix C for the results). â¢ Interviews with representatives from four transit providers and one metropolitan planning organization (MPO) on how technologies, new sampling techniques, and passive data can be integrated into OD studies and transit surveys in general. While new technologies and data sources have been changing both the methods and fre- quency of transit rider surveys in recent years, the fundamentals of good survey design and implementation have remained constant. Rider surveys, regardless of method, must overcome sample biases, reach a representative sample of transit riders, and be carefully designed to mini- mize misinterpretation of questions or response fatigue. The survey conducted for this report, along with the literature and case studies, highlights a wide range of practices in OD studies and transit passenger surveys. Some of the key findings include: Leveraging Passive and Automated Data Better data allow for improvements to sampling plans and response expansion: Automated data sources such as fare cards, automatic vehicle location (AVL), and automatic passenger counters (APCs) allow transit providers to develop more precise sampling plans, which can help increase the overall quality of survey results. These passive and automated data sources can also C H A P T E R 5 Conclusion
56 Public Transit Rider OriginâDestination Survey Methods and Technologies be useful inputs for expanding survey data; methods such as iterative proportional fitting would be much more challenging to implement without AVL or APC data. Passive data are powerful sources of information for transit providers but cannot completely supplant traditional surveys: Passive data and surveys both play a distinct role, and transit pro- viders should understand the trade-offs inherit in each data source when exploring how to utilize passive data. With surveys, transit providers can collect and cross-tabulate a wide range of rider characteristics and trip information. The downside, however, is that surveys are expensive, take a long time to conduct and analyze, and are susceptible to systematic bias. Passive data allow transit providers to collect information on a more narrowly defined set of characteristics (e.g., OD patterns) but across a large sample size quickly and affordably. This information will surely have an impact on how providers plan and monitor their systems, but cannot replace the range of information gathered in an on-board survey, including critical Title VI demographics. Survey Methods Tablets are becoming the preferred mode for collecting survey data: Transit providers felt that the benefits of utilizing tablets for surveys outweighed their downsides. The most com- monly cited benefit of tablets was improved data quality and reduced data entry needs. Tablet surveys yielded, on average, a higher response rate (when calculated by number of riders approached) and allow for a more sophisticated survey instrument that includes skip logic. The downside of tablets is that they are more labor-intensive because the survey is usually admin- istered by an interviewer instead of being self-administered by the rider. It is unclear whether the additional time spent administering the survey by employees counteracts the efficiencies gained from better data quality and reduced data entry costs; the literature review and survey provided inconclusive findings on this topic. Tablets need to be effectively deployed to yield benefits: Tablets by themselves do not neces- sarily yield any survey benefits over paper. It is the capability of tablets to validate data, reduce error, monitor progress in real time, and automate parts of the survey process that make them an appealing survey mode. Survey organizations may achieve poor results with tablets if such capabilities are not incorporated into the survey instrument. Transit providers report mixed results with two-step computer-aided telephone interviews (CATI): Metro in Los Angeles and MTC in the San Francisco Bay Area have both experimented with the two-step CATI method. Metro found that the method yielded improvements to data quality and high response rates among non-English speakers, typically an underrepresented population in rider surveys. By contrast, MTC continues to offer a CATI option to accommodate short trips and LEP riders but does not utilize it as their primary survey method. MTC found that CATI surveys suffered from a low response rate, raising survey costs and sample bias concerns. Survey Design and Fielding An organizationâs study objectives drive the ideal survey method and design: Transit pro- viders have managed the cost and complexity of their survey efforts by identifying their most crucial data needs. Collecting data to validate and support travel demand models requires large- scale surveys that include questions about a customerâs entire trip chain. Providers may choose to collect Title VI, customer satisfaction, fare usage, and sociodemographic data through a more limited survey with a smaller sample size. Information on boardings, alightings, and transfers can be collected by a simple ride check, or through passive data sources such as fare- card data and AVL or APC data. The varying purposes behind surveys can explain in part the large variation in survey cost and sample size. There are no transit industrywide standards for survey design and question wording: While there is literature on best practices in survey design and deployment, there are few
Conclusion 57 widely accepted standards within the transit industry. For example, when creating a new survey instrument, organizations often create questions from scratch or rely on previous examples due to the lack of standardized question wording. Moreover, survey methods vary widely across the industry, including the preferred sampling plan, survey mode, and expansion methodology. Finally, even some key measures used to monitor surveys, such as response rate, are inconsis- tently tracked or defined in different ways by the survey participants. This lack of standardized design and question wording makes evaluating the effectiveness of various survey practices a challenge. Transit providers find value in investing in internal capacity to design and deploy sur- veys: Transit providers often rely on contractors or outside partners to oversee, develop, and deploy passenger surveys. Conducting surveys in-house requires having staff with specialized expertise in survey design, deployment, and analysis, along with the equipment and data infra- structure necessary to manage surveys. The example of TriMet in Portland, Oregon, highlights the benefits of conducting surveys in-house, including knowledge retainment, increased flex- ibility to deploy surveys, and cost savings. Implementing an internal survey practice requires sustained investment and may not be feasible without the support of the organizationâs leader- ship or desirable if the agency conducts surveys too infrequently to realize cost savings. Common Challenges Organizations navigate a wide variety of survey methods and approaches: Transit providers face a daunting array of methods for conducting rider surveys. Choosing the correct approach is a basic challenge for every survey. Rather minor differences in methodology, such as the method used to accommodate short trips, question wording, or whether or not a pre-test is done, can greatly affect the quality of survey results. Originâdestination questions are frequently misinterpreted: Question wording and survey instrument design can greatly affect the quality of responses. Trip chain questions, which ask the public to describe their entire trip from origin to destination, are frequently misinterpreted. One participant in this study adopted an interview-administered survey approach after discovering that even their own agency staff did not fill out surveys correctly. Collecting a random and representative sample is an ongoing challenge: Case example par- ticipants felt that, even when utilizing recognized best practices in survey design and deploy- ment, they were unable to fully eliminate bias from their surveys. Organizations grapple with several sampling issues, including the bias of surveyors, undersampling of certain trip types and demographic groups, and systematic skipping of certain survey questions. Low-income riders, minorities, minors, and LEP individuals are frequently underrepresented in survey samples. Short trips are especially challenging to survey: Short trips are less likely to be encountered by surveyors because short-trip riders spend less time in the transit system. Such trips can suffer from a higher survey incomplete rate because respondents do not have the time to fully complete or participate at all in the survey. Survey organizations use a wide variety of strategies to increase records for short trips, ranging from allowing participants to complete the survey off-vehicle by paper, online, or over the phone, to the availability of a special short survey with only essential questions. Certain data expansion methods also allow surveyors to address systematic under- sampling of short trips. Future Research Questions To date, there has been no large-scale study evaluating the effectiveness of transit survey methods across a range of providers and regions. In the synthesis survey and case studies, this report yields sometimes conflicting findings. An approach that worked for one organization
58 Public Transit Rider OriginâDestination Survey Methods and Technologies was ineffective in another. Data collected in this report are insufficient to determine whether the success or failure of a survey method was attributed to the survey tool or other factors, such as availability of incentives, rider characteristics, or mode. A controlled study encompassing multiple transit providers would be needed to isolate the factors influencing survey quality and determine the ideal survey methods for different circumstances. Respondents to the synthesis survey defined or calculated key metrics such as response rate, response base, completed surveys, and survey cost in different ways. The use of passive data is in its infancy, and transit professionals are still refining and experimenting with how such data can be utilized to support or supplant survey methods. The current state of practice with passive data will likely be out of date in a few years. One limit of a synthesis report such as this one is that it captures current practice; additional research could explore theoretical and practical applications of passive data. Some of the key research topics for further inquiry could include: â¢ Standardization of survey questions, including an assessment of optimal wording; â¢ Establishing a consistent set of metrics to be reported on for study efforts, including a stan- dard method of calculating the response rate, count of usable surveys, and survey cost; â¢ Controlled study of various survey methods to determine the most effective survey mode and sampling strategies; â¢ Research on more sophisticated expansion methodology in a controlled environment where results can be validated against other data sources; â¢ Research on the emerging and theoretical uses of passive and big data to support or supplant transit OD surveys; â¢ Inquiry into the skills and training that transit providers and contractors need to better take advantage of new tools and techniques that rely on big data; â¢ Research into the potential of crowdsourced data (used extensively in public engagement) for OD studies; â¢ Research on the feasibility of collecting OD data voluntarily through a riderâs personal device (opt-in), including the response bias associated with this method; â¢ Assessment of incentives to increase response rates; â¢ Impacts of technology useâsuch as tablets and CATIâon the survey response rates of underrepresented populations; â¢ Understanding effective strategies to increase the participation rates of individuals with LEP; and â¢ Guidance on the appropriate sample size for OD surveys based on an organizationâs desired level of precision. This is an exciting time for the practice of rider surveys and OD studies. Transit providers and their partner organizations have never had so many data collection options at their disposal. Tablet surveys are allowing organizations to create more sophisticated survey instruments that incorporate features to monitor and control data quality. The use of passive data for OD studies is still in its infancy and is a space primed for innovation over the next decade. With all the changes to survey practice, it is important for organizations involved in OD studies to understand the various trade-offs in approaches. Although it is hard to predict what innovations to survey practice and OD data will arise over the coming years, the fundamentals of good survey design will remain constant, including the need to collect data from an unbiased and representative sample of riders in an adequate quantity to make statistically sound conclusions.