National Academies Press: OpenBook

Web-Based Survey Techniques (2006)

Chapter: Chapter Three - Current State of Practice for Web-Based Transit Surveys

« Previous: Chapter Two - Synthesis Survey Methodology
Page 7
Suggested Citation:"Chapter Three - Current State of Practice for Web-Based Transit Surveys." National Academies of Sciences, Engineering, and Medicine. 2006. Web-Based Survey Techniques. Washington, DC: The National Academies Press. doi: 10.17226/14028.
×
Page 7
Page 8
Suggested Citation:"Chapter Three - Current State of Practice for Web-Based Transit Surveys." National Academies of Sciences, Engineering, and Medicine. 2006. Web-Based Survey Techniques. Washington, DC: The National Academies Press. doi: 10.17226/14028.
×
Page 8
Page 9
Suggested Citation:"Chapter Three - Current State of Practice for Web-Based Transit Surveys." National Academies of Sciences, Engineering, and Medicine. 2006. Web-Based Survey Techniques. Washington, DC: The National Academies Press. doi: 10.17226/14028.
×
Page 9
Page 10
Suggested Citation:"Chapter Three - Current State of Practice for Web-Based Transit Surveys." National Academies of Sciences, Engineering, and Medicine. 2006. Web-Based Survey Techniques. Washington, DC: The National Academies Press. doi: 10.17226/14028.
×
Page 10
Page 11
Suggested Citation:"Chapter Three - Current State of Practice for Web-Based Transit Surveys." National Academies of Sciences, Engineering, and Medicine. 2006. Web-Based Survey Techniques. Washington, DC: The National Academies Press. doi: 10.17226/14028.
×
Page 11
Page 12
Suggested Citation:"Chapter Three - Current State of Practice for Web-Based Transit Surveys." National Academies of Sciences, Engineering, and Medicine. 2006. Web-Based Survey Techniques. Washington, DC: The National Academies Press. doi: 10.17226/14028.
×
Page 12
Page 13
Suggested Citation:"Chapter Three - Current State of Practice for Web-Based Transit Surveys." National Academies of Sciences, Engineering, and Medicine. 2006. Web-Based Survey Techniques. Washington, DC: The National Academies Press. doi: 10.17226/14028.
×
Page 13
Page 14
Suggested Citation:"Chapter Three - Current State of Practice for Web-Based Transit Surveys." National Academies of Sciences, Engineering, and Medicine. 2006. Web-Based Survey Techniques. Washington, DC: The National Academies Press. doi: 10.17226/14028.
×
Page 14
Page 15
Suggested Citation:"Chapter Three - Current State of Practice for Web-Based Transit Surveys." National Academies of Sciences, Engineering, and Medicine. 2006. Web-Based Survey Techniques. Washington, DC: The National Academies Press. doi: 10.17226/14028.
×
Page 15
Page 16
Suggested Citation:"Chapter Three - Current State of Practice for Web-Based Transit Surveys." National Academies of Sciences, Engineering, and Medicine. 2006. Web-Based Survey Techniques. Washington, DC: The National Academies Press. doi: 10.17226/14028.
×
Page 16
Page 17
Suggested Citation:"Chapter Three - Current State of Practice for Web-Based Transit Surveys." National Academies of Sciences, Engineering, and Medicine. 2006. Web-Based Survey Techniques. Washington, DC: The National Academies Press. doi: 10.17226/14028.
×
Page 17
Page 18
Suggested Citation:"Chapter Three - Current State of Practice for Web-Based Transit Surveys." National Academies of Sciences, Engineering, and Medicine. 2006. Web-Based Survey Techniques. Washington, DC: The National Academies Press. doi: 10.17226/14028.
×
Page 18
Page 19
Suggested Citation:"Chapter Three - Current State of Practice for Web-Based Transit Surveys." National Academies of Sciences, Engineering, and Medicine. 2006. Web-Based Survey Techniques. Washington, DC: The National Academies Press. doi: 10.17226/14028.
×
Page 19

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

7This chapter details what was learned from, and discusses the results of, the web-based survey done for this synthe- sis study. The survey was completed by 36 transit profes- sionals (25 transit agency employees and 11 other transit researchers). It is worth noting that this was a survey about surveys, or a meta-survey, which aimed to understand how research is being conducted by transit researchers. The sur- vey has a relatively small sample and, as mentioned in chapter two, used convenience sampling owing to the rel- atively small number of researchers in the field and to the limited scope of the synthesis project. As also mentioned in chapter two, this survey likely has some nonresponse bias that might overstate the amount of web-based survey research currently occurring in transit. Even with these caveats, the survey provides a basis for understanding what is occurring in the transit industry with regard to transit web-based surveys. This chapter describes what is and what is not happening with web-based research and ana- lyzes the reasons for the current state of practice of transit agencies and other transit researchers regarding web-based survey research. The topics covered in this chapter include: • Current use of web-based surveys in the transit industry; • Frequency, types, and areas of usage of transit surveys currently being conducted (web-based and not web- based); • Areas where web-based survey techniques are most effective for the five types of surveys explored in this synthesis (origin–destination, customer satisfaction, mode choice, planning, and other); and • Methods being employed for web-based surveys, including advice and concerns. CURRENT USE OF WEB-BASED SURVEYS IN TRANSIT INDUSTRY The use of web-based surveys is limited in the transit indus- try, although there are a significant number of respondents from transportation agencies and other transit professionals using this method, which could be somewhat overstated as a result of the previously mentioned limitations of the sample. Those using the web in some form comprised 39% (14 of 36 completed questionnaires) of our survey sample. Agencies that responded to the survey varied in size: 40% large (more than one billion passenger miles annually), 30% medium (between 75 million and one billion passenger miles annu- ally), and 30% small (fewer than 75 million passenger miles annually). Responding agencies currently using web-based surveys are distributed relatively evenly by size, indicating that not just large transit agencies are conducting such sur- veys. Table 2 shows the breakdown of respondents to the synthesis survey by region and agency size and by whether or not they currently use web-based surveys. For the most part, web-based survey use appears to be specialized for many agencies as a result of coverage con- cerns and because agencies are moving into the technology slowly (e.g., using the web as a tool on small, specialized studies before using it on major research surveys such as origin–destination studies). However, there are some excep- tions where transit agencies are using web-based survey research on significant studies. In general, trends indicate movement toward the increased use of web-based methods, with more than 70% of those currently not using the web noting that they are “somewhat likely” (44%) to “very likely” (28%) to begin using web-based surveys within two years. The primary reason provided in the synthesis survey in support of the use of web-based surveys is the efficiency with respect to time and money. Seventy percent of respondents currently using web-based surveys made favorable comments about the technology, citing its effectiveness and efficiency in being able to reach certain target populations. Respondents stated that they believe that web-based surveys increase response rates because they are convenient and “provide an option for those who wish to use it [to] reach a certain group of people.” Those using web-based survey technology also appreciate the “ability to present complicated subject matter, question design, and graphics.” They value the opportunity for “fast turn-around and cost-effectiveness.” Respondents recognize that the cost savings derived from conducting web- based surveys stems not only from the efficient manner of data collection that does not require significant on-the-street fieldwork, but also because data are brought in consistently and easily with real-time data validation providing a clean data set more quickly than other survey methods. One partic- ipant stated, “Respondents tend to answer more questions and work at it longer,” which further improves data quality. When asked about possible disadvantages to using the tech- nology, nearly all respondents currently using web-based sur- veys cited their concern over a coverage bias resulting from limited Internet penetration in the target population. (One respondent asserted that “it [web-based surveys] can only be used as an optional response mechanism because of limited penetration.”) Survey respondents worry that they may not be CHAPTER THREE CURRENT STATE OF PRACTICE FOR WEB-BASED TRANSIT SURVEYS

reaching a reliable cross section of their target audience and as such they may not be able to discern what portions of their tar- get market may be missing. “Web-based surveys will not reach less-literate people or people without computers. If that is your primary ridership, then web-based surveys may not capture the attitudes or behavior of these customers.” Over- or underrepresentation of various population seg- ments raises problems when presenting valid research on behalf of transit systems, making the results “difficult to gen- eralize to the public.” Other concerns included technical problems limiting each respondent to completing only one survey (see chapter four) and the need to limit the focus of the survey research to only certain topic areas. Concerns about costs of web-based surveys, at least in terms of time, were also expressed: “However long you think it will take to implement the survey, double it!” Words of advice given by respondents conducting web- based surveys directed to those considering starting the use of them were twofold: • “Consider the target market segment and assess Inter- net availability among those people.” • “Make sure to incorporate it with other methods to get a greater response.” A discussion of multi-method administration follows later in this chapter and is also discussed later in this report (chapter four). FREQUENCY, TYPES, AND AREAS OF USAGE OF TRANSIT SURVEYS CURRENTLY BEING CONDUCTED When asked about the types of surveys their organizations conduct and how often, respondents indicated that they do 8 survey research in the following proportions, with some sur- veys conducted more than once each year: 25% customer satisfaction, 25% planning, 19% origin–destination, 13% mode choice, and 17% “other.” “Other” types of surveys noted were household travel surveys, transit onboard sur- veys, interactive map studies, policy and issue analyses, marketing, market share, station evaluation, new offers and programs related to fares or fare cards, safety and security issues, product tests, new technology, copy testing, and employers/employees. Tabulations for all survey questions are in Appendix C. In the synthesis survey, individuals were asked to describe the number of different surveys they con- duct and, as noted earlier, 39% of respondents described sur- veys that had a web-based component. The uses for which these various surveys were conducted are shown in Figure 1. Customer satisfaction surveys show a high percentage of many different purposes, indicating that such surveys often do the work of multiple surveys (such as origin–destination surveys) at once by obtaining trip and other information beyond just satisfaction data. Recruitment question results showed that for every type of survey researchers usually recruit respondents using a combination of methods (see Figure 2). For their most recent origin–destination surveys, more than 60% of researchers surveyed reported recruiting in person, by means of inter- cepts, or on board and/or at stations; 25% recruited on road- ways or at toll plazas; another 25% recruited using the telephone; and just 6% indicated recruiting by e-mail or with a web link. For their most recent customer satisfaction survey, three- quarters of respondents reported recruiting in person and by means of intercepts on board and/or at stations, with 50% com- bining that with a telephone recruit. Again, only 6% are adding an e-mail/web link recruit method to the other two methods. Total Number by Size Currently Using Web-Based Surveys (%) Not Currently Using Web- Based Surveys (%) Total (%) 11 27 73 100 8 37 63 100 8 37 63 100 Agency Size Large Medium Small Total Agency Respondents 27 10 20 80 100 3 33 67 100 7 15 85 100 7 71 29 100 Region Northeast/Mid-Atlantic Southeast Midwest West Total Agency Respondents 27 TABLE 2 WEB-BASED SURVEY USE BY AGENCY SIZE AND REGION (includes MPO respondents)

9Mode choice surveys had the highest percentage of researchers who indicated recruiting using e-mail and/or a web link (36%), with half of them using the e-mail/web link exclusively to recruit. Thirty percent of the mode choice sur- veys described by respondents were conducted over the web or had some component being conducted over the web. Two-thirds of planning surveys used in-person recruiting by means of intercepts on board and/or at stations; 42% of respon- dents also combined this method with a telephone recruit; another 13% used e-mail and/or a web link. “Other” surveys were divided fairly evenly, with approximately 40% in person and somewhat more than 40% by telephone recruitment. Sampling methods described by respondents varied primarily based on the type of survey being conducted (see Figure 3). Random sampling was used most often as the sam- pling method for all survey types; however, “total popula- tion” sampling, where all respondents in the sampling frame were given a survey, was also used between 10% and 30% of the time depending on the survey type. Methods used for weighting of the data set varied by the type of study (see Figure 4). Origin–destination and customer satisfaction studies were most often weighted by ridership figures; 47% and 35%, respectively. “Other” weight- ing schemes mentioned included, “at the Day-Time-Route 0 20 40 60 80 100 Define traveler markets by geography Determine distribution of station/stops used Determine trip frequency Generate demographic profile of travelers Determine trip purpose Determine distribution time-of-day facilities/system used Update origin-destination trip tables Percent Origin-destination Customer satisfaction Mode choice 0% 20% 40% 60% 80% 100% Origin-Destination Customer Satisfaction Mode Choice Planning Other In person, at stations, on board vehicles, at roadways/toll plazas Telephone E-mail and/or web link Mail FIGURE 1 How research from origin–destination, customer satisfaction, and mode choice surveys are used (multiple responses allowed for this question; therefore, percentages for each purpose may be greater than 100%). FIGURE 2 Recruitment methods.

level—each survey was weighted and expanded based on the day of the week (i.e., weekday or weekend), time-of-day (a.m. peak, mid-day, p.m. peak, and evening) and route” as well as weighting “based upon the size (number of employ- ees) of the employer.” As seen here, many studies were not weighted at all, which can be a valid approach if the popula- tion is well represented and general behaviors are under consideration rather than specific representation of certain population characteristics. Respondents were asked to evaluate the success of the surveys they are currently conducting, and 88% to 94% 10 believed that they were either “very successful” or “successful.” However, only 25% of those conducting origin–destination surveys believed that they were “very successful,” whereas cus- tomer satisfaction, mode choice, planning, and other types of surveys received approximately 45% “very successful” responses (see Table 3). Use of transit-related research is often unique to a study; however, 40% of all respondents noted that they present research results to their own internal clients or management (see Figure 5). Overall, 15% of results are presented to the general public, with customer satisfaction 0% 20% 40% 60% Origin-Destination Customer Satisfaction Mode Choice Planning Other Total population Random Systematic Convenience Other 0% 20% 40% 60% 80% Origin-Destination Customer Satisfaction Mode Choice Planning Other Ridership/traffic Demographics Other factors Did not weight FIGURE 3 Sampling methods. FIGURE 4 Weighting methods.

11 results highest on this type of presentation at 25%, fol- lowed by 16% of planning studies and 16% of origin– destination studies. Just under one-quarter of survey respondents reported that their organization conducts panel surveys, and of those who do, one-half are conducting longitudinal panel surveys. Pan- els are defined as studies that draw from an already collected sample source, which is called a panel. To conduct the study, the researcher samples the panel to obtain their responses to the research questions. A longitudinal panel is when the same people are surveyed over time about the same topic to see how they or their environment are changing, often using the same set of questions. For example, to track transit customer satis- faction, a researcher might track the same riders over time to see how their customer satisfaction is changing, either as a result of changes in the transit service (e.g., better on-time performance or higher fares) or to personal changes (e.g., a job change that caused a route change for the respondent). Conducting panel studies can be effective and efficient for transit research because many transit agencies have contact information for their riders and therefore have the ability to conduct repeated research using that same sample popula- tion. Recontacting the same group of people in a longitudi- nal study allows a researcher to measure improvements over time and identify areas of concern that continue to need attention (1). Longitudinal panel studies allow more robust statistics, can better determine changes in behaviors, and can detect behavior trends over time because the research ana- lyzes responses to the same questions from the same respon- dents at a different point in time (2). Two case studies of web-based panel surveys used by transit providers are presented in chapter six of this report. Web-based panel surveys make conducting panel studies easier, because once contact is made and a respondent has become part of the panel it is very efficient to recontact them using automated e-mail routines (2). It should also be noted that panel surveys have complex issues such as attrition of the sample (respondents who drop out over time) and the need to replenish the panel to ensure that new riders are continually added to the panel data set. Very Successful Successful Neither Unsuccessful Survey Type Count Percent Count Percent Count Percent Count Percent Origin–destination 4 25 11 69 1 6 Customer satisfaction 8 47 7 41 2 12 Mode choice 10 48 9 43 2 10 Planning 11 46 11 46 2 8 Other 7 41 9 53 1 6 0 20 40 60 80 100 Other Customers Constituents External clients General public Internal clients/management Percent Origin-destination Customer satisfaction Mode choice Planning Other TABLE 3 SUCCESS RATING BY SURVEY TYPE FIGURE 5 Where research results are presented.

Without replenishment of the sample to include new riders, panel members over time would reflect only long-time users of the system. Therefore, every time the study goes out to survey, a sample of new or relatively new riders should be obtained so that the longitudinal panel reflects the ridership tenure for the transit system. Attrition issues also need to be watched closely and addressed by longitudinal researchers, as respondents who drop out of a study may be different from those who remain in the study (a form of nonresponse bias). Therefore, it is important to ensure that any respondent attrition sample is replaced with others of similar characteristics (3). This addi- tional replacement sample for the panel is typically con- ducted along with replenishment. Owing to the issues of replenishment and attrition, miss- ing values in the data set are common for longitudinal stud- ies. Analyzing longitudinal data with missing values can be statistically complex (3). However, if enough of a sample is collected and an analysis of attrition does not show significant bias in attrition (e.g., attrition is found to be mostly random and not the result of a systematic effect) then it is possible to analyze the data with those records that are complete (2). The additional effort of conducting longitudinal panels, although significant at times, allows the transit researcher to gain sig- nificant insight and robustness for their study in comparison with cross-sectional studies (3). Furthermore, over time these studies may be less costly, because most of the sample work has already occurred and the survey instruments and analysis routines are already in place, providing researchers with the potential to have a more robust study with lower costs than if they were to conduct the study using more typical repeated cross-sectional sampling techniques. Cross-sectional studies are defined as sampling a cross section of the population at a given time. Often, repeated cross-sectional sampling of customers is undertaken, where the same survey is used with a new cross-sectional sample each time (2). This method is much more common than lon- gitudinal panels, with 63% of synthesis respondents indicat- ing that they conduct repeated cross-sectional studies. Although differences in satisfaction scores are detected using repeated cross-sectional studies, the measurement of the dif- ference may be confounded owing to differences within the sample itself, because of demographic differences or some other nonquantifiable difference between individuals. Cross- sectional studies require a larger sample than longitudinal studies to measure changes over time. AREAS WHERE WEB-BASED SURVEY TECHNIQUES ARE MOST EFFECTIVE Web-Based Technology’s Effect on Survey Design Reasons cited for using web-based surveys in the synthesis survey were “the ability to present complicated subject 12 matter, question design, and graphics.” The experience of the synthesis team shows that web-based surveys can be useful in different ways depending on the type of study they will support: origin–destination, customer satisfaction, mode choice, planning, or other. These various types of sur- veys will be discussed in the following subsections and spe- cific examples will be cited from project experience to underscore the ways in which using web-based technology can benefit survey design. Origin–Destination Surveys Origin–destination surveys can be well served by web- based technology, because when respondents are asked to describe their locations they can be instantly geocoded online to a latitude and longitude, making for substantial cost savings compared with other survey methods (e.g., Resource Systems Group: New York MTA Bridges & Tun- nels Origin–Destination Study 2004; NY State Thruway Authority Westchester, Rockland, & Orange County Travel Study 2003; and Florida’s Turnpike Origin–Destination Study 2003). Two-thirds of respondents to this synthesis survey who are currently using web-based technology men- tioned that they have collected geographical data by means of the web, coded by latitude and longitude, and the other one-third noted that they have collected data coded by zip code. Online geocoding is a very difficult technical aspect of web-based surveys and is discussed further in chapters four and five. Clean geocoded data can be used in geo- graphic information system software to analyze and present information that is often very important to transit research, such as the commuter shed of a station or the number of ori- gins on the system within each zone. In recent years, geocoding survey and analysis tools have been used successfully in several major transit mar- kets on a variety of projects. An example is the Metropol- itan Transportation Authority–New York City Transit’s (MTA NYC) JFK Airport–Lower Manhattan 2005 study in which survey respondents were asked to provide origin and destination information using one of three search methods in the geographic information system component of the survey: by selecting a location on a map, by entering a spe- cific address, or by entering a nearby intersection. By clicking on the mapping option, the respondent is shown a map of the local area and simply clicks on the area of his or her location to indicate where the trip began or ended (see Figure 6). The map zooms in one or two times, enabling the respondent to select an exact location that is instantly assigned a latitude and longitude in the project database. This option enables the respondent to indicate the location relatively easily and allows researchers to screen the response (i.e., the geocode must reside within the study area or the respondent will be screened out of the survey). The system automatically geocodes the location in real time, thereby avoiding the need to geocode later,

13 FIGURE 6 Screen shot of geocoding technique on the JFK Airport–Lower Manhattan 2005 study. which is frequently based on erroneous word descriptions of geographical data. Another way origin–destination surveys can be enhanced for data validation is by using web-based technology to show maps of transit systems and linking them directly with the schedules of specific lines and stations. An example of this can be seen in a series of screen shots captured from NJ TRANSIT’s 2003 Rail ePanel study. In this survey the respondent was first asked which commuter rail line they used (see Figure 7). Once a specific line was selected (color coded to match NJ TRANSIT’s schedules), the respondent was directed to a page showing only the stations on that rail line (see Figure 8). Each rail line’s train schedule has been processed into a database with exact times and stations, for weekdays and weekends, for the entire system. When the respondent chose his particular station, the schedule data that was linked to the survey offered the respondent only actual train times and train numbers available (Figure 9). Offering correct available train times and numbers is one example of how web-based technology can help improve data quality and, in this case, decrease item nonresponse in surveys. A discussion of item nonresponse follows in chapter four: Item Nonresponse in Web-Based Surveys. Problems resulting from guessing and/or faulty memory on the part of the respondent are therefore mitigated, resulting in clean data for the planners at NJ TRANSIT. Mode Choice Surveys A mode choice study can be difficult to do using paper-based survey methods, particularly for stated preference surveys. Mode choice can be evaluated much more efficiently using computer-based technology because customized branching can obtain a clearer picture of each distinct respondent’s choices based on his mode path; and then realistic alternative scenarios can be constructed to understand the respondent’s behaviors to variables such as time, cost, and comfort. As will be discussed in chapter four, offering the survey by means of the Internet can increase response rates over the survey offered only to those respondents who can be recruited in person. One respondent stated that a web-based survey can be “an easy tool for the end user and our staff to gather data on work trips for employees at large employers in the county.” Planning and Other Surveys Respondents indicated that for planning surveys, web- based surveys are beneficial “as a way to gather public input on our planning studies, in addition to holding pub- lic meetings which are usually poorly attended.” The abil- ity to quickly and easily reach out to the public, provided agencies have a satisfactory list of e-mails and/or a well- publicized website, is another benefit to using web-based surveys.

14 FIGURE 8 Screen shot to select boarding station of chosen rail line from NJ TRANSIT’s Rail ePanel survey. A variety of uses for “other” surveys were also noted in the synthesis survey. One agency researcher described an interactive map study that had been conducted where they needed to “solicit customer feedback on their experiences with the interactive map” on their trip planning section of their website. Using a web-based survey, they were able to “determine if there are any fatal flaws that need immediate attention.” This particular study is detailed as a case study in chapter six. Another “other” type of survey mentioned by a respondent was a household travel survey, and this type of survey can benefit greatly by being conducted online. First, respondents have a difficult time remembering all of their daily trips for an assigned travel day. With web-based tech- nology, respondents can be prompted to include all trips by simplifying data entry. Depending on what the respondent describes for activities and/or purposes, they can be shown FIGURE 7 Screen shot requesting commuter rail line from NJ TRANSIT’s Rail ePanel survey.

15 FIGURE 9 Schedule page from NJ TRANSIT’s Rail ePanel survey. FIGURE 10 MI Travel Counts (Michigan DOT) activity input page. customized screens and drop-down boxes on those screens. For example, if a respondent starts out a trip from home to work by walking, he or she can be shown a drop-down box with a variety of choices for the second mode on their trip to work (see Figure 10). The respondent can be prompted to enter all trips for the survey day, and can be shown various trip purposes for each trip in drop-down boxes. In the example shown here, the respondent went to work at his construction site, then went out for lunch with five friends. Each trip requires a start and an end

time, and these times are validated such that no trip can have overlapping times with any other trip (see Figure 11). The data set for a study such as this will be clean and validated, saving on agency costs for data entry and data cleaning. As a respondent enters each new trip made throughout the course of the day, the details of the validated trips can be shown at the bottom of the page (see Figure 12). This makes for an easy reference, and the respondent can easily check to confirm that every trip made has been entered and that all the times are correct. Again, this technology ensures complete and validated data. As with many transit and transportation surveys, house- hold travel diaries require the geocoding of trip start and end locations and web-based technology can provide major ben- efits in this aspect of the survey: • As mentioned previously, online surveys can offer a respondent several ways to input an address: by enter- ing the specific address, by entering a nearby intersec- tion, or by offering an interactive map to search. • Online surveys can “remember” all addresses input by a respondent and easily offer that address, if it comes up again, which then only needs to be checked, not rewritten. METHODS BEING EMPLOYED FOR WEB-BASED SURVEYS Survey Design, Hosting, and Invitations Approximately half of respondents using web-based surveys have contracted with an outside consulting or web develop- ment firm to design and host their surveys (see Figure 13). SurveyMonkey and SurveyTracker were the two software applications identified by those who used an online survey development tool to create their surveys. These were the applications identified during the survey process and are not an endorsement of specific products and services. Web-based survey invitations are frequently sent by e-mail to potential respondents. However, one serious and frequent downside to e-mail is the tendency for mass e-mail to be rerouted by spam filters meant to capture unsolicited junk e-mail. Several solutions to the problem do exist, including these two cited by survey respondents: using e-mail lists con- taining existing customers from transit agencies and/or using third-party bulk-e-mailer reputation monitoring tools. Third- party monitoring tools will automatically notify a sender if they have been placed on a filtered list and are not having e-mail delivered at the Internet service provider (ISP) level. In using lists provided by transit agencies, the sender would 16 FIGURE 11 MI Travel Counts (Michigan DOT) trip details page.

17 FIGURE 12 MI Travel Counts (Michigan DOT) trip rostering page. 55% 22% 33% 45% 18% 27% 0 20 40 60 80 100 Hosted in house on organization's computers Hosted by a survey provider Hosted by consulting or web development firm Designed in house using web page layout software Designed with online survey development tool Contracted out to consulting or web development firm Percent FIGURE 13 Survey design methods.

18 Administration Type Percent Online web survey 36 Online web survey, paper 29 Online web survey, telephone, paper 21 Online web survey, telephone, paper, computer-based, personal interview 14 Total 100 TABLE 4 WEB-BASED SURVEY ADMINISTRATION COMBINATIONS hope that the e-mail recipients recognize the subject and con- tent of the e-mail and have expressly permitted mail regard- ing that agency. Other tools to increase e-mail delivery include hosted e-mail solutions, in which a third party sends the e-mail; sender authentication; and software tools to iden- tify words, phrases, and common e-mail structures that often trigger spam filters. These solutions are discussed further in chapter four. Three-quarters of respondents using e-mail invitations also noted that they send e-mail reminders to those who do not respond within a certain time frame and indicated pro- viding an average of two reminders. Other means to remind respondents were by telephone or mail. Survey Administration As mentioned previously, 39% of researchers described one of their most recent surveys as a web-based survey for this synthesis. Of those web-based surveys, approximately one- third were exclusively web-based, with two-thirds using a “multi-method” administration, combining the web-based portion with either paper, telephone, or a personal interview (see Table 4). Reasons given for doing a multi-method survey included reaching a larger sample “to cover all target audiences,” max- imizing response rate by making the survey more easily avail- able by “. . . giving people who are in a hurry an alternative to taking time on the spot,” and getting more in-depth details following a broad survey “. . . later in the year a telephone survey is done with a smaller sample and fewer questions.” One researcher noted, “We use the online survey because it is so easy to disseminate and no data entry is required. We use paper because some employers . . . have large populations of employees without access to computers.” To facilitate the response to web-based surveys, researchers reported that they provided several means of support for respondents including a toll-free contact number for questions, e-mail support, a link on the survey website to frequently asked questions, and/or links with context-specific help on the web page. Researchers also pointed out that they are not using web- based surveys across the board, but are using such surveys for smaller, more focused studies. “The online web survey was a different type of planning survey. It was focused on planning for a new regional transit ticket. The paper survey is our basic planning survey.” One respondent noted that they were conducting web-based surveys on a limited basis “as they relate to marketing promotions,” whereas another cited such a web-based survey “to university and college students.” Signs indicating increased web-based survey use coming in the near future are linked to increased access to e-mail and the Internet. One respondent noted that, “We are waiting for our customer base of smart card users to grow . . . and give an e-mail address. Then we will have the opportunity to e-mail them a survey, but we need to create the questionnaire online . . .” Data Quality and Validation in Web-Based Surveys In addition to being more convenient for many respondents to access, researchers appreciate web-based surveys for their high-quality data with online validation, consistency, and geocoding. Researchers also believe that a technical benefit of web-based surveys is the ability to link between tables in databases to prevent incorrect entries, as in the NJ TRANSIT example connecting train numbers and train times. Another technical benefit mentioned was, “The ability to ask ques- tions and evaluate concepts that may be too complicated to present on the phone.” On the individual response level, web- based technology yields “superior data quality” and allows collection of “customer comments that are more unbiased than from other survey methods.” Moreover, researchers felt “respondents give more honest answers” because of the anonymity of completing surveys over the Internet and that they obtain “more complete answers” to questions because of the ease of entering comments and not being rushed in their response. In sum, those using web-based surveys are generally satisfied with the quality of their resulting data sets (see Figure 14). Concerns About Web-Based Survey Use As discussed before, sample bias is the primary concern of both those currently using and those currently not using web- based surveys. Two-thirds of respondents not currently using web-based surveys mentioned that they are concerned with their inability to completely reach their target market and with the resulting sample bias owing to a lack of Internet access by transit users; “[we] are skeptical about assuming [web-based] results will reflect our riders.” Respondents also expressed concern that the sample for web-based surveys might be viewed as “self-selected” and

19 worried about their inability to guarantee “one survey com- plete per person.” One researcher noted that their “organi- zation feels that web-based response will bias the results because of differences in demographic characteristics of those with and without Internet access.” The next most important concern, given by one-quarter of respondents, was the lack of in-house expertise in web-based survey technology or inadequate funds to enable them to develop their capabilities in the area. Several researchers expressed the reality that their organizations “are slow to change” as far as their current ways of conducting research. Approxi- mately 10% of respondents said that they had no need for conducting web-based surveys because they are looking only for on-board users of their transit systems and are therefore able to conduct their surveys on board their trains, buses, etc. A few respondents reported that they were just beginning to try out this method or would be in the next few years. Despite these concerns, 70% of those not currently conducting web-based surveys said that they are likely to begin within the next two years. 0% 20% 40% 60% 80% 100% O rig in - D es tin at io n Cu st om er Sa tis fa ct io n M od e Ch oi ce Pl an ni ng O th er O rig in - D es tin at io n Cu st om er Sa tis fa ct io n M od e Ch oi ce Pl an ni ng O th er Survey dataset was clean Respondents completed every question Agree Neutral Disagree FIGURE 14 Data set results from web-based surveys.

Next: Chapter Four - Web-Based Survey Methodologies and Successful Practices »
Web-Based Survey Techniques Get This Book
×
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

TRB's Transit Cooperative Research Program (TCRP) Synthesis 69: Web-Based Survey Techniques explores the current state of the practice for web-based surveys. The report examines successful practice, reviews the technologies necessary to conduct web-based surveys, and includes several case studies and profiles of transit agency use of web-based surveys. The report also focuses on the strengths and limitations of all survey methods.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!