Click for next page ( 35

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement

Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 34
34 CHAPTER FIVE TECHNOLOGY This chapter explains the basic technologies and technology Using a survey service means that researchers must direct issues researchers must address when implementing a web- the survey creation themselves. This includes creating all based survey instrument. questions, selecting how to display the questions, and deter- mining all the logic and validation rules. Researchers are also Transit researchers typically use one of the following directly responsible for all respondent recruiting and cus- three methods to implement and conduct web-based surveys: tomer support. There is likely no direct help for the researcher using such services; therefore, any questions or 1. Use a survey service and/or software--This method is problems are usually answered by e-mail on the time frame inexpensive, but only allows predetermined question of the service provider (not necessarily the time frame of the types and is therefore very limited in its flexibility. It survey researcher). is a viable choice for creating simple surveys, but ser- vice from the provider is minimal and one must be For many transit researchers, survey services are a very aware of hidden costs. good solution to develop a survey at low cost and to learn 2. Create and conduct the entire survey in-house using first hand about web-based surveys and how the process information technology skills and resources within the works. However, researchers often find that online services researcher's agency or organization--This requires and generic survey software do not meet their needs. For technical expertise, but gives the researcher complete example, longitudinal surveys cannot be created that track control. Depending on the skills of the researcher and one respondent over time using such tools. Nor can stated the resources available, this option can range from preference surveys for mode choice studies be produced being inexpensive to very costly. effectively using less expensive online survey services, 3. Hire a consultant--This can be expensive, but pro- although there is much more expensive software that does vides experience, expertise, and the ability to conduct allow for advanced online mode choice surveys to be created. complex and highly customized questionnaires. Other Features such as online geocoding and linking transit sched- aspects of a survey, such as sampling plans and non- ules are typically not incorporated into these surveys. web survey instruments, recruiting, and reporting may Advanced validation cannot be accomplished, as these tools also be addressed by a consultant. are not capable of, for example, comparing a zip code with a data table of zip codes to confirm if a respondent's answer is Responses to the synthesis survey indicated that most an existing zip code or not. researchers contract out their web-based survey work to consultants, but that online survey tools and in-house devel- In the case of more advanced needs and sophisticated sur- opment are often used as well (Figure 19). Researchers veys, a consultant is often hired to conduct the survey unless using a consultant for survey development also had those there is significant expertise in-house, although as men- surveys hosted by the consultant. The same is true for those tioned earlier there is software available that allows transit who developed surveys using online tools. Researchers researchers to do more of this themselves. Although some electing to develop the survey in-house also hosted the sur- advanced software is very good, it does not mean that the vey in-house. process is simple; therefore, a consultant is hired to help implement or guide a study using third-party software. All three options might be used by the same transit Table 8 lists some of the strengths and limitations found researcher or agency for different reasons, depending on the with different survey providers. needs of the study. For example, a survey-service question- naire from a provider could easily be created to supplement The following list addresses various technologies that are a simple paper-based instrument. For the annual cost of important to consider when conducting a web-based survey. approximately $200 (plus other charges), a transit researcher can use such a provider to create simple web-based ques- The web-based technology itself--is it server driven? tionnaires that are analogous to their paper questionnaires. Server-driven technology for web-based surveys is They can then invite respondents to the survey by means of important, because it means that most of the logic and an e-mail list or a web link. technology reside on the survey provider's server as

OCR for page 34
35 Contracted out to consulting or web 55% development firm Designed with online survey 27% development tool Designed in-house using web page 18% layout software 0 20 40 60 80 100 Percent FIGURE 19 Web-based survey development methods. TABLE 8 ADVANTAGES AND DISADVANTAGES OF VARIOUS WEB-BASED SURVEY DEVELOPMENT METHODS Approach Strengths Limitations Contract out to consulting Provides experience and expertise Expensive or web development firm Not many available consultants with high degree Can develop complex, highly customized of expertise questionnaires Ability to display any desired graphics/logos on survey pages Allows recruit by any method Can develop sampling plans Can develop non-web survey instruments to accompany web-based survey Can develop complex (e.g., multi- dimensional) weighting schemes as part of the study May provide advanced data validation; i.e., trip lengths in time and distance May provide online geocoding Provides advanced longitudinal survey capabilities, such as presenting respondents with the option to simply confirm previous answers from previous waves of a longitudinal survey May provide ability to search open-end answers in database in real time May provide ability to link transit schedules and maps of systems Hosts survey on its own servers Can develop password scheme to screen out multiple answers per respondent Can see results live Provides clean, validated dataset Will provide data in format require by agency (continued on next page)

OCR for page 34
36 TABLE 8 ADVANTAGES AND DISADVANTAGES OF VARIOUS WEB-BASED SURVEY DEVELOPMENT METHODS (continued) Use online survey Inexpensive, although costs can be hidden Requires predetermined development tool question types Limited flexibility in questionnaires by templates *Some examples include: Easy to use for simple questionnaires used to create them Limited ability to display SurveyMonkey graphics/logos on survey Zoomerang Allows recruit by e-mail or web link pages Infopoll Agency can "get their feet wet" and try out May require software SurveyCrafter new technology downloads SuperSurvey Can see results live Must program survey oneself Perseus May provide managed list of opt-in survey Must devise own data respondents validation No need to involve information technology Must create own logic (IT) department scheme Must create own survey Hosts survey on its own servers screens Cannot create stated preference surveys Cannot longitudinally track respondents over time Cannot provide advanced data validation Develop/program/host in- Dependent on in-house expertise Dependent on in-house house expertise Significant work to produce Complete control complex surveys Can develop complex, highly customized May require a high learning questionnaires curve Requires time and effort of Can integrate directly to other in-house IT department, which is often systems (such as agency website, database, already busy with many other etc.) priorities Can develop sampling plans Can develop non-web survey instruments to accompany web-based survey Can develop complex (e.g. multi- dimensional) weighting schemes as part of the study May provide ability to search open-end answers in database in real time Hosts survey on its own servers Can develop password scheme to screen out multiple answers per respondent Can see results live Provides clean, validated dataset Will provide data in format require by agency *These include those known to the research team at the time the research was conducted. Any omissions were inadvertent. TCRP does not endorse specific products and services. opposed to requiring this technology to reside on the This is often very important for transit studies, as there respondent's computer through a late-version browser are frequently proponents and opponents to many of the with sophisticated technologies. The objective is for the projects that a transit agency is attempting to imple- survey provider to be responsible for as much technol- ment. A web-based survey that allows the same user to ogy as possible on their end, thereby enabling older repeat the survey again and again can create a serious computers and software to still easily run the survey and bias in the survey. This problem is easily taken care of to benefit from its advanced features. if each respondent is provided a single, unique pass- Recruiting--are you able to recruit using e-mail? Can word that allows him or her to take the survey only you use your own e-mail system? Will you be sending once. reminders, or merging fields into the e-mail? Can the survey be made multi-method? As discussed in Security--does the survey use an authentication system chapter four, multi-method surveys are often an excel- so that each respondent can take the survey only once? lent way to conduct a transit survey. Can the web-based

OCR for page 34
37 survey be used on stand-alone computers without Inter- Repeatability--once a transit agency has invested in net access in a field environment for intercepting developing a survey can it be reused or repeated? respondents? Technical support--will respondents be able to call or Hosting--it is almost always easiest to host the data e-mail with questions as they take the survey? with the same provider as the entity developing and Data validation--does the software allow for error implementing the survey. Although it is certainly pos- checking and answer inconsistencies? sible for a consultant to implement a survey somewhere else (e.g., on a transit agency's servers), this often adds Internet technology has been a hot topic over the last decade, significant costs without much gain. It is important that and often the very small details of technology are the focus the hosting entity has been through a security audit and, when discussing any web-based technologies. When imple- if necessary, has been approved by the agency's legal menting, these small details are critical, and each web applica- department. tion requires making a large number of technical decisions. Data--most survey service providers will provide data This chapter intentionally focuses on the broader considera- in export formats. It is important to ensure that tions of technology in web-based surveys. It is important for the providers give all data over to the client and that data transit researcher to understand the benefits and costs of differ- such as how many respondents started the question- ent broad technology decisions first. Transit researchers need to naire but did not finish it, the date of the survey, the understand what they want out of their research plans and this sample size, the response rate, etc., are all included in understanding will drive their decisions about which of the the data set. three types of technology methods described earlier they should Real-time data access and reporting--to what extent are be pursuing. Many researchers might pursue them all. Other up-to-date results required to check the progress and researchers might pursue them in varying degrees (e.g., a con- distribute interim results? sultant helping them using third-party software).