Click for next page ( 39

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement

Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 38
38 CHAPTER SIX CASE STUDIES This chapter details three case studies describing projects customer satisfaction can be much lower in panel surveys. conducted by NJ TRANSIT, Metrolink, and TriMet. These In addition, panels provide opportunities to directly deter- case studies show what can and are currently being done with mine the reasons for those changes. web-based research by transit agencies. Various themes described in earlier chapters are repeated and can be under- Longitudinal panels can be administered using a variety of stood in a real-world context. methods. For transportation studies, intercept recruiting is an efficient approach to assembling panels. Although telephone NJ TRANSIT RAIL ePANEL and mail-out/mail-back instruments are commonly used, web-based instruments can be a highly cost-effective alterna- The benefits of using web-based longitudinal panels for cus- tive for many applications. Web access has increased signifi- tomer satisfaction studies can be clearly seen based on the cantly across the population and it is possible to construct experience of such a study for NJ TRANSIT's rail cus- demographically representative panels from among those tomers. The study revealed numerous benefits of this method who have web access. In addition to their cost-effectiveness, over cross-sectional studies, including more robust statistics, an important advantage of using web instruments with panels better understanding of customer satisfaction, and the ability is that the time required to complete and analyze data from a to analyze customer satisfaction trends. A variety of innova- survey wave can be dramatically reduced. tive Internet technologies was implemented, adding value to the study by ensuring data quality, timeliness, reductions in NJ TRANSIT's Rail Customer Satisfaction ePanel was respondent burden, less random error in respondent answers, designed to be a continuous survey, providing monthly data and techniques to pair qualitative data to quantitative analy- on customer satisfaction. It used web-based technologies to sis. Online geocoding of respondents' origins and destina- invite respondents from one of three panels each month and tions was also described as another aspect of the survey, to administer a customer satisfaction survey. The resulting which provided NJ TRANSIT more value from the study. survey data monitor customers' concerns on a monthly and even daily basis, as study data were continually being Customer satisfaction studies are conducted by many major received from respondents throughout each month. organizations, including those who provide transportation ser- vices. Typically, customer satisfaction studies are carried out Web-based survey technology allowed for great flexibility using repeated cross-sectional sampling of customers. Satis- in obtaining both quantitative customer satisfaction responses, faction scores are compared across these repeated cross such as typical satisfaction scores, and qualitative responses, sections. Differences in satisfaction scores resulting from per- such as written answers to open-ended questions that are used ceived changes in service are measured; however, the mea- to explain why quantitative scores have changed. This was a surement is confounded in part by differences between the critical part of the study, because the reasons for change in sat- cross-sectional samples. Demographic differences can be isfaction scores can be quickly understood when they are accounted for by weighting the samples so that they are equiv- paired directly to responses from open-ended questions (see alent; however, there are significant differences in satisfaction Drill Down Questions section and Figure 24). scores between individuals that are not explained by demo- graphic or other easily measured characteristics. The result is Advanced web-based survey technologies also allowed for that relatively large samples are required to measure changes a number of innovative features that improved data integrity in customer satisfaction over time. and currency. These features included online geocoding of origin and destination data, automatic updating and querying Longitudinal panels offer a potentially attractive alterna- of train schedule data so that respondents could select only tive to repeated cross sections for measuring customer sat- valid trains in their surveys, and full validation of responses isfaction. Measuring changes in satisfaction of the same to questions. Web-based longitudinal panel survey instru- individuals from one period of time to another eliminates the ments can be designed in ways that minimize respondent confounding caused by variations between the different fatigue. This was accomplished using a number of techniques individuals in repeated cross-sectional samples. The result that required respondents only to confirm that various aspects is that the sample sizes required to measure differences in of their travel have not changed since their previous survey.

OCR for page 38
39 Background respondents to rate. It continued by asking general customer satisfaction questions (e.g., would you recommend NJ The NJ TRANSIT ePanel Customer Satisfaction Study was TRANSIT to a friend?, etc.) and also determined respon- conducted to provide continuous monthly and quarterly dents' origin and destination locations, and ended by asking tracking of NJ TRANSIT commuter rail riders' satisfaction additional background questions and demographics. along 65 satisfaction measures. These measures had previ- ously been tracked in surveys conducted less than annually, The 65 customer satisfaction ratings were crucial to deter- using cross-sectional sampling with handout/handback paper mining where NJ TRANSIT was performing well and where questionnaires. improvements would be needed on its rail system. Ratings were on a scale of 0 to 10, with the option to answer not The ePanel study measured rail customers' satisfaction applicable. Data validation was used for many questions, scores in what NJ TRANSIT calls "functional areas," which such as the customer satisfaction questions, to ensure quality included questions related to parking, boarding stations, des- data and complete responses. Wording was customized for tination stations, train scheduling, and customer service. The each respondent on many of the survey screens as well. For survey also measured "key-driver areas," which include on- example, in the screen shot below, the question asks about time performance, personal security, employee performance, "parking at Woodcliff Lake Station" instead of simply say- fares, and mechanical reliability. The study provided the ing "your boarding station" (Figure 20). Wording cus- ability to segment the customer satisfaction measures based tomization makes the questionnaire clearer for respondents on different train lines, destination markets, customer demo- and by extension improves data quality. graphics, stations, etc. NJ TRANSIT's ePanel was designed to answer the fol- OriginDestination Data Collection lowing specific questions about commuter rail customers on a continuing basis: An important part of the study for NJ TRANSIT was to obtain origin and destination data. To accomplish this, What are the trends in customer satisfaction and what respondents were asked to geolocate their origin and desti- factors influence these trends? nation addresses by using a point-and-click map, a street On which train lines within the NJ TRANSIT system is address, a business name, or an intersection search. A screen customer satisfaction changing? In what direction are shot of the map search is shown as Figure 21. Regardless of these changes, how big are the changes, and why are the type of geolocation search used (map, address, business, they occurring? or intersection), a latitude and longitude for each origin and What are customers' main concerns? Where does NJ destination was determined. These were then automatically TRANSIT need to improve? coded into the proper NJ TRANSIT transportation analysis Where are customers satisfied? What performance does zones using an online, point-in-polygon routine. Therefore, NJ TRANSIT need to maintain? NJ TRANSIT received immediate real-time access to fully coded origindestination data with transportation analysis To address these questions, a longitudinal panel study plan zones already attached to the data. was developed in July 2002 that was driven by a monthly sur- vey that began in September 2002. This survey collected cus- Another important function of the survey was determining tomer satisfaction data every month from one of three separate what train the respondent rode. Respondents were asked the customer panels, which were each comprised of approximately appropriate questions to classify them into four categories: 4,000 participants. Each panel respondent was surveyed four frequent weekday rider, frequent weekend rider, infrequent times a year at three-month intervals, giving NJ TRANSIT new weekday rider, and infrequent weekend rider. Once the respon- monthly customer satisfaction data throughout the year and dent type was known, the survey then asked the respondent allowing them to track customer satisfaction trends and cus- what train they used and then displayed only the relevant tomer origin and destination patterns. Respondents were asked trains for their station and day of week (Figure 22). to take a survey only once every quarter, reducing respondent fatigue and also giving respondents enough time between sur- Anchoring vey waves to notice service changes. The differences between a respondent's first survey and their Web-Based Survey Instrument subsequent surveys could be subtle, but important, and served three main purposes: (1) to deliver respondents more efficient The survey used a web-based, multi-paneled, multi-waved second, third, and fourth surveys by asking them only to con- customer satisfaction questionnaire that had a number of sec- firm answers from their previous surveys when the answers tions. The questionnaire first obtained background informa- are unchanged; (2) to use "anchoring" so that respondents tion about respondents' current NJ TRANSIT travel, then the knew how they rated satisfaction measures in the previous survey presented 65 customer satisfaction attributes for survey wave, which helped them make new judgments based

OCR for page 38
40 FIGURE 20 Example screen showing customer satisfaction attributes rating (NJ TRANSIT). FIGURE 21 Map search screen in NJ TRANSIT Rail ePanel survey.

OCR for page 38
41 FIGURE 22 Schedule page in NJ TRANSIT Rail ePanel survey. on their previous answers; and (3) to ask respondents "drill because the respondent had forgotten how they had previ- down" questions that requested a written explanation of rat- ously rated the service. Respondents were focused on the ing differences between the previous and current survey. change in service, reducing the random error in the mea- surement of this change. Anchoring was a technique used in the second, third, and fourth survey waves to enable respondents to see how Drill Down Questions they previously rated their customer satisfaction attributes (Figure 23). Anchoring was used to ensure that a changed "Drill downs" are open-ended questions that were asked answer was in response to a change in service, and not to determine the reasons for a respondent's change in FIGURE 23 Example screen showing "anchoring" functionality: Dotted arrows indicate rating given in previous survey wave (NJ TRANSIT).

OCR for page 38
42 satisfaction ratings. Drill downs provided the unique longi- (i.e., if the respondent did not change their answer in 10 or tudinal ability to ask respondents a qualitative question that more questions from their previous survey), then only is directly related to a changed rating score. The differences those differences that did exist for that respondent were between the 65 satisfaction scores from each respondent's shown. Once the 10 questions with the highest absolute dif- previous survey and their current survey were calculated and ferences were determined, respondents were asked why the 10 largest differences in satisfaction scores were deter- they had changed their answers to these questions using mined (differences could be both positive and negative; open-ended comment boxes (Figure 24). Again, changes therefore, absolute value was used). If there were ties, then could be either positive or negative, as NJ TRANSIT enough satisfaction questions to obtain up to 10 were wanted to understand both what is performing and what randomly selected. If there were fewer than 10 differences needs improvement. FIGURE 24 Drill down questions screen in rail ePanel Survey (NJ TRANSIT).