Customer-Focused Service Guarantees and Transparency Practices (2018)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

30 Another tool in the toolbox for transit agencies working to become more customer-focused is to be transparent about their performance. There are many ways to define and implement transpar- ency (see Chapter 2 for a detailed discussion). In this report, customer-focused transparency is defined as any open and public reporting updated at least annually that includes customer-focused metrics. A transit agency has customer-focused transparency if the following criteria are met: • Open and public: Reports are available to any member of the general public without the necessity of making a special request. • Updated at least annually: There must be 1 year or less between updates to the reports. • Reporting includes customer-focused metrics: The reported metrics must measure an aspect of the customer experience, for example: – On-time performance; – Headway adherence/reliability; – Excess wait time; – Missed, canceled, or not-operated trips; – Percent of scheduled time in operation (for automated systems); – Distance traveled between mechanical failures; – Passenger safety metrics (e.g., accidents, incidents, crimes); – Customer complaint rate; – Customer satisfaction score; and – Customer call center metrics (e.g., average time on hold). This chapter presents results of the customer-focused transparency section of the transit agency survey and is broken into the following sections: • Prevalence of customer-focused transparency. • Publication of performance, including – Methods of publishing data, – Frequency of updating and so forth, – Presence or absence of performance context (e.g., goals and baselines), and – Promotion of transparency. • Implementing customer-focused transparency. • Benefits, challenges, and lessons learned. Prevalence of Customer-Focused Transparency Of the 22 transit agencies that responded to the survey, 17 exhibited some form of customer- focused transparency. Although titles of metrics and precise definitions often varied from agency to agency, the most frequently reported metric was on-time performance (OTP, also referred to C h a p t e r 4 Customer-Focused Transparency

Customer-Focused transparency 31 as “reliability”). After OTP, accidents, customer comments, and customer satisfaction were reported by about two-thirds of transit agencies (see Figure 9). Transit agencies also listed some additional customer-focused metrics that are being reported to the public; examples include: • Cleanliness rating, • Customer’s safety rating, • Helpfulness rating, • Information rating (how well the transit agency is providing information), and • Elevator and escalator availability. Transit agencies that responded to the survey indicated how long their customer-focused transparency programs have been in place. Of 16 responses, half of the transit agencies’ programs were in place for 15 or more years. The most common answer was 17 years (i.e., four transit agencies started their programs in 2000). Figure 10 shows the distribution of transparency program durations across survey respondents. The performance metrics of a few European transit agencies appeared in the literature review. For example, Transport for London’s (TfL’s) performance metrics are almost all related to the customer’s perspective; TfL measures customer excess wait time, customer travel time reliability, customer satisfaction, and customer injuries (TfL 2017). STL’s performance metrics are all tied Seventeen of 22 surveyed transit agencies had some form of customer-focused transparency. On-time performance Accidents Customer comments (complaints / commendations) Customer satisfaction Distance between mechanical failures Crime Missed trips (service not operated) Headway adherence Call center metrics (e.g., hold times) Passenger excess wait time (or travel time) Percent up time (for automated systems) Trip denial rates (for demand-responsive) Number of Transit Agencies Reporting Pe rf or m a n ce M et ri c 0 2 4 6 8 10 12 14 16 18 Figure 9. Customer-focused metrics reported by transit agencies. D ur at io n of T ra ns pa rn ec y Pr og ra m 0 1 2 3 4 5 6 7 Less than 5 years Between 5 and 15 years Between 15 and 25 years 25 or more years Number of Transit Agencies Figure 10. Duration of customer-focused transparency programs.

32 Customer-Focused Service Guarantees and transparency practices to its quality commitment, and most of the metrics—for example, common courtesy, safety and comfort, cleanliness, and reliability—are collected from STL’s mystery shopper program (STL n.d.). (A mystery shopper program is when individuals ride the transit service posing as a typical customer but noting or scoring various aspects of the trip.) Société de Transport de Montreal’s (STM’s) performance metrics also include data collected by mystery shoppers, for example, a cordiality metric and an information metric (STM 2016). Publication of Performance For transit agency customers (and other stakeholders) to benefit from performance measures, transit agencies must publish their performance measures. There are several ways to make per- formance data available to the public—both in the type of publication medium (e.g., print or electronic) and distribution method (e.g., handouts or web pages). The issue of publishing is particularly important for customer-focused transparency. This project identified customer-focused transparency as reporting that was open and public, with the public not needing to make a special request or attend a special meeting to see the transit agency’s performance. Under this definition, documents distributed only at special-purpose public meetings (e.g., a board of directors meeting) but not available for download online or as a handout in public places are not considered customer-focused transparency. The literature review revealed significant variation in the amount of effort needed to find a transit agency’s performance reports. In some cases, transit agencies included an obvious link or display on their home pages. In other cases, readers would have to dig through several web pages or documents to locate a performance report. This section discusses how surveyed transit agencies published their customer-focused transparency metrics, including: • The method of publishing (or the medium used), • The frequency of data updating and republishing, and • Whether reports provided customers with necessary context to interpret available data. Publishing Method The 17 transit agencies with customer-focused transparency reported how they published their performance data (see Table 13). Documents available for download were the most common publishing method. As shown by the high number of transit agencies using web pages and downloadable documents, performance reporting is moving beyond only hard- copy reports. Publishing Method Number of Transit Agencies Hard copies of a report 4 A board packet or similar document for an oversight body available for download 7 A PDF or other document available for download 12 A static web page 8 An interactive web page 5 Table 13. Performance publishing methods for surveyed transit agencies.

Customer-Focused transparency 33 Frequency of Updating Performance In addition to method of publishing, transit agencies had different approaches to how often data were updated and republished. Frequent updating of data provides customers with more meaningful, timely, and actionable information. Unless performance reporting systems are completely automated, there likely is a cost in staff time and effort to update and republish data, so transit agencies must find the appropriate frequency of updating that balances the cost and burden with the timeliness of data. Although transit agencies may update their performance data on multiple intervals (e.g., a monthly report gets summarized in an annual report), respondents provided the most frequent interval that data are updated and republished (results are in Table 14). Monthly updates were the most common among survey respondents; two transit agencies actually updated their per- formance data daily. Providing Context in Performance Reporting The last variable for publishing performance is the degree to which transit agencies provide some form of quantitative context to help customers understand and interpret performance data. This context can come in several forms; for example, simply showing historical data can help a customer interpret the transit agency’s current performance. Transit agencies may also display a target or goal for the metric or provide some other form of benchmark (whether an industry standard, performance in the same period last year, etc.). Without context, customers are presented with a single number but are unable to determine whether that number reflects good, average, or poor performance. Surveyed transit agencies provided information about whether context was provided for all, some, or no metrics. Results are displayed in Table 15 and indicate that transit agencies have realized how important it is to provide context to help customers interpret performance data. Promotion of Transparency There is a significant difference between publishing performance data and promoting the publishing of performance data. A transit agency could publish data but do so in such a way that no one, except for the most avid searcher, would find the data. On the other hand, a transit Context Provided Number of Transit Agencies No context 1 Context for some metrics 8 Context for all metrics 8 Table 15. Quantitative context provided with performance reporting. Frequency of Updating Performance Number of Transit Agencies Daily 2 Monthly 11 Quarterly 3 Annually 1 Table 14. Frequency of updating and republishing performance data.

34 Customer-Focused Service Guarantees and transparency practices agency could both publish and promote its transparency to make sure that customers (and other stakeholders) are aware of the performance data and when the data are updated. There are many different communications tools that transit agencies could use to promote and publicize their customer-focused transparency. Transit agencies indicated both the tools used to promote and publicize their customer- focused transparency and which audiences (transit riders, general public, public officials, and others) were targeted by each tool. Figure 11 presents the overall use of tools. Figure 12 presents the use of tools to target specific audiences. Of the 17 responses received, 14 transit agencies used home page links or displays. A few tran- sit agencies also used hard copies, e-mail or text message distribution lists, and press releases. Social media was the least-used communications tool to promote or publicize customer-focused transparency. As expected, transit agencies’ home pages are the most commonly used tool to reach many different audiences, but traditional tools (e.g., press releases and hard copies of reports) still play an important role in publicizing customer-focused transparency—particularly to public officials and the general public. 0 2 4 6 8 10 12 14 16 Link or display on home page Social media Press release E-mail or text message subscription lists Hard copies distributed at meetings or venues Number of Transit Agencies C om m u n ic at io ns T oo l Figure 11. Communications tools used to publicize customer-focused transparency. Link or display on agency home page Social media Press release E-mail or text message subscription lists Hard copies distributed at meetings or venues Number of Responses C om m un ic at io ns T oo l Transit riders General public Public officials Others 0 5 10 15 20 25 30 35 40 Figure 12. Communications tools used to publicize customer-focused transparency to different audiences. Note: Transit agencies were able to select more than one audience for each tool.

Customer-Focused transparency 35 Implementing Customer-Focused Transparency Customer-focused transparency may be implemented for many reasons and in many ways. In some cases, transparency might be prescribed by an oversight body or external mandate (e.g., an open data order or an oversight board might require reporting of selected data and metrics). On the other hand, the desire to implement customer-focused transparency may come from within a transit agency—perhaps as a part of a new strategic plan from the transit agency’s chief executive. This section briefly describes the implementation of customer-focused transparency based on data collected from surveyed transit agencies. Rationale for Implementing Customer-Focused Transparency Transit agencies were asked to provide the main reasons they implemented customer-focused transparency. Respondents were provided with a set of options to rank and were allowed to rank as many options as applied. Table 16 displays the overall ranking of each available option. Increasing customer confidence was the highest-ranked reason for implementing customer-focused transparency, which suggests that transit agencies saw a connection between transparency and their relationships with customers. The second-highest-ranked reason, “the transparency was part of a strategic plan or customer commitment,” suggests that transit agencies are including transparency in their broader plans and not just using transparency as a stand-alone initiative. Transit agencies were also allowed to provide their own reasons for implementing customer-focused transparency. Seven transit agencies provided their own responses, paraphrased here: • Meet legislative and department of transportation requirements, • Increase customer satisfaction, and • Have a data-driven approach to inform decisions in support of improved performance. Two transit agencies’ survey respondents reported that the reasons for implementing customer- focused transparency were not clear. Case example agencies provided additional insights into their implementations (see Chapter 6 for details). At GO Transit, transparency was part of a broader, customer-focused initiative Reason Program Implemented Overall Ranking Number of Rankings To increase customer confidence 1 14 The transparency was part of a strategic plan or customer commitment 2 12 Requested by board (or other oversight body) 3 11 To improve public perception 4 12 External mandate 5 4 Requested by public 6 5 To increase ridership (more passenger trips) 7 3 Temporary practice in response to service problems 8 2 To increase fare revenue 9 2 Note: Respondents were able to select more than one reason, so the total number of rankings exceeds the number of respondents. Table 16. Reasons transit agencies implemented customer-focused transparency. The main reasons for implementing customer-focused transparency were • Increasing customer confidence, • Enacting a strategic plan or customer commitment, • Requested by an oversight body, and • Improving public perception.

36 Customer-Focused Service Guarantees and transparency practices in which GO Transit developed a five-promise passenger charter. Each promise was assigned a key performance indicator (KPI). Performance is updated monthly and displayed on the pas- senger charter web page and on GO Transit’s scorecard. MBTA and SFMTA both implemented their transparent reporting practices as part of broader strategic initiatives to increase customer confidence and be more transparent overall. Perceived Challenges and Concerns Causing Agencies Not to Implement Customer-Focused Transparency Three of the six transit agencies that do not have customer-focused transparency indicated that they had considered implementing customer-focused transparency but perceived the fol- lowing challenges preventing implementation: • System limitations, • Internal resistance, • Lack of staff time to analyze and maintain customer-facing data, and • Website not yet equipped to handle the data. Although a small sample, these responses indicate concerns with both people and systems. In particular, it may be harder for transit agencies without automated data collection and reporting systems to maintain customer-focused transparency because of the amount of labor required to collect, analyze, and publish data. Also of note is the reported internal resistance at one transit agency—this agency is likely not alone in the transit industry. Becoming transparent with cus- tomers about actual service performance can appear to be a risky proposition for existing transit agency leadership, especially if performance is not as excellent as desired. Benefits, Challenges, and Lessons Learned Transit agencies initiate customer-focused transparency for many reasons, and whether the actual and perceived benefits and challenges line up with initial expectations is not guar- anteed. Transit agencies that completed the survey reported on the perceived benefits and challenges. Transit agencies that participated as case examples also provided some lessons learned. Benefits of Customer-Focused Transparency The 17 transit agencies with customer-focused transparency were asked, “To the best of your knowledge, how have customer-focused trans- parency practices benefited your transit agency?” and were directed to indicate their level of agreement with a list of possible benefits. Figure 13 shows the results. The benefits that garnered the highest levels of agreement (either moderately or strongly agree) were the three benefits concerning the transit agency’s image with: • The general public [88% agreement (15 of 17); 35% strongly agree]; • The media (88% agreement; 35% strongly agree); and • Public officials (88% agreement; 53% strongly agree). The second tier of benefits all related to customer service (70% agreement), service quality (70% agreement), and customer satisfaction (53% agreement). Remaining listed benefits had low levels of agreement or high levels of responses of “not applicable/don’t know.” The main benefits of customer-focused transparency were • Improving the transit agency’s image with the general public, media, and public officials; and • Improving customer service, service quality, and customer satisfaction.

Customer-Focused transparency 37 Transit agencies were able to provide their own list of perceived benefits in addition to those listed in the survey; a few notable responses are paraphrased here: • Agency decision making is informed by data; • Has helped in developing a customer-first culture; and • Has improved relationships and trust with member cities, media, and stakeholders. Case example agencies reported additional benefits of their transparency programs (see Chapter 6 for details). MBTA found that its interactive online dashboard not only provides a plat- form for customers to view MBTA’s progress in improving service but also increases employee awareness of performance. GO Transit also found an improvement in employees’ focus on cus- tomer service. GO Transit also reviews performance trends on a regular basis and develops action plans to improve problematic services and displays the plans on the GO Transit website. Challenges of Customer-Focused Transparency The 17 transit agencies with customer-focused transparency were also asked, “What have been the negative impacts or challenges associated with your transit agency’s customer-focused transparency practices?” Respondents were directed to indicate their level of agreement with a list of possible challenges. Figure 14 shows the results. The challenges that were viewed as most significant by survey respon- dents were • Administrative burden [59% (10 of 17) reported a moderate or sig- nificant challenge]; 6% 6% 12% 6% 6% 6% 6% 6% 12% 29% 41% 47% 47% 6% 6% 6% 18% 12% 18% 18% 18% 18% 53% 53% 35% 41% 29% 35% 6% 6% 35% 35% 53% 29% 41% 12% 24% 18% 18% 6% 6% 6% 6% 6% 0% 100% The program has improved customer satisfaction The program has brought new customers to the transit agency The program has increased ridership The program has increased fare revenue The program has improved the transit agency’s image with the general public The program has improved the transit agency’s image with the media The program has improved the transit agency’s image with elected officials The program has improved the transit agency’s employees’ commitment to customer service The program has helped the transit agency improve service quality Strongly disagree Slightly disagree Neutral Moderately agree Strongly agree Not applicable / Don't know Figure 13. Perceived benefits of customer-focused transparency. Note: Totals may not equal 100% because of rounding. The main challenges of customer-focused transparency were • Administrative burden, and • Request for more data or inquiries about data.

38 Customer-Focused Service Guarantees and transparency practices • Request for more data or metrics (65% reported a moderate burden); and • Responding to inquiries about the data (41% reported a moderate burden). Only one transit agency rated any challenges as significant, including financial expense of the program and administrative burden. (One agency equates to 6% of the sample.) On the other hand, employee buy-in and support was rated as not a challenge by 71% of respondents. Transit agencies were also able to provide their own challenges. Four responses were received, two of which referred to data integrity issues. One response discussed the challenges posed by altering calculation methods for performance metrics, which results in historical inconsistencies that are difficult to explain. Last, one transit agency provided a comment that is worth paraphrasing in detail: Performance reports appear simple, but are not. They involve data mining, summarization, gathering of departmental comments, and assembly into a presentation. The final report must be error free, despite the number of data elements involved. Changes to data or report documents are very difficult to imple- ment. Performance reporting requires skilled programmers and analysts, meetings to coordinate depart- mental responses, and administrative time. Case example agencies provided additional insights into some of the challenges of customer- focused transparency (see Chapter 6). SFMTA found monthly performance reporting—including the assembling and analyzing of data from various internal departments—to be somewhat chal- lenging. The difficulty of assembling the data is also exacerbated by a shortage of employee buy-in. MBTA experienced an increase in data requests—both for the raw data underlying its interactive dashboard and for new data or metrics. Lessons Learned Information from the surveyed and case example transit agencies provided several lessons that transit agencies implementing or considering implementing customer-focused transpar- ency may find beneficial. • Lessons related to employees – Engaging employees early on in the development of a customer-focused transparency program may encourage employee buy-in. Allowing for employee feedback can help Not applicable / Don’t know Not a challenge Moderate challenge Significant challenge 12% 18% 6% 12% 12% 12% 29% 65% 71% 65% 24% 47% 53% 12% 24% 24% 65% 41% 6% 6% 0% Administrative burden of managing the program Financial expense of the program Lack of employee buy-in and support for the program Difficulty implementing transparent reporting for transit services provided by transportation contractors/providers Requests for more data or metrics Responding to inquiries regarding the data 100% Figure 14. Perceived negative impacts or challenges of customer-focused transparency practices. Note: Totals may not equal 100% because of rounding.

Customer-Focused transparency 39 ensure employees are empowered to participate in and benefit from the performance reporting. – Some employees may feel threatened or vulnerable when performance of individuals or single departments is being analyzed, scrutinized, and publicized. – Transit agencies may benefit from focusing on multidepartmental performance metrics and encourage departments to work together to improve performance as a team. • Lessons related to managing the transparency program – The administrative burden of a customer-focused transparency program may become an issue for transit agencies, depending on the complexity of the data and level of reporting automation. Transit agencies may consider automating as much of the process as possible or, alternatively, bringing additional resources to the reporting process (e.g., hiring addi- tional staff). – It is recommended that any reporting system (developed or purchased) supporting a customer-focused transparency program be easily scalable and built anticipating that more metrics, more data, and more features are likely in the near future. – Requests for more data or metrics may become a challenge. Agencies may consider establish- ing a formal data request process or build tools to allow users to download data themselves. – Executive buy-in and leadership support is a necessary condition for any customer-focused transparency program to succeed and be sustained. • Lessons related to customers – Customer-focused transparency has played some role in improving transit agencies’ image with elected officials, the media, and the general public and improving customer satisfaction. – Transit agencies may need specific communication tools to discuss performance data. Agencies may have to alter historical data, explain complicated metrics, or discuss data flaws and need a platform to document these issues in straightforward, meaningful ways. – Using plain language in performance reporting is critical—especially when tying perfor- mance to specific promises or service quality commitments. – Engaging customers when selecting KPIs and developing the content for performance- reporting web pages and documents can help ensure that performance reporting is focused on what the customer cares about and how customers want to see and consume perfor- mance information.