Minutes Matter: A Bus Transit Service Reliability Guidebook (2020)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

30 4.1 Introduction This chapter provides information regarding Step 2 and Step 3 of a reliability improvement program: selecting reliability measures and selecting standards. It first describes the different types of measures for monitoring reliability, then presents an overview of the many measures that can be used to assess reliability identified through the review of the literature, the survey of transit agencies, and the detailed case studies. It then summarizes the wide variety of measures into a set of general measures based on the data required. Each of the measures is described in greater detail in Chapter 7. This chapter then provides a comparison of the general reliability measures considering their suitability for the various needs of a given transit agency. It concludes with a discussion of possible data sources for reliability measures. The measures discussed in this chapter are those used to monitor reliability. Tools and measures used to diagnose reliability problems identified through such monitoring are discussed in Chapter 5. 4.2 Reliability Measures There has been much debate regarding the merits of having a single measure of reliability as opposed to several measures that reflect different aspects of reliability, and the matter is far from settled. However, with widespread implementation of AVL and APC systems and other automated data collection technologies, transit agencies and researchers have access to an abun- dance of data, which is allowing them to use more fine-grained and data-intensive methods that were previously cost-prohibitive or infeasible. While the reliability measures discussed in this guidebook capture the vast majority of what has been documented in research and identified through the agency survey, they should not be considered an exhaustive list. Nevertheless, the measures summarized in this chapter and discussed in Chapter 7 will likely be sufficient for most practitioners. 4.2.1 Understanding the Different Types of Measures With many different reliability measures proposed over the years, selecting the most appro- priate one for a specific situation and purpose requires an understanding of the various measures available, how they differ, and for what purposes they are most useful. The various reliability measures address different aspects of reliability and consider reliability from different perspec- tives. In addition, most measures can also be applied at different levels of application, often for different purposes, and the same data may be manipulated using different statistical techniques to produce different, but related, reliability measures. C H A P T E R 4 Reliability Measurement Tools

Reliability Measurement Tools 31 Three Aspects of Reliability Many different definitions of bus service reliability have emerged over the years, and there is a great deal of overlap and similarity between them. As time passes, however, researchers seem to be less interested in explicitly defining reliability and more focused on identifying the best measures and measurement techniques for tracking and reporting on bus service reliability. Most definitions of reliability, however, appear to address one or more of three commonly cited aspects of reliability, each of which leads to its own set of measures. • Punctuality refers to the actual time a bus arrives or departs compared with the schedule or expected time (i.e., does the bus service operate at 10 minutes past the hour, when it is scheduled to operate, or does it operate at 15, 20, or 8 minutes past the hour?) • Variability considers the degree of consistency of a service (i.e., does the bus operate at 10:10 a.m. today, 10:20 a.m. tomorrow, and 10:12 a.m. the next day?) • Non-Operation refers to whether a service is provided at all on a particular day (i.e., does the trip fail to run?) Level of Application Reliability measures can be applied at multiple levels. While many measures can be applied at several different levels, they may perform different functions depending on the level applied. Higher levels of aggregation can provide a good overall portrayal of system performance, while more disaggregate levels are more useful for identifying specific problems. • System-level measures are calculated across all services (or all services of a particular type) provided by a transit agency and are useful as an overall report card for management and the public. They provide a general assessment of an agency’s performance while not providing specific information on where problems may exist. • Route-level measures are calculated separately for each route provided by a transit agency and are useful for comparisons across routes and for targeting for further attention those with poor performance. • Trip-level measures are calculated separately for each trip on a route over multiple days and are useful for identifying specific times of day where delays occur; they can also be used to identify specific operators who are having difficulty with reliability. • Stop-level measures are calculated separately for a particular stop on a route over multiple trips and are useful in comparing reliability at different points along a route; they can be used to identify at what point along the route reliability begins to decline. Statistical Techniques The measures that can be computed from a given set of data can take several different math- ematical forms. Simple calculation of differences in headways or travel times, measured in minutes, is an essential first step in gathering reliability data and can be useful in understanding the extent of reliability issues for a particular route, trip, or stop. However, normalizing data to dimensionless measures, such as percentages, variability, or indices, allows for easier comparison across routes or services. • Measures expressed in units of time are easy to understand and can be useful in characterizing the reliability of a specific route or at a specific stop for a specific trip or period of time. How- ever, they can be misleading when used to compare different routes, stops, or periods with very different service characteristics. An example would be that the average wait time experienced by a customer at a given stop is 10 minutes, or that a bus is on average 5 minutes late at a given time point. • Percentages and distributions are probably the most common form of reliability measure. Such measures are the percentage of observations that fall within a specified range, usually

32 Minutes Matter: A Bus Transit Service Reliability Guidebook a range centered on a base value such as a mean, median, scheduled, or expected time. Such measures are easy to understand and to explain to the public but fail to capture the true variability of a service characteristic because they typically make an all-or-nothing assessment of an observation to determine whether it fits into an acceptable range. The most common use is the percentage of trips on time for a given time point, route, or agency. It may be appropriate for some measures to create multiple ranges to reflect a more fine-grained distribution of reliability performance. Ranges and distributions may be defined as absolute values or as percentages of the base value. An example would be categorizing headways at a time point and showing that 20 percent are less than half the scheduled headway, 50 percent are close to the scheduled headway, and 30 percent are more than 1.5 times the scheduled headway. • Variability measures, such as variance (Var), standard deviation (SD), and coefficient of varia- tion (CV), identify the extent of variability of the data and are therefore useful in measuring the variability of data such as headways and travel times. Of these, the CV may be most useful in that it is dimensionless, consisting of the ratio of the standard deviation to the mean, making it a measure that allows comparison between different routes or stops that have quite different running times or headways. An example would be comparing the running time variability of two routes of different lengths. A route with a 20-minute average running time with a standard deviation of 3 minutes would have the same CV (0.15) as a route with a 30-minute running time and a standard deviation of 4.5 minutes. • Indices, often based on the 90th or 95th percentile value, provide an indication of the limit of the more commonly observed high values of a measure, while ignoring the rare extreme values. A value representing the 95th percentile (in other words a value that is exceeded in only one of 20 instances) can be thought of as a value exceeded roughly once per month on a trip made every weekday. The 95th percentile value can be compared to the mean to provide a dimen- sionless index for comparing services with differing mean values for a measure. An example would be using data on running times on a given trip collected for 100 days. While average running time might be 30 minutes, on 2 rare days construction or bad weather resulted in running times of over 50 minutes, but the 95th highest running time was only 39 minutes. That 95th percentile value is divided by the average running time to yield an index of 1.3. 4.2.2 Reliability Measures in the Literature More than 150 different measures for bus transit reliability were found in the several decades of research that were reviewed in the preparation of this guidebook. While some of these measures are only slight variations on others (schedule adherence and on-time performance, for example), a few are novel approaches to quantifying bus service reliability. Certain measures, such as schedule and headway adherence, have been widely used for decades, but to date, there is no single measure that has emerged as clearly superior in all cases. Overall, it seems that one (or more) of a variety of reliability measures could be suitable for transit agencies and other organi- zations to use, depending on the characteristics of the service(s) they offer and the audience(s) they are targeting, among other factors. Common Measures Service reliability measures found in the literature included measures of on-time performance/ schedule adherence and headway adherence, as well as various measures of the variability of travel times and running times. In the literature review, “on-time performance” was cited in 38 studies, with the analogous measure “schedule adherence” cited 20 times. (By most definitions, these two measures are essentially the same.) Figure 4.1 shows the top 15 reliability measures identified from the literature review. While on-time performance addresses the punctuality aspect of reliability, many of the remaining frequently cited measures relate to the variability aspect of reliability—both the

Reliability Measurement Tools 33 variability of travel times and of wait times. It is interesting to note, however, that none of the frequently cited measures in the literature address non-operation, even though the agency survey found that such measures are commonly used in the industry. Frequently cited measures in the literature do show a balance between agency and customer perspectives, with some from the agency perspective (e.g., on-time performance, headway regularity, and running time variance) and some from the customer perspective (e.g., wait times, travel time, and buffer time). In recent years, more customer-focused measures have become popular, as is evident from at least 26 different studies that mentioned the importance of measuring and reporting reliability from the perspective of customers. Customer-oriented measures tend to focus on aspects of travel or journey time, including wait time, and may include elements of customer feedback, such as survey ratings or the number of complaints. Novel Measures As transit performance data have become more widely available, many new reliability measures have been proposed and tested, with many researchers seeking a single measure or set of measures to provide a holistic understanding of reliability from the operator and customer perspectives. Like the widely used measure of traffic level of service, one researcher has proposed a reliability quality of service measure [14]. Other researchers have proposed various measures of buffer time, which is essentially the extra time a customer needs to budget, over and above the average or scheduled travel time, to account for unreliability. 4.2.3 Transit Agency Application of Reliability Measures Eighty-six agencies operating fixed-route bus service responded to the agency survey as part of the development of this guidebook. As part of the survey, agencies provided information concerning their definition of reliability and the reliability measures used. Definition of Reliability When asked how their agencies defined “fixed-route bus service reliability,” most agencies answered that they defined it in terms of one or more specific measures. Most replied that 38 20 18 15 15 13 12 11 10 10 9 9 8 8 7 0 5 10 15 20 25 30 35 40 On-time performance Schedule adherence Wait times Delay Travel times Travel time variance Schedule deviation Travel time coefficient of variation Excess wait time Headway regularity Buffer time Running time variance Headway adherence Headway variance Running times Number of Documents Reviewed that Included Measure M ea su re Figure 4.1. Top 15 bus reliability measures by count of reviewed documents.

34 Minutes Matter: A Bus Transit Service Reliability Guidebook “on-time performance” or “schedule adherence” was used, with varying targets and standards identified. Other less-used measures that agencies used as definitions included “missed trips,” “travel time variability,” and “vehicle reliability.” Several agencies had no formal definition. Table 4.1 shows the most common definitions stated by survey respondents. While most agen- cies listed a definition, it was clear from the survey responses that most had no formally adopted definition. Adopting a formal definition as part of the goal setting process would help agencies identify the best measures to use as well as the most effective treatments to address goals. Measures of Reliability Respondents were asked to identify which, from a list of 18 performance measures, their agen- cies use to identify fixed-route bus service reliability at the system, route, trip, and stop levels. As shown in Figure 4.2 through Figure 4.5, most responding agencies used “on-time performance” as their primary performance measure for reliability at all levels (system, route, trip, and stop). How does your agency define fixed-route bus service reliability?* On-time performance (including variation of definitions of “on time”) 51% Schedule adherence 30% No formal agency definition 17% Missed trips 7% Travel time 4% Vehicle reliability 4% *Agencies were able to choose more than one option. Table 4.1. Most common agency definitions of reliability. 83% 56% 40% 35% 29% 28% 28% 28% 24% 16% 16% 14% 14% 13% 9% 6% 2% 2% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% On-Time Performance Missed Trips Number of Accidents Distance Between Vehicle Breakdowns Pull-Out Performance Driver Attendance Missed Pullouts/Missed Driver Runs Missed or Unfulfilled Scheduled Hours Headway Adherence Recovery Time Time Between Vehicle Breakdowns Added Trips Maintenance Personnel Attendance Distance Without Service Interruption Travel Time Variability Excess Wait Time Average Wait Time Wait Assessment Figure 4.2. System-level reliability measures.

Reliability Measurement Tools 35 At the system level, 87 percent of the 86 agencies responding indicated that they measured reliability performance. If an agency responded that it used a single performance measure, it was “on-time performance.” On the other hand, one large agency indicated that it used almost all (15 out of 18) of the listed performance measures at the system level. In the open-ended section, agencies indicated that some other measures they used were “hours of service interruption,” “passenger miles,” “farebox recovery,” and “customer feedback.” At the route level, 74 percent indicated that they measured reliability performance. If an agency responded that it used a single performance measure, it was either “on-time performance” or “missed trips.” Two agencies indicated that they used most of the listed performance measures at 81% 45% 33% 22% 21% 20% 17% 12% 5% 3% 2% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% On-Time Performance Missed Trips Headway Adherence Recovery Time Travel Time Variability Number of Accidents Missed or Unfulfilled Scheduled Hours Added Trips Excess Wait Time Average Wait Time Wait Assessment Figure 4.3. Route-level reliability measures. 82% 48% 31% 20% 18% 17% 14% 11% 6% 6% 6% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% On-Time Performance Missed Trips Headway Adherence Travel Time Variability Recovery Time Number of Accidents Missed or Unfulfilled Scheduled Hours Added Trips Average Wait Time Excess Wait Time Wait Assessment Figure 4.4. Trip-level reliability measures.

36 Minutes Matter: A Bus Transit Service Reliability Guidebook the route level. In the open-ended section, agencies indicated that some other measures they used were “hours of service interruption,” “pullout adherence,” “passengers/revenue hour,” and “customer feedback.” At the trip level, only 26 percent indicated that they measured reliability performance. If an agency responded that it only used one performance measure, it was either “on-time performance” or “missed trips.” One agency indicated that it used many of the listed performance measures at the trip level. In the open-ended section, agencies indicated that some other measures they used were “monitoring roadway traffic,” “trip times,” “passengers/revenue hour,” and “customer feedback.” At the stop level, half of the agencies indicated that they measured reliability performance. In the open-ended section, agencies indicated that some other measures they used were “passengers/day,” “time point on-time performance,” and “dwell time.” It is notable that measures identified by the agencies were more focused on the operator perspective and measures of non-operation, while measures identified in the literature tended to focus on the agency and customer perspectives and were mostly measures of punctuality and variability. Agencies appeared to be making basic assessments of on-time performance and of travel time and wait time variability while focusing on measures that addressed some of the perceived causes of unreliability. The 10 case studies completed for this project also provided some input on the measures used. Nine of the 10 agencies studied used on-time performance as their primary measure of reliability (the exception being Transport for London, which focused on excess wait time). Half of the 10 agencies [Denver Regional Transportation District (RTD), Chicago Transit Authority (CTA), LA Metro, Southwest Ohio Regional Transit Agency (SORTA), and Kingston (CA)] also used a measure of service availability or frequency of breakdowns. The only agencies monitoring head- ways were Kingston and CTA, while VIA and Transport for London considered excess wait times. Only RTD and Kingston indicated that they monitored running time variability. 4.2.4 Generalized Measures for Fixed-Route Bus Service Reliability The research conducted to prepare this guidebook identified a comprehensive list of measures from the literature and from agency practice. However, the list can be consolidated to a smaller number since some measures (schedule adherence and on-time performance, for example) are synonymous, and others (such as running time variability and distribution of running times) 56% 7% 4% 4% 3% 3% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% On-Time Performance Headway Adherence Average Wait Time Excess Wait Time Wait Assessment Number of Accidents Figure 4.5. Stop-level reliability measures.

Reliability Measurement Tools 37 are just different calculations from the same set of data. Table 4.2 presents a consolidated list of common measures grouped by the aspect of reliability addressed. Each measure represents a set of measures that can be derived from a specific dataset using different calculations. Chapter 7 provides further discussion of each of the measures, including the aspect addressed, perspective (customer/agency/operator), data sources used, typical standards, methods of calculation, typical analyses, common usage, and suitability for various purposes. 4.2.5 Standards, Targets, and Thresholds To lend meaning to the data produced through reliability measurement, standards and targets are needed to measure whether a specific goal is being accomplished. Measures evaluated using percentages and distributions (such as on-time performance) need standard definitions for the range of acceptable performance for a given observation (such as how early or late a trip can be to be considered “on time”). All measures then need some sort of target, related to agency goals, to compare reliability performance for a given route, trip, or stop, or for the system. These can be agency-set fixed targets, or they may be a benchmark value derived from actual relia- bility data for that agency or from peer agencies. (An agency may also set threshold values, below which corrective actions are required, that may differ from the more aspirational target values.) An agency may also choose to use different standards and targets at the various levels of application. Common standards for the most frequently used measures, on-time performance, headway, and wait time, were identified through the agency survey and case studies and are discussed in detail under each of those measures in Chapter 7. For any measure, agencies typically set a target for acceptable reliability. At the system level, agencies may have a policy-set goal for reliability used as part of an overall assessment of system performance. Agencies may also monitor trends over time to assess whether reliability perfor- mance is improving. Without any generally accepted industry-wide target levels for reliability, agencies may choose to assess reliability in comparison to peer agencies, although, for on-time performance, this would require a standardized definition of the on-time window. At the route, Aspect of Reliability Data Needed Reliability Measure Perspective Punctuality Arrival and departure times On-time performance/schedule adherence Agency Variability Trip start and end times Running time Agency Dwell time at stops Dwell time Agency Customer travel times Travel time Passenger Buffer time indices Passenger Time between buses Headways Agency Customer wait times Wait times Passenger Non-operation Records of missed service Pullouts missed Operator Missed hours of service Operator Scheduled trips cancelled Operator Counts of service disruptions Number of crashes Operator Mean distance between failures Operator Multiple Customer surveys Passenger ratings of reliability All Table 4.2. Common bus service reliability measures.

38 Minutes Matter: A Bus Transit Service Reliability Guidebook trip, or stop level, different targets or thresholds may be used for a given measure to determine whether more detailed data or analysis is needed to determine the reasons for poor reliability. Most of the case study agencies set specific reliability targets for on-time performance and schedule adherence, ranging from a low of 60 percent [New York City Transit (NYCT) and Manatee County Area Transit (MCAT)] to 88 percent (Denver RTD and SORTA). System size appears to have little to do with the goal since the largest (NYCT) and the smallest (MCAT) have the same low goal, while the highest goals were larger and medium-sized agencies. Transport for London set route- specific targets for average excess wait time, based on an assessment of achievable improvements given past performance. The standards and targets chosen should be neither unrealistic nor too easy to achieve. Achieve- ment should require work, but the benefit should outweigh the cost of achieving the improved reliability, and the goal should not be set so high that it can never be reached. More information of performance standards can be found in TCRP Report 88: A Guidebook for Developing a Transit Performance-Measurement System [15]. 4.3 Comparison of Reliability Measures To develop a comparison of the various measures of reliability, a set of comparison criteria is needed. Chapter 2 reviewed several sets of desired qualities in measures that are instructive in identifying evaluation criteria. One of the themes that appears frequently is that measures should be not only simple and easy to understand but should be useful for the target audience. Measures should also be focused on identifying impacts on customers but should use cost-effective methods of data collection. Being able to make comparisons not only across routes but across agencies and even different modes has been noted by several transit reliability researchers. Considering these qualities, several criteria were identified for comparing measures for assessing bus transit service reliability. Reliability measures should: • Address the Definition of Reliability – A clear understanding of how bus transit reliability is defined will aid in identifying appropriate measures. • Accurately Represent the Impact on Customers – Reliability is ultimately experienced by the customer, and measures should go beyond reliability of service provision from the agency point of view and reflect the full set of impacts on customers. • Involve Low-Cost Data Collection, Take Advantage of Existing Data and Resources, and Be Easy to Calculate – Measures that can be produced without undue burden are more likely to be adopted by a wide range of agencies with varying levels of resources and sophistication. • Allow Comparisons with Other Transit Services and Across Transit Modes – Measures should be scalable across all levels within an agency or geographic area and should be defined, collected, and reported consistently so that they allow comparisons among routes and between different transit modes and agencies. • Allow Multimodal Comparisons – Measures should allow comparisons among all modes, including nontransit modes, to assess transit competitiveness. • Be Useful in Identifying and Targeting Corrective Actions – Going beyond recognizing the current reality and identifying causes of unreliability and corrective actions to address unreliability will help agencies improve the experiences of their customers, increase rider- ship, and promote transit use. • Be Easy to Communicate to Customers, the Public, and Elected Officials – Measures should be understandable to the target audience, and preferably to all audiences, including agency staff, customers, non-riders, board members, and public officials. • Relate to the Agencies’ Goals and Objectives.

Reliability Measurement Tools 39 The first criterion requires further discussion, including consideration of the various defini- tions of reliability. The earliest definitions identified in the literature focused on variability and consistency of performance. More recent definitions have taken on a more customer-focused perspective. While many definitions have been proposed, consistency of service to customers is a common theme. The three components of reliability that appeared most frequently in the definitions were: • Short and consistent wait times, • Consistently on-time arrivals at the destination, and • Consistent travel times. These three points are used as the definition of reliability in the following discussion. In viewing the measures in Table 4.2, the measures from the agency and customer perspectives (those that address punctuality and variability) address one of these three components. Those from the operator perspective appear less oriented to such direct impacts on customers, but rather they address the non-operation aspect of reliability and represent factors over which agencies have more direct control. Such operational measures may not directly address the definition of reliability but do so indirectly and may aid in identifying causes of unreliability as well as ultimately leading to corrective actions. The measures shown in Table 4.2 were compared using seven of the eight criteria described previously. (The eighth criterion, relate to agency goals and objectives, is not used because this criterion would be too agency-specific.) A summary comparison is shown in Table 4.3, with each Aspect of Reliability Data Used D ef in iti on C us to m er Im pa ct C os t/E as e Tr an si t C om pa ris on s M ul tim od al C om pa ris on s C or re ct iv e A ct io ns C om m un ic at in g R es ul ts Punctuality Arrival and departure times very good very good very good very good fair poor very good Travel time variability Route running time very good good very good good poor poor good Dwell time, travel speed, and signal delay good fair fair good poor fair fair Customer travel times very good very good poor very good very good poor good Buffer time indices very good very good poor very good very good poor very good Wait time variability Time between buses (headways) good good good very good poor poor fair Customer wait times very good very good fair very good poor poor good Non-operation Missed service fair fair very good very good poor fair fair Service disruptions fair fair very good very good good good fair Multiple Customer surveys very good very good fair good good fair very good Table 4.3. Comparison of reliability measures.

40 Minutes Matter: A Bus Transit Service Reliability Guidebook group of measures rated as very good, good, fair, or poor depending on how well each group of measures addresses each criterion. A measure was given a rating of very good if it addresses the criterion in the most direct fashion. Good indicates that a measure addresses the criterion less directly, partially, or only with some difficulty. Fair indicates that the measure may have only some limited relevance to the criterion. Poor means that the measure provides little to no ability to address the criterion. A more detailed discussion of the possibilities for calculating measures in each group, and the reasons for the ratings given in the table, are presented in Chapter 7. 4.4 Data Sources and Data Collection Techniques Measuring bus service reliability has historically been difficult due to the large amount of data needed to calculate even the most basic reliability measures. For generations, performance data were collected manually by field supervisors who both regulated and monitored service. Over the years, due to budget restrictions, many agencies have reduced the number of supervisors, which reduced their ability to monitor service as their reduced numbers caused them to focus their efforts more on problem resolution than on data collection. More recently, however, the introduction of GPS-based, real-time service monitoring technologies has offered transit agencies richer data and much more powerful tools to monitor service. Some agencies make effective use of real-time data, while other agencies are still learning how to use these tools to the greatest benefit. The vast amount of data produced by these monitoring systems provides a rich source of information on bus operations that can be analyzed over a period to assess the variability of service and, thus, its reliability. There is currently a wide range of traditional, emerging and innovative data collection techniques available to assemble the necessary raw data for calculating reliability measures. • Agency Operations Reports – Agencies have historically kept track of the amount of sched- uled service operated and the amount missed. Agency reports can provide the data needed for measures of non-operation, such as missed pullouts, missed trips, and missed hours of service. Reports on the amount of scheduled service provide the basis for calculating the percentage of service missed. • Manual Data Collection – Prior to the introduction of automated AVL, APC, and AFC data systems, transit agencies had to depend on data collected manually by supervisors or dedi- cated data collection “checker” staff. Some agencies that have yet to install automated systems still rely on manual methods for checking reliability. Data collected by supervisors can achieve large enough samples to assess on-time performance. However, because data collection is not a supervisor’s primary responsibility, data collection is sometimes sacrificed when other duties intrude, which is typically when service is least reliable. This can bring into question the representativeness of the data. Using dedicated checkers can provide a more representa- tive sample for both on-time performance and travel time variability; however, the cost of dedicated data collection staff usually means that the amount of data that can be collected is limited and may not be sufficient to calculate most measures of variability. • Agency Reports, Logs, and Observations – While many agencies have replaced manual data collection with automated AVL and APC systems, supervisor reports and dispatch logs remain a valuable source of information on reliability. Though maybe less valuable at quanti- fying reliability, routine agency reports and logs can provide valuable insights into reliability issues. When combined with other measures of reliability, such reports can help provide a more complete picture of the service being provided. • AVL Data – With the more widespread implementation of automated data collection technolo- gies in recent years, transit agencies have access to an abundance of data, which is allowing them to use more fine-grained and data-intensive methods that were previously cost-prohibitive or unfeasible. AVL systems are typically installed on the entire fleet and can provide observa- tions of on-time performance and travel time for every trip, every day, resulting in a very

Reliability Measurement Tools 41 large sample of data from which to calculate reliability measures. The challenge for agencies often lies with developing the data management and reporting capabilities to analyze the huge amount of data collected. • APC Data – APC systems are often implemented alongside AVL systems; some agencies only equip a fraction of their fleet with APC systems, while others equip the entire fleet. APC systems provide data on passengers boarding and alighting at each stop as well as the number of passen- gers on board. They can also provide a finer-grained breakdown of bus running times, including the time spent with the doors open while loading and unloading passengers. Analyzing travel times and passenger data can also provide an assessment of the number of customers affected by unreliable service. • AFC Data – AFC systems can track the movements of customers who use smart cards and help in understanding transfer patterns and (in conjunction with AVL data) transfer times. Most AFC systems do not have the ability by themselves to measure customer travel times because most bus AFC systems (unlike many rail systems) do not require customers to “tap out” when leaving the bus. However, origin/destination (O/D) transfer inference tools [often referred to as open data exchange (ODX) tools] have been developed to create full journey origin/destination metrics by merging AFC data with AVL data. These tools can estimate full journey times as well as travel times and wait times for each trip segment (excluding the initial wait for service). • New Technology Data Sources – The prevalence of cell phone use has resulted in new customer-focused travel data collection techniques that make use of cell phone signals to track customer movements. Data can be obtained that indicate travelers’ trip origins, destina- tions, routes, time of travel, and frequency of travel. The level of detail available from these data can allow the inference of things like travel mode, transit services used, and trip purpose. However, the data lack any information on traveler demographics. • Customer Surveys – Customer surveys have always been a technique used to understand customer perceptions of the quality and reliability of bus service. Traditionally, paper forms distributed to customers on buses, interviews by checkers, and phone surveys were used. More recently, surveys completed on handheld tablets and web-based surveys have become more common. Each of these forms is similar in that they all allow customers to describe their experiences and express opinions on the quality of the service provided. • Complaints and Social Media Input – For years, transit agencies have collected, reported, and acted on customer complaints. More recently, the expansion of the use of social media platforms has created a new source of data on customer perceptions of reliability. Mining these data can provide guidance to agencies on specific problem areas. • Reports from Other Transit Agencies – Reports from other transit agencies can be helpful in comparing reliability across agencies. However, to be useful, agencies must be using the same measures and standards. Most case study agencies used CAD/AVL data, and several supplemented the AVL data with APC data, although combining data from the two systems into a single assessment of reliability can prove challenging. Many agencies also used other sources of information, such as supervisor reports and social media. Some agencies, including NYCT and RTD, identified a goal of develop- ing an integrated reliability reporting system making use of reliability data, supervisor reports, customer complaints and reports, and social media feedback. 4.5 Reliability Management Organizational Structure In terms of the department responsibility for reliability data, Table 4.4 shows that most agency survey respondents indicated that either the “transportation” or “planning” departments were responsible for collecting, reporting, and analyzing reliability data, with some involvement by “IT” and “other” departments.

42 Minutes Matter: A Bus Transit Service Reliability Guidebook Among the 10 case study agencies, the medium- and large-sized agencies tended to have multiple departments involved. In some cases, such as LA Metro, it was a collaborative respon- sibility of several departments, including transportation, IT, planning, scheduling, and main- tenance. At SORTA, planning, scheduling, radio control, transportation, and maintenance all worked together to address reliability, but without any formal structure. At other agencies, one department had overall responsibility. At VIA Transit, Denver RTD, and Pierce Transit, service planning was the lead department charged with addressing reliability. At NYCT, addressing reliability fell under road operations, with route managers assigned to sets of four to five routes to view individual bus operator on-time performance in real time and address issues and delays as they occurred. Road operations worked with the bus service planning, IT, and system data and research departments to develop data systems and internal processes to manage service as well as compare performance against previous years. Transport for London had a bus policy team that reviewed reliability with the eight bus sector planners to develop solutions. Collection Reporting Analysis Transportation (45%) Planning (45%) Planning (52%) Planning (30%) Transportation (38%) Transportation (33%) IT (13%) Other (17%) Other (15%) Other (11%) Table 4.4. Department responsibilities for reliability data.