Guidebook for Rural Demand-Response Transportation: Measuring, Assessing, and Improving Performance (2009)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

The research underlying this TCRP project identified more than 60 different performance measures that have been used to evaluate DRT service, with 16 or so measures used more com- monly. But one does not need a litany of measures to capture the important aspects of DRT per- formance, and six measures have been selected as the key measures for the Guidebook, given that one of the objectives of the project was to select a limited number of measures. 4.1 Key Performance Measures for Assessing Rural DRT The six measures selected for the Guidebook for assessing rural DRT performance are identified in Table 4-1 and discussed in this section. Depending on the results of those measures, a rural DRT system may need to delve deeper into certain aspects of its operations, examining more detailed data and assessing additional measures to address questions or questionable performance. Importantly, rural DRT systems must consider the extent to which their mission influences their day-to-day performance. When a rural system is tasked with serving the needs of riders who are transit-dependent, its DRT service will often include lengthy trips for critical purposes with limited opportunity for shared-riding. Its performance measures will then reflect lower produc- tivity and higher cost per passenger trip than might otherwise be the case. Passenger Trips per Vehicle-Hour Passenger trips per vehicle-hour measures the productivity of a DRT system. Many consider productivity to be the most important single measure of DRT performance, assessing the system’s effectiveness. As a performance measure, productivity captures the ability of the DRT system to schedule and serve passenger trips with similar origins, destinations, and time parameters, using the least number of in-service vehicles and hours. This is the essence of shared-ride, public DRT service. However, there are various important factors that affect the ability of a DRT system to be productive: the size of the service area; the distribution of residential areas and destination areas; and the patterns of riders’ trips, including the extent of group trips. Particularly for rural DRT systems, large service areas with dispersed trip patterns make it harder to effectively schedule two or more riders on the same vehicle; this will result in a lower productivity. The extent to which the rural DRT system serves pre-scheduled group trips will also impact productivity, such as group trips to the senior center or other frequented desti- nation. If there are limited group trips—that is, few opportunities to schedule riders on the same 23 C H A P T E R 4 Performance Measures for Rural DRT productivity = total passenger trips ÷ total vehicle-hours

vehicle at the same time for travel to a common destination—this will also mean a lower pro- ductivity. Other factors that impact productivity include the level of no-shows and late cancellations, scheduling efficiency, dispatcher skills, and ability to schedule trips in real-time, vehicle opera- tor experience and familiarly with the service area and their passengers’ trip-making patterns, and the operating environment including the roadway network and geographic barriers that impact that network. The type of DRT service—particularly whether it functions as ADA paratransit—also affects productivity because ADA regulations have effective limits on the flexibility that a DRT system has to maximize shared riding. This also may mean a lower productivity. Some of these are exam- ples of uncontrollable factors affecting the performance of DRT systems. Such factors impacting DRT performance are discussed in more detail in Chapter 5. From a DRT performance perspective, the emphasis on productivity stems in great part from the fact that small changes in productivity can be very cost effective. Larger changes can be even more cost effective. The productivity measure can be calculated with either revenue-hours or vehicle-hours in the denominator. For this Guidebook on rural DRT, the measure uses vehicle-hours, in keeping with one of the project’s objectives to use NTD data definitions. To the extent that vehicle-hours are generally fixed, at least in the short run, while revenue- hours occur only during passenger service, using vehicle-hours to measure productivity may provide a better indication of how well vehicle resources are being used over the course of a day, week, or month (5). However, in the guidebook for urban DRT (TCRP Report 124), the productivity measure uses revenue-hours as the denominator. Direct comparisons, then, between productivity data included in TCRP Report 124 and that included in this Guidebook for rural systems are not valid since deadhead time is included in the measure for rural DRT systems. In some cases, deadhead can be a significant proportion of vehicle-hours. For those rural DRT systems that collect revenue-hours data, productivity can be calculated using revenue-hours. Use of revenue-hours for measuring productivity has generally been used for demand-response transportation where revenue-hour data are available. Revenue-hours are typically used for measuring fixed-route productivity as well. Moreover, since revenue-hours are less than vehicle-hours, the productivity figure will be higher when revenue-hours are used as the numerator in the measure—passenger trips—is divided by a smaller number. Productivity is sometimes measured as passenger trips per mile. Given the low passenger vol- umes on DRT relative to mileage, this ratio usually results in a number less than 1. Such resulting numbers are not particularly logical given that an actual passenger trip is not less than 1; passenger trips per hour is an easier number to visualize. 24 Guidebook for Rural Demand-Response Transportation: Measuring, Assessing, and Improving Performance Table 4-1. Six key DRT performance measures selected for Guidebook. 1. Passenger Trips per Vehicle-Hour 2. Operating Cost per Vehicle-Hour 3. Operating Cost per Vehicle-Mile 4. Operating Cost per Passenger Trip 5. Safety Incidents per 100,000 Vehicle-Miles 6. On-Time Performance

Performance Considerations If a rural DRT system has a lower performance on passenger trips per vehicle-hour than desired, it should look to various possible reasons, only some of which the system can affect: • A large service area, where passenger trips are lengthy; • Low density of passengers within the service area; • System policies that allow riders to travel to destinations beyond the primary service area, for example, service to a distant medical facility; • Significant deadhead time, related to service area size and long-distance trips; • Service policies and scheduling practices that facilitate individualized trip-making (“one-to- one” trips rather than “few-to-one” or “many-to-one” trips); • Limited dispatch control that lacks the tools to manage service operations and respond to changes on a real-time basis; • High rates of no-shows and late cancellations; • Scheduled vehicle-hours that are not aligned with ridership demand; and • Less-than-reliable vehicles, with excess breakdowns, requiring replacement service to be deployed. Operating Cost per Vehicle-Hour Operating cost per hour is a key cost efficiency measure, assessing the financial resources needed to produce a unit of service, defined for this measure as an hour of service. What does it cost the DRT system to put service on the street? This measure, however, does not evaluate use of the DRT service; because of this, it should be assessed in conjunction with the measure passenger trips per vehicle-hour or other ridership use measures. Similarly to the productivity measure, practices vary as to whether the mea- sure uses revenue-hours or vehicle-hours in the denominator. Since the produc- tivity measure has used vehicle-hours, this measure also used vehicle-hours. Performance Considerations The elements in this measure are (1) the DRT operating costs, with the major components of costs related to staff labor and vehicle operations and their maintenance and (2) the amount of DRT service provided as measured by vehicle-hours. For a rural DRT sys- tem, deadhead time may have a significant impact on operating costs if large amounts of time are needed to travel to and from pick-up and drop-off locations at the start and end of passen- ger service. There are various reasons that a DRT system’s performance on operating cost per vehicle-hour may not meet objectives, including • Costs for labor, particularly vehicle operators, since these compose the largest share of tran- sit staff; • Costs for maintenance from an older fleet, from problem vehicles, from accidents, and from fuel costs; • High costs for administration or other overhead; • High proportion of paid hours for vehicle operators related to vehicle-hours of service— that is, a significant proportion of operator pay–hours that is attributed for cost items other than providing passenger service (e.g., administrative efforts); • High fringe benefits costs for items such as vacation, sick leave, family leave, and med- ical benefits; Performance Measures for Rural DRT 25 operating cost per vehicle-hour = total operating cost ÷ total vehicle-hours

• Costs for significant amounts of deadhead time because of service-area size and/or long- distance trips; and • Inefficient number of vehicle-hours, resulting from a poor service design or from schedul- ing practices. Labor is a major cost center. For the transit industry in general, labor including fringe bene- fits may account for up to 70% to 80% of total operating costs, with the majority of employees working in vehicle operations and vehicle maintenance. The labor rates paid to vehicle opera- tors and mechanics are somewhat controllable, but will depend on the local job market and wages paid for similar positions at competing organizations. For some DRT systems, the rates may be influenced by a labor contract. Maintenance is an important functional cost center. Based on NTD data for the transit indus- try in general, vehicle maintenance may account for up to 20% of operating expenses. DRT man- agement has some control over this factor, but costs will also depend on the type of vehicles, their age, and the vehicles’ operating conditions—the latter of which is influenced by service-area characteristics and weather. Operating Cost per Vehicle-Mile Operating cost per mile is another service efficiency measure often used for performance assess- ments, either in addition to or instead of operating cost per hour. While cost per hour is often the more important measure because the largest proportion of costs (wages and salaries) is paid on an hourly basis, the measure operating cost per vehicle-mile is included for rural DRT sys- tems since rural systems with limited data reporting practices are more likely to report vehicle-mile data than vehicle-hour data. As a cost efficiency measure, operating cost per vehicle-mile assesses the financial resources needed for the rural system to produce “vehicle-miles.” Similarly to the related measure, operating cost per vehicle-hour, this mea- sure does not evaluate the use of those vehicle-miles, so the measure should be assessed along with measures of DRT utilization. Performance Considerations Factors that influence the operating cost per vehicle-mile measure for DRT include the oper- ating costs as well as number of miles operated, which is influenced by the average speed of ser- vice and deadhead requirements, among other factors. Reasons that a rural DRT system may have a relatively high operating cost per vehicle-mile include some of the same as listed above for the measure operating cost per vehicle-hour: • Relatively high operating costs stemming from high costs for labor, maintenance, and/or administration; • Costs for significant amount of deadhead miles because of service-area size and/or long- distance trips; and • Low average operating speed, which could result from a number of factors, including excess dwell times at riders’ pick-up and drop-off locations or other factors which slow down service—for example, weather-related factors such as fog or poor road conditions. While the primary cost factor is the hourly operating cost, the measure cost per vehicle- mile is impacted because the costs are spread over a smaller number of miles. For urban DRT systems, high operating cost per vehicle-mile may result in part from excess traffic congestion, which slows down average operating speed, but this is less a factor in rural areas. 26 Guidebook for Rural Demand-Response Transportation: Measuring, Assessing, and Improving Performance operating cost per vehicle-mile = total operating cost ÷ total vehicle-hours

Operating Cost per Passenger Trip Operating cost per passenger trip is a critical cost-effectiveness measure. It combines elements of the first two measures—operating cost per vehicle-hour and passenger trips per vehicle-hour, relating productivity to the hourly oper- ating cost. As a composite measure, a DRT system may have low operating costs but if productivity is also low, the operating cost per passenger trip may be relatively high. Conversely, a DRT system may have a relatively high cost on a vehicle- hour basis, but if its productivity is high, the cost per passenger trip may be low. Performance Considerations A key element of this measure is productivity. Efforts to improve the cost per passenger trip measure should first focus on increasing the number of passenger trips served within given resources. Reasons that a DRT system might show high operating cost per passenger trip include: • High operating costs: – Costs for labor, particularly vehicle operators; – Costs for maintenance from an older fleet, from problem vehicles, from accidents, and from fuel costs, and – High administrative costs. • Low productivity: – Large service area where passenger trips are lengthy; – Low density of passengers within the service area; – System policies that allow riders to travel to destinations beyond the primary service area; – Significant deadhead time related to service-area size and long-distance trips; – Service policies and scheduling practices that facilitate individualized trip-making (“one-to-one” trips rather than “few-to-one” or “many-to-one” trips); – Limited dispatch control that lacks the tools to manage service operations and respond to changes on a real-time basis; – High rates of no-shows and late cancellations; and – Scheduled vehicle-hours that are not aligned with ridership demand. Safety Incidents per 100,000 Vehicle-Miles Safety needs to be a primary concern for all transit systems, including DRT. Rural DRT sys- tems should track and monitor their safety record and make adjustments as needed to ensure safe operations. As a performance measure, the safety incident rate can be seen as one that incor- porates an assessment of both service operations as well as passenger service quality. The safety of the DRT system may not be an attribute that passengers consider each day that they ride the DRT system, but safety is a dimension of customer service quality. Calculation Given the different ways the DRT systems define and measure safety and their accident rates, it was determined that the Guidebook use the NTD definitions to assess safety. As described in Chapter 3, these are very specific definitions. The performance measure uses the sum of NTD safety incidents, which is a required Rural NTD data element, divided by 100,000 vehicle-miles. The measure compares the raw number of Performance Measures for Rural DRT 27 operating cost per passenger trip = total operating cost ÷ total passenger trips

28 Guidebook for Rural Demand-Response Transportation: Measuring, Assessing, and Improving Performance NTD safety incidents with the miles traveled by the system, which places the raw number into the perspective of miles traveled by the system. However, since the reporting thresholds for NTD safety incidents are rela- tively high (e.g., for a property damage incident, the reporting threshold is $25,000 worth of damage), rural systems should monitor safety incidents of all types and distinguish between preventable and non-preventable accidents, without regard to a pre-determined dollar threshold. When assessing this more detailed safety data, comparisons over time for the individual rural system will likely be more meaningful than comparisons to peer systems since individual system definitions of accidents and preventable versus non-preventable vary. Performance Considerations A DRT system’s performance on safety can be improved by ensuring that vehicle operators are well trained, vehicles are well maintained, and operating policies and procedures support safe operations day to day. Lower than expected or desired performance on safety may result from a variety of reasons: • Limited vehicle operator training and/or retraining; • Inexperienced vehicle operators; • Vehicle issues such as the vehicle type or design and their condition; • Scheduling practices that result in a system speed that forces vehicle operators to rush, which then increases opportunities for accidents; • Environmental factors such as bad weather; and • The system’s commitment to safety and efforts to communicate that commitment to all its employees. On-Time Performance On-time performance is an important measure of service quality from a DRT rider’s perspective. On-time performance measures the reliability of the system: does the vehicle arrive for the pick-up when it was promised? While this measure may get more attention at urban DRT systems that operate ADA paratransit, it is important for all DRT systems. Rural systems should routinely monitor and assess their on-time performance. On-time performance may also be important at the drop-off end. In fact, timeliness at the destination end may be more important for riders with time- sensitive trips such as to work or medical appointments. DRT systems should consider assessing on-time performance at the drop-off end for time-sensitive trips, those with a pre-determined “appointment” time. This would be a separate assessment since only those trips with an appointment time would be included for this assessment. However, even if a DRT system sched- ules a rider’s trip to ensure timeliness at the destination, the system needs to give the rider a pick-up time (or time window) so that the rider can be ready when the vehicle arrives. As discussed in the last chapter, the definition of “on-time trips” varies among DRT systems. Data collection also varies although most rural systems use vehicle operator-reported data from operators’ manifests. Calculation On-time performance can be calculated based on data for all trips (which may require more data processing time unless the system has MDTs) or on a sample of trips. For a rural system that safety incidents per 100,000 vehicle- miles = [(NTD reportable safety incidents) ÷ (total vehicle-miles)] x 100,000 on-time performance = (total on-time trips, including no-shows) ÷ (total completed trips + no-shows + missed trips)

provides general public or specialized service, calculation of the measure on a sampled basis, for example, based on one week during the month or even on one sample day in the month is ade- quate (6). The sample day or sample week should be chosen randomly to avoid bias in the results. To calculate the measure, the following data elements are needed for the time period being addressed, a full month or a sampled day or week: the number of trips on-time (based on how- ever the rural system defines “on-time”) and the total number of completed trips, plus no-shows (assuming those trips have arrived on-time!) as well as missed trips, should there be any. (A trip labeled as a no-show that in fact was a late trip where the rider did not travel should be classified as a missed trip, rather than a no-show.) Regarding the assessment of on-time performance in relation to no-shows, rural DRT systems should make efforts to ensure that no-shows are in fact “legitimate” no-shows—that is, the vehi- cle operators have arrived on-time for the scheduled pick-up even though the rider does not show. There are cases where a vehicle operator may claim that a rider was a no-show, but the operator was not at the rider’s pick-up location when the claim is made. DRT systems might con- sider procedures to ensure that vehicle operators wait at scheduled pick-up locations for the pre- scribed waiting period, such as having operators contact dispatch at arrivals, and that dispatch try and contact riders when they do not appear for a trip to avoid no-show trips. If a DRT sys- tem has AVL technology, dispatch can check on a vehicle’s location to verify an operator’s where- abouts if there are questions related to no-shows. Performance Considerations DRT systems can look to a number of factors that can impact on-time performance, including • Vehicle operator schedules that are not adequately prepared or that overbook trips so that vehicle operators cannot maintain the schedule; • Incorrect information on schedules so that vehicle operators do not have the proper infor- mation for timely service (bad addresses, lack of details on just where to pick up the pas- senger such as a back door, a side street, etc.); • Staffing issues such as no back-up operators (such back-up vehicle operators are often referred to as the “extra board”), inexperience, or an inadequate number of operators; • Vehicle breakdowns or road calls resulting from vehicle design issues or maintenance prac- tices that do not keep vehicles in good working order; • Limited dispatch practices to make real-time changes to tackle service problems and help vehicle operators who are running late; • Passengers’ habits (e.g., excessive dwell time because passengers are not ready to board upon vehicle arrival, use of wrong mobility aide, etc.). 4.2 Additional Performance Measures In addition to the six performance measures discussed in the preceding section, there are others that are used by DRT systems to monitor their service. Some of these can be thought of as interim measures in that they assess a specific aspect of DRT service that affects effi- ciency, effectiveness, or service quality. No-Show/Late Cancellation Rate Generally the no-show and late cancellation rate measures the percent of scheduled trips that are not completed due to passenger no-shows and late cancellations. The Guidebook recommends that this be calculated as the sum Performance Measures for Rural DRT 29 no-show/late cancellation rate = (total no-shows + total late cancellations) ÷ total number of scheduled trips

of passenger no-shows and late cancelled trips (with late cancellations defined differently by DRT systems) divided by the total number of scheduled trips. The denominator—number of sched- uled trips—is the total of the trips that are placed onto vehicle schedules for service, as defined in Chapter 3. It is noted that the no-show/late cancel rate can be considered an interim measure, monitored because of the important affect that no-shows and late cancellations have on productivity and operating costs. They are combined together for performance measurement purposes as they have a similar negative impact on DRT operations: for most DRT systems, they represent lost resources with adverse impacts on productivity. In an effort to minimize the negative impacts of no-shows and late cancellations, most DRT systems have adopted policies addressing no-shows and late cancellations. There is considerable variation among these policies, but broadly they all establish penalties for passengers who repeat- edly cancel their trips with little notice or fail to appear for their scheduled trips. Implementa- tion and enforcement of such policies can significantly reduce the occurrences of no-shows and late cancellations; this is discussed in detail in Chapter 7. Complaint Rate In addition to monitoring and responding to complaints, some DRT systems measure and report their rate of complaints by comparing the number of complaints received to service pro- vided, such as total service complaints per 1,000 passenger trips. The denominator may be total passenger trips completed or it may be total trips scheduled. Rather than pas- senger trips, some DRT systems compare complaints to revenue-hours of service provided. The complaint rate can be monitored over time as an indicator of customer satisfaction. It is important that DRT systems maintain a consistently defined measure so that trends and comparisons from month-to-month or year-to- year are meaningful over time. If the calculation method is modified, it should be clearly noted on any trend line comparison to ensure proper assessment. Some systems have established a standard related to complaints—for exam- ple, the DRT system should have no more than x complaints per 1,000 passenger trips. Such a standard may be included in a contract document for a contracted DRT operator, with associated incentives and liquidated damages. Average Passenger Trip Length The size of the DRT system service area, distribution of riders’ origins and destinations, and degree of shared riding will affect the average pas- senger trip length. This can be a useful measure for a DRT system to mon- itor as it has an important affect on system productivity, with longer trip lengths having a negative affect on productivity. The average trip length can be measured on a sampling basis over time, and any changes assessed to monitor the impact of trip length on service operations and especially on productivity. In particular, rural DRT systems that serve more than an insignificant number of longer dis- tance trips may want to routinely collect and report average trip length data to monitor trends over time and to compare any changes in related performance measures such as productivity. Additionally, if a rural system collects average trip length data by type of trip and by trip sponsor 30 Guidebook for Rural Demand-Response Transportation: Measuring, Assessing, and Improving Performance complaints per 1,000 passenger trips = (total valid complaints ÷ total passenger trips) x 1,000 or complaints per 1,000 revenue- hours = (total valid complaints ÷ total revenue-hours) x 1,000 average passenger trip length = total passenger miles ÷ total number of passenger trips

(e.g., non-emergency Medicaid trips), the system can better understand the performance impacts of those specific trips on its systemwide performance. Average Travel Time Average travel time is computed as the sum of all passengers’ travel times divided by the total number of passenger trips. This is not a measure that is routinely reported by DRT systems, but it is useful, indicating both the degree of shared riding and service quality for the passengers. One of the premises of DRT is the grouping of passengers with similar trip patterns—ride sharing—to maximize productivity. If passengers’ travel times are short comparable with travel by private vehicle, it indicates that the scheduling function has not achieved much ride-sharing. On the other hand, if many passengers’ travel times are long, it may indicate too much ride-sharing and passengers may be overly inconvenienced with long on-board times to reach their destinations. Balancing ride-sharing with passenger travel times is a key objective of the scheduling function. 4.3 Transit-Impact Performance Measures Chapter 3 introduced transit-impact-related performance data in addition to the more stan- dard DRT data elements such as passenger trips and vehicle-hours and -miles. These less tradi- tional data elements (e.g., the number of seniors transported to congregate meal sites) can be used for transit-impact performance measures. Since rural transit often serves predominately those who are dependent on transit, a rural DRT system may want to capture its impact in improving the quality of life for those rural residents who rely on DRT service. The performance measures that assess the more qualitative aspects of rural DRT service and that might be appropriate for a particular system will depend upon that rural system’s mission and what it wants to measure. Passenger survey data can also be used, providing qualitative infor- mation from the riders. Comments and testimonials from riders can also be useful when service is evaluated, particularly for local elected leaders who may be weighing funding decisions. Addi- tionally, another way to assess the rural transit system’s “performance” in the community is to monitor press coverage. News media coverage can influence (and be indicative of) public percep- tions of the transit system. Monitoring positive/neutral and negative press coverage can be one way of assessing the system’s marketing efforts and perceived quality of service on a very general level. Assessment of the less traditional measures would be done over time to evaluate how the rural DRT system was “performing” in the areas of interest. There are many different measures that might be used, depending on the areas of interest for a rural DRT system. A small sample of pos- sibilities includes • Rate of use by seniors (total trips by seniors divided by total trips)—data obtained from schedulers and operators’ manifests. • Rate of use by people with disabilities—data obtained from schedulers and operators’ manifests. • Percentage of trips to/from congregate meal sites to total trips—data obtained from sched- ulers and operators’ manifests. • Percentage medical trips (total trips to/from medical facilities divided by total trips)—data obtained from schedulers and operators’ manifests. • Number of employment trips provided per day—data obtained from schedulers and oper- ators’ manifests. The FTA Section 5316 Program (Job Access and Reverse Commute) Performance Measures for Rural DRT 31 average travel time = total passengers’ travel time ÷ total number of passenger trips

requires that the actual or estimated number of jobs that can be accessed be reported on an annual basis. Employment trips per day on the DRT service is an indicator of how many jobs are being accessed on an ongoing basis on this service, which is, in turn, indicative of the importance of the transportation service to the local economy. For purposes of this measure, a round trip to and from an employment site would be counted as one job accessed. • Number of individuals using DRT for independent living (e.g., individuals using service to access medical services, counseling, education, employment, grocery shopping, personal business such as banking, and other basic life needs—essentially the customer base of rural DRT service—can be an important measure in demonstrating the value of the service in community). Total unduplicated individuals served per month is a suggested measure— with data obtained from schedulers. This could also be measured on a per-capita basis (i.e., percent of population relying on rural DRT on a regular basis). • Passenger feedback on service quality—for example, ratings on items such as service relia- bility, timeliness of service, helpfulness of scheduling staff, drivers—data obtained from passenger survey data as well as individual compliments and complaints received on an ongoing basis. 32 Guidebook for Rural Demand-Response Transportation: Measuring, Assessing, and Improving Performance