Transit Service Evaluation Standards (2019)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

8 This chapter summarizes findings from a literature review related to transit service evalu­ ation standards. To aid the literature review, a TRID search of the following keyword phrases was conducted: “performance standards,” “performance evaluation,” “performance indicators,” and “service evaluation” with the filter “public transportation.” The organization of the litera­ ture review is as follows: • Early history, • TCRP reports, • Summary by metrics used and mode, and • Additional research. Early History Section 15 of the National Mass Transportation Act of 1974 required UMTA to develop a National Urban Transportation Reporting System. UMTA contracted with the University of Pennsylvania to assist the agency in implementing Section 15. The subsequent report (Vuchic et al. 1976) was based on the concept that sufficient data should be collected on a systematic and permanent basis to allow various types of reviews, analyses, and evaluations of urban transporta­ tion. The report sought a reasonable compromise between an ideal set of data (extremely com­ prehensive but very difficult and costly to collect) and a very limited set of data (easy to collect but inadequate for many of its intended uses). Extensive recommendations that considered the availability and reliability of various data were presented in detail for both transit and highway, reflecting the vision of an “urban transportation reporting system.” UMTA used this report to develop its Section 15 data requirements for transit agencies and monitored transit agencies closely during the first few years of data collection to ensure that data definitions were under­ stood and applied uniformly so that the data collected were truly comparable across agencies. The Vuchic study, while intended to develop a transit data collection system, was attuned to the usefulness of proposed measures and made several recommendations regarding the most appropriate measures (Vuchic et al. 1976). Vuchic et al. argued that the true product of a tran­ sit system was usable passenger space, which translates to capacity miles (seats plus standing spaces) as opposed to vehicle miles (not a measure of space) or seat miles (penalizes systems with heavy volumes, short trips, and/or few seats). They also stressed that equity in funding allocation should not be measured on a per capita basis but rather on the basis of ridership, passenger miles, service levels, or utilization, because different cities have different propensities to use transit. G. J. “Pete” Fielding was among the pioneers in transit performance measurement in the 1970s. In a seminal 1978 report, Fielding identified nine performance measures (three related C H A P T E R 2 Literature Review

Literature Review 9 to efficiency, four to effectiveness, and two overall measures) and applied these to 46 California transit agencies (Fielding et al. 1978). The report stressed that the measures should not be viewed as final, noting that unavailability or unreliability of data caused omission of several useful measures among the 21 originally considered. Issues with the comparability of different transit agencies owing to political and geographic factors outside the control of transit managers were acknowledged (researchers continue to wrestle with these today). The report anticipated development of uniform performance indicators on the basis of commonly defined statistics that would create a language for evaluating the effects of new programs and technologies and disseminate findings. Fielding et al. grouped the nine performance measures used in the study into three categories: efficiency, effectiveness, and overall indicators, as shown in Table 2. Fielding noted that many potentially valuable metrics were not collected on a regular basis, including passenger miles and employee hours. Both Vuchic and Fielding emphasized that passenger miles was a much more valuable measure of transit service consumed than passengers because it takes distance traveled into account. Unlike Vuchic, Fielding considered seat miles (number of seats * revenue miles) as the appropriate measure of transit service provided. There had been previous efforts to specify service standards and measurement techniques (National Committee on Urban Transportation 1958; Tomazinis 1975), but the availability of reliable transit data for all transit agencies in the country spurred a wave of quantitative analy­ sis of transit performance. Many of the early analyses were undertaken by state departments of transportation (DOTs). Vuchic’s study was based on previous work with the Pennsylvania DOT, which continued to develop and refine a transit performance measurement system (Allen and DiCesare 1976; Booz­Allen and Hamilton, Inc. 1981) which it extended to include small and rural transit systems (Carter, Goble, Roberts, Inc. 1982). The California DOT developed and applied transit performance measures (Fuller 1978) and served as a case study for linking these with transit funding (Drosdat 1977). The Florida, Michigan, New York, and Washington State DOTs developed performance evaluation systems for transit agencies within their respec­ tive states in the late 1970s and early 1980s (Anderson 1981; Holec and Schwager 1981; Kelley & Rutherford 1983; New York State DOT 1982; Peat, Marwick, Mitchell and Company 1980; Post, Buckley, Schuh & Jernigan 1979; Zerrillo 1981). The Texas A&M Transportation Insti­ tute developed a level­of­service evaluation approach that used similar indicators (Bullard and Christiansen 1981). Transit agencies were not passive bystanders but began to develop their own standards. The Seattle Metro (METRO Transit Planning Division 1977) and Toronto Transit Commis­ sion (Munro and Biemiller 1977) published service evaluation criteria in 1977. An assess­ ment of performance indicators in the United States included case studies of the Washington Metropolitan Area Transit Authority and the Santa Clara County Transit District (Peat, Marwick, Mitchell and Company 1978). The Tidewater Transportation District Commission (TTDC) in Norfolk, Virginia, was another early adopter of performance evaluation (Becker Efficiency Effectiveness Overall Indicators Revenue vehicle hours per employee Revenue passengers per service area population Operating expense per total passengerRevenue vehicle hours per vehicle Percentage of population served (within 1/4 mile) Operating expense per revenue vehicle hour Total passengers per vehicle Operating expense per revenue passenger Revenue passengers per vehicle Source: Fielding et al. 1978, pp. 4–8. Table 2. Performance measures used in analysis by Fielding et al.

10 Transit Service Evaluation Standards et al. 1981; TTDC 1979), along with the Montgomery (Alabama) Area Transit System (Dorfman 1979), Central Ohio Transit Authority in Columbus, Ohio (Bowles 1979), Dallas Transit System (American Public Transportation Association 1979), Southwest Ohio Regional Transit Authority (1979) in Cincinnati, and the Berkshire Regional Transit Authority in Pittsfield, Massachusetts (Cook 1979; Flusberg et al. 1979). UMTA provided funding for most of these local studies in service evaluation. The state of the art in transit performance evaluation at this time was summarized in a UMTA­funded review of bus service evaluation procedures (Attanucci et al. 1979). The report, which included an extensive bibliography of work done to date, presented the results of a survey of 71 transit properties in the United States and Canada. The focus of the study was to identify service performance indicators and criteria used to evaluate bus service on a route­by­route basis. Table 3 describes the indicators in each of three categories. Indicators in bold were mentioned by at least 20 survey respondents in the Attanucci study, and indica­ tors in italics were mentioned by at least 10 respondents. For per­mile and per­hour indica­ tors, revenue miles and hours were implied but not clearly stated in all cases. Loading standards were generally higher during peak periods (sometimes even higher in peak hour, peak 30 minutes or peak 20 minutes). Service availability and route spacing criteria varied by residential density and were often expressed in terms of the percentage of popula­ tion or households within one­quarter mile of a transit route. Headways could be system wide (e.g., all routes operate every 30 minutes during peak periods and every 60 minutes at other times) or route specific (depending on ridership and population density). Some agencies followed a policy of not operating any route with a headway greater than 60 minutes. Direct­ ness of service considered bus mileage relative to the most direct path between route termi­ nals, percentage of transfers and opportunities for through­routing, and person­minutes of travel relative to the number of passengers served by a route deviation. Agencies used different definitions of on­time performance to measure schedule adherence. The most common definition was zero to five minutes late, with a requirement that a certain percentage of trips (typically 80% to 90%) must meet this criterion. Revenue per cost indicators varied from 10% to 50%. Passengers per revenue mile indicators ranged from 1.0 to 2.5, with variation by time of day. Passengers per revenue hour indicators varied by type of service and time of day, and were expressed either as values (e.g., 33 for urban routes, 20 for suburban) or as a percentage of the system value (e.g., 50% of the overall passen­ gers per revenue hour). Service Design Operating Performance Economic/Productivity Indicators Loading standard Schedule adherence Revenue/cost (farebox recovery ratio) Service availability/route spacing Complaints Passengers per mile Policy headways Lost runs/missed trips Passengers per hour Directness of service Travel speed Subsidy per passenger Bus stop spacing Accidents Revenue and cost per mile Passenger shelters Miles/trouble call Ridership trend New service design Average fare per passenger Exclusive bus lane Source: Attanucci et al. 1979, Tables 2, 4, 5, and 6. Note: Indicators in bold were mentioned by at least 20 survey respondents. Indicators in italic were mentioned by at least 10 respondents. Table 3. Performance evaluation indicators reported by Attanucci et al.

Literature Review 11 The Attanucci report concluded that many transit agencies of various sizes have made com­ mitments to move toward a systematic evaluation effort but that very few had achieved this goal fully. The conclusions also noted that data collection was a major issue, that criteria tended to be used as a screening tool to identify where more rigorous analysis was needed, and that each agency tailored its evaluation program to its own operating environment. In the 1980s, additional studies by UMTA, states, and transit agencies (Allen and Grimm 1980; Delaware Authority for Regional Transit and UMTA 1984; Greater Bridgeport Transit District 1984; Kocur 1982; Wilbur Smith and Associates 1989) developed and applied transit performance evaluation systems. These studies found that issues with continuing data inconsis­ tency, equity, and political feasibility affect the application of performance evaluation in transit. On the basis of her analysis of 30 transit systems in California, Giuliano (1981) was among the first to note that peer performance analysis may not be appropriate, owing to differing oper­ ating environments. A theoretical examination of transit efficiency and effectiveness concluded that transit systems should be evaluated from the perspective of the agencies rather than that of government goals such as maximizing social well­being, economic development, and envi­ ronmental quality (Talley and Anderson 1981). Sinha and Guenthner (1981) surveyed 19 bus transit operators in the five­state area of Illinois, Indiana, Michigan, Ohio, and Wisconsin to assess the adequacy of performance indicators. Several published articles analyzed the choice of indicators and the development of a resulting evaluation procedure. Debo (1981) suggested both a strategy for establishing appropriate and attainable service standards and also criteria for deciding which services should be retained and which should be cut back. Everett (1978) differentiated between service indicators (reliability, frequency of service, speed of travel, and ease of transferring) and performance indicators (per­ centage of population served, efficiency, effectiveness, and productivity). He notes that the two categories cannot be cleanly separated and the possibility for conflict between the two (equity versus efficiency, for example). Dajani and Gilbert (1978) summarized the need for the devel­ opment of transit performance measures and highlighted research needs with respect to data collection requirements, cross­jurisdictional comparability, and the utility of the proposed per­ formance measures for decision­making. Goble (1983) was among the first to note the interest in performance evaluation among small and rural systems; most of the surveys of transit opera­ tors in connection with performance standards focused on large and midsize transit systems. Houston Metro conducted a review of transit industry service evaluation methods in 1984 as part of its efforts to develop more rational approaches for improving system efficiency and effectiveness (Metropolitan Transportation Authority of Harris County and UMTA 1984). This review categorized performance indicators in a different way from the study by Attanucci and colleagues in 1979 and included a summary of criteria developed by transit agencies. Table 4 presents findings from this industry review. Indicators in bold were mentioned by at least 50% of survey respondents, and indicators in italics were mentioned by at least 33%. Vehicle loading, headway, and schedule adherence indicators were used by more than half of responding agencies. One of the important questions that emerged as transit agencies and states developed perfor­ mance evaluation systems was: which indicators are most useful in measuring transit perfor­ mance? Several papers examined this issue in the 1980s and early 1990s. Fielding continued to explore issues in transit performance evaluation in several published reports using the newly available Section 15 data (Fielding and Anderson 1983; Fielding et al. 1985). Fielding and Hanson (1988) developed and utilized the Irvine Performance Evaluation Method (IPEM) to identify common factors underlying performance of high­efficiency transit agencies. The IPEM uses nine performance indicators to assess overall system performance (Table 5).

12 Transit Service Evaluation Standards Indicatora Typical Criteria Route Design Bus stop spacing Differ by service, density, land use Local urban: 400 feet to ¼ mile Route coverage Differ by service and density Route spacing: ¼ mile to 1–2 miles Accessibility local urban: 85%–95% within ¼ mile Route deviation Time/distance/number of limits (5–8 minutes, 1 mile/route, 2/route) Maximum 120%–140% of direct distance/200% auto travel time Route length Maximum round-trip travel time varies related to headways: 3–4 hours (large), 2 hours (medium), 75 minutes (small) Route duplication Overlapping routes only near major activity centers Route structure Limits on number of branches/turnbacks Service Quality Vehicle loads Local: 125%–150% peak Lower for shuttles/feeders Higher for express 100% base Vehicle headways Maximum 15 peak/30 base largest agencies, 30/60 others Schedule adherence 0 early to 5 minutes late 80%–90% peak trips 90%–100% base trips Passenger safety Preventable: 0.7–3.0 per 100,000 vehicle miles Total accidents: 4.0–8.0 Passenger accidents: 6.0–10.0 Bus shelter placement Shelter: 65–100 daily boardings OR transfer location/heavy use by specialized clientele Bench: 50 daily boardings Passenger transfers Maximum number of transfers: 1–3 Maximum transfer rate: 20%–30% Missed trips 5%–10% of total trips Span of service Varies by system size and day: Large: 5–3 a.m. weekdays Small: 5 a.m. to 7–9 p.m. weekdays Customer complaints Varies per driver/day/month Economic/Productivity Passengers per vehicle hour Varies by service type/time of day—3 types of measures: Minimum percentage of system average (50%) Minimum standard (20–25 for local) Above lowest quartile of routes Cost recovery 50%–95% of system average Minimum 15%–30% Above lowest quartile of routes Passengers per vehicle mile 60%–80% of system/group average Minimum 1.0–2.5 Passengers per trip Minimum 80% of system average Minimum 5–15 Above lowest quartile of routes Cost per passenger 75% of system average Maximum $1.25–$2.00/rider Above lowest quartile of routes Source: Metropolitan Transportation Authority of Harris County and UMTA (1984), pp. 6–31. aIndicators in bold were mentioned by at least 50% of survey respondents; indicators in italics were mentioned by at least 33%. Table 4. Performance indicators and criteria from Houston Metro review (1984).

Literature Review 13 These groundbreaking studies in the early days of transit performance evaluation set the stage for further developments in service evaluation. The next section summarizes TCRP studies that addressed transit performance evaluation. TCRP Reports Related to Performance Evaluation TCRP Synthesis of Transit Practice 10 (Benn 1995) updated the 1984 Houston Metro report regarding bus route evaluation standards. The report noted that standards have evolved gradu­ ally, typically becoming more discrete, and that this trend was expected to continue as route­ level data become more available. Table 6 presents performance indicators and criteria used by at least one­third of all responding agencies unless otherwise noted. This report focused on indicators used and changes since the 1984 Houston Metro report, with less information on specific evaluation criteria. The report noted an increased focus on customer orientation and satisfaction as standards have evolved and anticipated evaluation standards to address paratransit services in the near future. With increasing use of technology, evaluations will take place more often and real­time information at the bus stop level will be available for passengers. In the mid­1990s and early 2000s, the explosion of data availability through new tech­ nologies enabled the development of new analytical techniques applied at more disaggregate levels. TCRP Report 88: A Guidebook for Developing a Transit Performance-Measurement System (Kittelson & Associates Inc. 2003a) provided a step­by­step guide in developing a performance evaluation system that includes both traditional and nontraditional performance indicators to address customer­oriented and community issues. The guidebook provides an eight­step process for implementing or updating a performance measurement program as well as a core set of suggested performance measures by agency size. As automatic vehicle location (AVL) and automatic passenger counter (APC) systems became increasingly available and affordable for transit agencies, the proliferation of data threatened to outstrip the capacity of agencies to analyze the data. TCRP Report 113 (Furth et al. 2006) provided guidance for the effective collection and use of archived AVL and APC data to improve the performance and management of transit systems. Early research in the field of transit performance evaluation noted that the selection of peer agencies for comparisons of transit performance could produce misleading results if not done with care. TCRP Report 141 (Ryus et al. 2010) provides important resources in using Indicator Measurement Revenue vehicle hours per operating expenseCost efficiency Service effectiveness Passenger boardings per revenue vehicle hour Cost effectiveness Corrected operating revenue Labor efficiency Total vehicle hours per total employees Vehicle efficiency Total vehicle miles per peak vehicle Maintenance efficiency Total vehicle miles per maintenance employee Total vehicle miles per maintenance expense Safety effectiveness Total vehicle miles per collision accident Total vehicle miles per collision insurance expense Source: Fielding and Hanson 1988, Table 1. Table 5. Irvine performance evaluation method.

14 Transit Service Evaluation Standards Indicator Typical Criteria Route Design Bus stop spacing 6 to 8 stops per mile Population density Not specified Route coverage 0.5–1.0 mile, depending on density Employment density Not specified Route directness Maximum deviation 5–8 minutes or 1 mile per route Route distance no more than 20%–40% greater than auto Route travel time no more than twice auto Route structure Limits on number of branches Schedule Design Maximum number of standees 20 standees or 150% of capacity as examples; limit on duration of standee time Maximum intervals 15 peak/30 base; 30/60 or 60 minutes as examples Peak versus off-peak periods Different standards Timed transfers Used more in smaller systems (possibly with less frequency) Clockface schedules Used more in smaller systems Span of service Split on whether same hours are operated for all routes versus longer span for busiest routes Economic/Productivity Standards Passengers per revenue hour Passengers per revenue mile Passengers per trip Not specified Fixed minimum standard or percentage of system average or bottom 10%–20% of routes Passengers per trip may be simplest to explain Cost or subsidy per passenger 2–3 times system average typical for subsidy per passenger Route-level minimum variable cost recovery ratio Below 25% most common 25%–40% next most common Above 40% rare Service Delivery Standards On-time performance 90%–94% peak/94%–97% off-peak most common Headway adherence Evenness of interval between buses; generally for internal use Passenger Comfort and Safety Standards Passenger complaints Per mile or per passenger boarding Missed trips/unscheduled extras Not specified Accidents Not specified Passenger environment Cleanliness; vehicle condition; destination signs Special information in areas with safety concerns Rare; request a stop at night; different stops in inclement weather Source: Benn 1995, pp. 9–22. Table 6. Performance indicators and criteria from TCRP Synthesis 10.

Literature Review 15 performance measurement and benchmarking as tools to identify the strengths and weak­ nesses of their organization, set goals or performance targets, and identify best practices to improve per formance. The research developed and tested a methodology for performance measurement and peer comparison for (a) all fixed­route components of a public transit system, (b) the motor bus mode specifically, and (c) major rail modes specifically (i.e., light rail, heavy rail, and commuter rail). This report complements TCRP Report 88 (Kittelson and Asso­ ciates, Inc. 2003a), which described how to implement and use performance measurement on an ongoing basis at a transit agency. Finally, performance evaluation served as the basis for the Transit Capacity and Quality of Service Manual (TCQSM). The first edition was published as a web­only document in 1999 (Kittelson & Associates, Inc. 1999). The second edition was released in 2003 (Kittelson & Asso­ ciates, Inc. 2003b), and the third and most current edition was updated in 2013 (Kittelson & Associates, Inc. et al. 2013). The manual introduces a new framework for measuring transit availability and quality of service from the passenger point of view. The TCQSM met a need in the transportation profession for a consolidated set of transit capacity and quality­of­service definitions, principles, practices, and procedures for planning, designing, and operating vehi­ cles and facilities, just as the Highway Capacity Manual defines quality of service and pre­ sents fundamental information and computational techniques related to quality of service and capacity of highway facilities. A key contribution of the TCQSM was an “A” through “F” classification framework for measuring availability and quality of transit and paratransit service at the transit stop, on the route segment, and for the system. The third edition of the TCQSM was developed to reflect the new multimodal focus and pro­ cedures in the 2010 Highway Capacity Manual (TRB 2010) and the substantial research related to transit capacity and quality of service that has occurred since 2003. The third edition has sepa­ rate chapters for quality­of­service concepts and quality of service methods and adds chapters addressing operations concepts and demand­responsive transit. The next section considers the evolution of performance metrics and criteria from the 1970s to today and references service standards provided by transit agencies included in the sample. Evolution of Performance Metrics and Criteria Tables 7 through 14 summarize the evolution of performance metrics and criteria from the studies cited previously through the present. Twenty­three agencies provided current perfor­ mance standards and guidelines for this synthesis report. Additional Research This section summarizes specific research studies related to aspects of transit performance evaluation. As can readily be seen, research on this topic continues today. Several studies emphasized that no single performance indicator should be relied upon exclusively to assess overall performance (Hensher 1991; Hodge and Orrell 1995; Holec 1985). Despite this caution regarding the use of a single measure of performance, the literature reflected a strong desire to develop a composite index that combined individual performance indica­ tors into a single number (Galindez et al. 1997; Lago 1985; Murugesan and Moorthy 1998; Talley 1991). There have also been attempts to devise the ideal performance evaluation process (Giannopoulos 1989; Takyi 1993).

16 Transit Service Evaluation Standards Passengers per revenue hour — Varies by type of service and time of day; could be absolute (33 urban, 20 rural) or percentage standard (50% of system average) 1–2.5; varies by time of day Varies by service type/time of day. Three types of measures: • Minimum of system average (50%) • Minimum standard (20%–25% for local) • Above lowest quartile of routes • Minimum 80% of system avg. • Minimum 5–15 • Above lowest quartile of routes 60%–80% of system/group average Minimum 1.0–2.5 Fixed minimum standard or percentage of system average or bottom 10%–20% of routes Fixed minimum standard or percentage of system average or bottom 10%–20% of routes Fixed minimum standard or percentage of system average or bottom 10%–20% of routes 13 agencies By route type Most fixed, some percentage of system average by route type 1 agency No standard; tracked versus peers 2 agencies Improvement Greater than population growth 2 agencies Trip must have percentage of system average A factor in performance index Passengers per trip may be simplest to explain Yes Standard Metrica Fielding et al. 1978 Attanucci et al. 1979 Houston Metro Reviewb (1984) IPEMc (1988) TCRP Synthesis 10d (1995) Current Standards/ Criteriae Passengers per revenue mile — — 0 agencies Passengers per vehicle Yes — — — — 0 agencies Revenue passengers per vehicle Yes — — — — 0 agencies Passengers per service area population Yes — — — — Ridership trend — Yes — — — Passengers per trip — — — Revenue hours per operating expense — — — Yes — 0 agencies Nonfixed routes — — — — — 1 agency — — — — — 2 agencies Specific goal Passenger miles per platform hour — — — — — 1 agency Percentile aOther metrics cited by at least one agency: standards for nonfixed routes; passenger miles per platform hour. bMetropolitan Transportation Authority of Harris County and UMTA 1984. cFielding and Hanson 1988. dBenn 1995. eTwenty-three agencies provided current performance standards and guidelines. Ridership Table 7. Productivity/effectiveness metrics and standards.

Literature Review 17 Operating expense per revenue hour Yes — — — — Operating expense per revenue mile — Yes — — — 0 agencies Revenue or vehicle hours per employee Yes — — Yes — 0 agencies Revenue hours per vehicle Yes — — — — 0 agencies Operating expense per total passengers Yes — — Yes Operating expense per revenue passenger Yes — — — — 0 Subsidy per passenger — Yes — — Farebox recovery ratio — 10%–50% Minimum 15%–30% Yes Revenue per mile — Yes — — — 0 agencies Average fare per passenger — Yes — — — 0 agencies Vehicle miles per vehicle — — — Yes — 0 agencies aOthers cited by at least one agency: fare increases; municipal operating contribution per capita; ratio of actual to budget net costs; percentage of capital funds invested. bMetropolitan Transportation Authority of Harris County and UMTA 1984. cFielding and Hanson 1988. dBenn 1995. eTwenty-three agencies provided current performance standards and guidelines. Standard Metrica Fielding et al. 1978 Attanucci et al. 1979 Houston Metro Reviewb (1984) IPEMc (1988) TCRP Synthesis 10d (1995) Current Standards/ Criteriae 2 agencies Increase less than consumer price index A factor in performance index 2 agencies Percentage and actual 7 agencies Varies from 17% to 60% Typically 2 to 3 times the system average Below 25% most common May include other revenue sources25%–40% next most common Above 40% rare 75% of system average Maximum $1.25–$2.00/rider Above lowest quartile of routes Above lowest quartile of routes 50%–95% of system average Table 8. Financial/efficiency metrics and standards.

18 Transit Service Evaluation Standards Yes — — — Route spacing — — Service span — — — aOthers cited by at least one agency: service levels; express service; transit market area; revenue hours per capita; bus fleet reliability; escalator availability; elevator availability; rail structure availability. bMetropolitan Transportation Authority of Harris County and UMTA 1984. cFielding and Hanson 1988. dBenn 1995. eTwenty-three agencies provided current performance standards and guidelines. Standard Metrica Fielding et al. 1978 Attanucci et al. 1979 Houston Metro Reviewb (1984) IPEMc (1988) TCRP Synthesis 10d (1995) Current Standards/ Criteriae Percentage of population served (within ¼ mile) Varies by residential density Route spacing: ¼ mile to 1–2 miles Varies by service and density Accessibility local urban: 85%–95% within ¼ mile Varies by system size and day; on weekdays: • Large: 5 a.m.– 3 a.m. • Small: 5 a.m. to 7–9 p.m. 0.5–1.0 mile, depending on density Split on whether same hours are operated for all routes versus longer span for busiest routes 12 agencies Varies by type of service, from 12 to 18 hours Monday–Friday; usually less on weekends 9 agencies Varies by density 8 agencies ½ mile typical, varies by density Table 9. Service availability metrics and standards. Researchers proposed the extension of the concept of performance evaluation from urban fixed­route transit agencies to rural agencies and paratransit service (Bitzan and Hough 1994; Carter and Lomax 1992; Community Transportation Association 1993; Radow and Winters 1996). There was still interest in multimodal approaches and reviews of indicators and proce­ dures at peer agencies or throughout the transit industry. The Miami­Dade Transit Agency reviewed its existing service planning guidelines to determine whether more formal guidelines or standards should be adopted (Perk and Hinebaugh 1998). An early step in this study was to conduct a survey of peer transit systems across North America. Survey results uncovered passionate feelings for and against the use of strict standards among the agencies. Results also cited strategies for persuading governing bodies to adopt staff recommendations for service adjustments and processes for obtaining public input and making decisions. During this period, transit agencies were also revising and updating their service evaluation process, often based on a review of evaluation procedures at similar agencies. The San Diego Metropolitan Transit Development Board revised and updated its service management process to provide for a more objective set of procedures to evaluate existing services and to rank new ones (Williamson and Boyle 2001). Peer reviews of standards and processes at similar­sized agencies were often a first step for a transit agency in the development of its own performance evaluation system. A 2009 report identified existing best practices in transit service plan­ ning and developed a generic model approach that could be adapted and used to develop service design standards, service performance measurements, and a standard service evalua­ tion methodology (Mistretta et al. 2009).

Literature Review 19 Loading standard Varies by time of day Standard Metrica Fielding et al. 1978 Attanucci et al. 1979 Houston Metro Reviewb (1984) IPEMc (1988) TCRP Synthesis 10d (1995) Current Standards/Criteriae Policy headways Systemwide or route specific Directness of service Mileage/ percentage transfers/ deviations Bus stop spacing New service design Dedicated bus lanes — Yes Yes Yes — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — 20 standees or 150% of capacity; limit on duration of standee time Varies by time of day 15 minutes peak/30 minutes base; 30 minutes/60 minutes or 60 minutes Maximum deviation 5–8 minutes or 1 mile per route Route distance no more than 20%–40% greater than auto Route travel time no more than twice auto Limits on number of branches 6 to 8 stops per mile Yes More common in smaller systems (possibly with less frequency) More common in smaller systems 18 agencies 135% average for peak periods; 100% off peak; express 100%; time limits for standees; space per passenger 19 agencies 30 minutes peak/60 minutes off peak is common; more frequent in peak (10–15 minutes) and off peak (15–20minutes) for strongest routes 8 agencies Bus travel time/distance no more than 150%–175% of auto; 125%–133% for express/limited; Added passenger-minutes per boarding/alighting < 3, 5, or 10; Limit on terminal loop length No midroute loops 10 agencies Median 660–1,250 feet; varies by central business district/urban/ suburban 9 agencies Meet performance standards within a given time period, typically 1 year, up to 3 years 0 agencies 1 agency 3 agencies Offset schedules on common segments of routes 6 agencies Limits to number of branches/turnbacks (0–2) 3 agencies Enhanced service levels in areas with greater density 2 agencies 2 agencies 6 agencies Combines productivity and financial metrics, sometimes demographics Point system or percentage of average as standards Local: 125%–150% peak; lower for shuttles; feeders higher for express—100% base Maximum 15 minutes peak/30 minutes base largest agencies, 30 minutes/60 minutes others Maximum number of transfers: 1–3 Maximum transfer rate: 20%–30% Maximum round-trip travel time varies: 3–4 hours (large); 2 hours (medium); 75 minutes (small) related to headways Overlapping routes only near major activity centers Limits on number of branches/ turnbacks Route length Route duplication Route structure Population and employment density Timed transfers Clockface schedules Performance index Table 10. Service design metrics and standards. (continued on next page)

Standard Metrica Fielding et al. 1978 Attanucci et al. 1979 Houston Metro Reviewb (1984) IPEMc (1988) TCRP Synthesis 10d (1995) Current Standards/Criteriae — — — — — — — — — — aOthers cited by at least one agency: two-way service; route terminals; distribution of service; recovery time; connections; type of service; minimum service levels; timepoints; blocks; interlining; network; transfer time; investment priorities; reduction priorities. bMetropolitan Transportation Authority of Harris County and UMTA 1984. cFielding and Hanson 1988. dBenn 1995. eTwenty-three agencies provided current performance standards and guidelines. 4 agencies Different standards or consideration for transit-dependent areas 4 agencies: 2 prefer farside, 1 prefers nearside Pullouts on high-speed arterials Sociodemographic characteristics Stop placement Table 10. (Continued). Bus shelters — Yes — — Complaints — Yes — Lost runs/missed trips — Yes — Yes Travel speed — Yes — — — Schedule adherence — 0–5 minutes late; 80%–90% adherence — Headway adherence — — — — 3 agencies (all rail) Passenger environment — — — — 1 agency aOthers cited by at least one agency: facilities at park-and-rides; platform accessibility; vehicle accessibility; passenger comfort; heating/air conditioning on bus; trash receptacles; maximum number of connections or transfers per trip; complaint resolution rate; information at bus stops; commendations. bMetropolitan Transportation Authority of Harris County and UMTA 1984. cFielding and Hanson 1988. dBenn 1995. eTwenty-three agencies provided current performance standards and guidelines. Standard Metrica Fielding et al. 1978 Attanucci et al. 1979 Houston Metro Reviewb (1984) IPEMc (1988) TCRP Synthesis 10d (1995) Current Standards/ Criteriae 10 agencies Minimum daily boardings for shelter 20–50, for bench 10–25; fewer if certain criteria met 3 agencies Per boarding Some use “valid complaints” 5 agencies 0.5%–1.5% of all trips or <1/day 1 agency Range by type of trip/area 20 agencies 0–5 minutes late common; 65%–95%,varies by stop location/time of day/type of service Wait time or time between arrivals headway (or HW+2); 87%–90% standard Clean exterior twice/week; sweep interior daily Cleanliness; vehicle condition; destination signs Evenness of interval between buses; generally for internal use Per mile or per passenger boarding 90%–94% peak/94%–97% off-peak most common 0 minutes early to 5 minutes late; 80%–90% peak; 90%–100% base Maximum 5%–10% of total trips Varies; per driver/day/month Bench: 50 daily boardings Shelter: 65–100 daily boardings OR transfer location/heavy use by specialized clientele Table 11. Service quality metrics and standards.

Literature Review 21 — Yes — Yes 0 agencies — Yes — — — — — — Yes — 0 agencies — — — Yes — 0 agencies — — — — 0 agencies — — — — — aOthers cited by at least one agency: street network/sidewalks; mechanical failure; security equipment on buses; preventable maintenance inspections; nonpreventable bus accidents; employee preventable/nonpreventable injuries; safety meetings; mean distance between rail delays; rail passenger offloads; percentage of track with performance restrictions; red signal overturns; fire incidents on rail; crimes; crime rate; rail collisions; rail derailments; bus passenger strikes. bMetropolitan Transportation Authority of Harris County and UMTA 1984. cFielding and Hanson 1988. dBenn 1995. eTwenty-three agencies provided current performance standards and guidelines. Standard Metrica Fielding et al. 1978 Attanucci et al. 1979 Houston Metro Reviewb (1984) IPEMc (1988) TCRP Synthesis 10d (1995) Current Standards/ Criteriae Accidents per 100,000 vehicle miles traveled Vehicle miles per collision accident Vehicle miles per collision insurance expense Special information in areas with safety concerns Preventable bus accidents Revenue miles per road call Preventable accidents: 0.7–3.0 Total accidents: 4.0–8.0 Passenger accidents: 6.0–10.0 Rare; request a stop at night; different stops in inclement weather 5 agencies Minimum 4,000–20,000 6 agencies 0.25/100,000 kilometers to 2/100,000 miles Table 12. Safety metrics and standards. — — — Yes — 0 agencies — — — Yes — 0 agencies aOthers cited by at least one agency: uniform infractions, customer calls received versus recorded, lost calls; average time for abandoned calls; vehicle assignment. bMetropolitan Transportation Authority of Harris County and UMTA 1984. cFielding and Hanson 1988. dBenn 1995. eTwenty-three agencies provided current performance standards and guidelines. Standard Metrica Fielding et al. 1978 Attanucci et al. 1979 Houston Metro Reviewb (1984) IPEMc (1988) TCRP Synthesis 10d (1995) Current Standards/ Criteriae Vehicle miles per maintenance employee Vehicle miles per maintenance expense Table 13. Other metrics and standards.

22 Transit Service Evaluation Standards Passengers per revenue hour Complaints Revenue miles per road call Preventable accidents On-time performance — — — — — — — — — — — — — — — — — — — — — — — — — aOthers cited by at least one agency: farebox recovery ratio; nonpreventable accidents; commendations; hold times for where’s my ride/reservations. bMetropolitan Transportation Authority of Harris County and UMTA 1984. cFielding and Hanson 1988. dBenn 1995. eTwenty-three agencies provided current performance standards and guidelines. Standard Metrica Fielding et al. 1978 Attanucci et al. 1979 Houston Metro Reviewb (1984) IPEMc (1988) TCRP Synthesis 10d (1995) Current Standards/ Criteriae 2 agencies Maximum: 1–2/100,000 miles Goal: 0.64–0.7 2 agencies Minimum/goal: 93%/98% no more than 15 minutes late; 85%/92% within 15 minutes 2 agencies Maximum: 28,000–30,000 Goal: 35,000–45,000 2 agencies Maximum: 3.0–3.71/100,000 boardings Goal: 1.26 to +1.5 2 agencies Minimum 1.5 Goal 2.0–2.5 Table 14. Paratransit standards. The Winston–Salem (North Carolina) Mobility Manager project examined data collection, validation, and performance evaluation for Trans­AID, the local paratransit system (Stone et al. 2001). A national study of Americans with Disabilities ACT (ADA) complementary para­ transit practice addressed methods for determining quality and performance standards and for measuring all aspects of daily operations (Multisystems 2004). The North Carolina DOT revisited the issue of performance standards for urban and rural transit agencies and recom­ mended that performance­based funding not be instituted, noting that several preparatory steps were needed before implementation is considered. Another study extended performance evaluation to transportation demand management programs throughout the United States (Ungemah and Dusza 2009). The advent of bus rapid transit (BRT) introduced a new mode for consideration in performance evaluation (Behrooz and Jalali 2012; Flynn et al. 2011; Gandhi and Tiwari 2017; Wan et al. 2016). A broader multimodal study developed a handbook that can be used as a reference by transportation agencies when implementing network performance measures across modes and/or jurisdictions (Cambridge Systematics, Inc. et al. 2010). Following TCRP Report 113 (Furth et al. 2006), two studies specifically addressed service reliability and schedule adherence (Colin Buchanan and Partners 2000; Mandelzys and Hellinga 2010).

Literature Review 23 In the early 21st century, performance evaluation began to encompass measures beyond the productivity and efficiency indicators that had become standard throughout the industry. Marx (2004) was among the first authors to introduce an alternate set of performance measures that take as a premise the concept of customer service. European public transport operators, first in Paris and then in Brussels, Belgium, developed a framework incorporating customer satisfac­ tion and performance evaluation with service standards and service indicators that are customer oriented yet objectively measurable (Liekendael et al. 2006). Understanding what is important to customers and the standards they use to evaluate service is a paradigm shift for many agencies (Glascock 2006). A Brazilian study identified service attributes that influence users’ opinions negatively (Tobias et al. 2009). An Australian study explored ways in which passengers experi­ ence crowded situations (Thompson et al. 2012). A study of Bangkok’s mass rapid transit system analyzed customer satisfaction on the basis of 31 service quality attributes (Choocharukul and Sriroongvikrai 2013). Benchmarking is a process of identifying standards against which appropriate comparisons can be made. Cook and Lawrie (2006) developed a recommended benchmarking process for use by public transit systems in North Carolina. Another example of benchmarking in transit is New York City Transit’s adoption of a balanced scorecard approach, which is widely used in private industry and the public sector to monitor key performance indicators (KPIs) and to help achieve strategic outcomes. Advantages of KPIs include high­level visibility and ease of com­ munication, timely report availability, and detailed diagnostics. Wait assessment, the principal component of the service KPI, improved 2.5% on the heavily crowded No. 1 through No. 6 lines in 2012 compared with 2011, even as ridership increased (Reddy et al. 2014). Data envelopment analysis (DEA) is a very powerful service management and benchmark­ ing technique developed by Charnes et al. (1979) to evaluate nonprofit and public sector orga­ nizations. As explained by Sherman and Zhu (2006), linear programming is the underlying methodology that makes DEA particularly powerful as compared with alternative productivity management tools. DEA identifies the most efficient units or best practice units (branches, departments, individuals) and the inefficient units in which real efficiency improvements are possible. DEA calculates the amount and type of cost and resource savings that can be achieved by making each inefficient unit as efficient as the most efficient (best practice) units. Man­ agement receives information about the performance of service units that can be used to help transfer system and managerial expertise from better­managed, relatively efficient units to the inefficient ones. The first research paper found in the TRID database that used DEA in a transit context appeared in 1992 (Chu et al. 1992) and developed a single measure for the efficiency and a single measure for the effectiveness of a transit agency relative to other agencies within the same peer group. Beginning in 2007, the transit literature includes an increasing number of DEA applications in the worldwide transit field. Claims made for the DEA approach include the ability to allocate resources optimally across the transit network and to achieve targets for soci­ etal variables (Sheth et al. 2007). Another paper emphasized the ability to compare the perfor­ mance of multiple bus routes of one transit system and to adjust raw DEA scores to account for the environmental influences beyond the control of the transit agency (Barnum et al. 2008). A Norwegian study applied DEA to evaluate and improve the performance of ferries (Odeck and Brathen 2009). A study from China identified bus routes in Suzhou which need to be optimized and regulated (Yan et al. 2010). Another study used DEA to evaluate the technical efficiency of bus routes in terms of carbon emissions (Liu et al. 2013).

24 Transit Service Evaluation Standards DEA has also been used to assess the efficiency of bus depots and bus lines in Athens, Greece (Vlahogianni et al. 2015), to study the performance of bus depots in Bangalore, India (Hanumappa et al. 2016), to analyze 13 transit operators in the Yangtze Delta Region of China (Zhang et al. 2016), and to identify potential improvements in 42 bus routes in Brisbane, Australia (Tran et al. 2017). Two recent studies used DEA in the context of balancing service performance with either operational efficiency or service equity (Güner and Coskun 2016; Wei et al. 2017). Despite the increasing research focus on the usefulness of DEA in transit, this is not a tool commonly used by transit agencies. Sherman and Zhu (2006) have noted that managers have not widely adopted DEA to improve organization performance. The reason, in part, is because most DEA publications are in academic journals or books requiring the ability to understand linear programming and supporting mathematical notation. Over the past several years, several research papers have addressed new or revised performance indicators for inclusion in the performance evaluation process and extended the applicability of the process. One example examined the effectiveness and attractiveness of various control and regulatory arrangements for public transport provision within the United Kingdom and Ireland (Smyth and Kelleher 2018). Another discussed the challenges faced in developing performance regimes (Veeneman and Smith 2014), especially for contracted services. A new method has been proposed to spatially evaluate customer satisfaction survey data through examining satisfac­ tion with bus service across neighborhoods of varying levels of socioeconomic status (Grisé and El­Geneidy 2017). The development of a performance evaluation framework for traffic management and intel­ ligent transport systems emphasized transit policies (Kaparias et al. 2015). Another study pro­ posed a two­dimensional index system concerned with urban travel based on travel modes and user satisfaction (Weng et al. 2013). Choice of performance indicators is another area of research. A framework was proposed to ease the development of a monitoring system in the public transport domain (Benvenuti et al. 2017). A logical step in improving management of public funds is to identify methods that thor­ oughly assess the impacts of infrastructure and service investment (Bruun and Vanderschuren 2017). Another study proposed a method based in structural equation modeling to develop key indicator systems to evaluate transit performance during peak hours and off­peak hours (Wang et al. 2018). Current research also continues to seek a single summary indicator of transit performance. An example of this is a study focusing on transit quality of service that synthesized typical availability measures into several single variables (Kashfi et al. 2016). In the United States, a study examined performance evaluation for rural systems, noting that traditional transit per­ formance indicators more conducive to urban systems are not necessarily applicable in rural settings (Akoto 2016). Two articles addressed the use of Google Transit Feed Specification (GTFS) in performance evaluation. Using a newly developed open­source tool, the World Bank’s Open Transport Team has combined GTFS­formatted transit data sets with geographic information system (GIS) census data to derive globally comparative transit service, performance, and accessibility indicators (Qu et al. 2016). In the United States, Catala (2016) noted that the broad accep­ tance of GTFS and its rich repository of data offers an opportunity to develop evaluations on a nationwide scale.

Literature Review 25 Endnote to the Literature Review There will undoubtedly be continuing developments in transit performance evaluation. Many of the studies cited in this section make cogent arguments for a specific approach or analytical process to achieve the best possible results. This emphasis may be misguided. It is true that the vast expansion of available data and advances in analytic techniques offer new possibilities. The survey results and case studies described in the following sections emphasize that any perfor­ mance evaluation scheme needs to be tailored to the specific goals and objectives of a given transit agency and that simplicity is useful in explaining the results of performance evaluation to decision­makers and other stakeholders.