Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
24 4.0 OPERATIONALIZING THE PERFORMANCE MEASURES Incorporating sustainability into a performance-based planning and programming approach requires an understanding of both sustainability and principles of performance measurement. Several excellent resources are available for reference by transit agencies (see Section 4.5 and Appendix E). In general, operationalizing performance measures should follow five broad steps (see Figure 1). Each of these steps is isolated in order to emphasize the likely transfer of responsibility that would occur from each step to the next. While there may be overlap in the group of stakeholders in each step, different stakeholders are crucial for each of the steps. The makeup of each group will also change depending on the number of times the process has been iterated. Figure 1. Performance Management Framework Source: Authors Early in the process, it is critical for the traveling public and local communities to inform the transit agencyâs goals, either directly or indirectly, to ensure that the publicâs concerns are represented in the definition of sustainability and the transit agencyâs approach to it (step 1). Technical staff who would be responsible for collecting data and implementing plans must ensure that individual objectives are realistic and speak to the transit agencyâs internal priorities (step 2). Technical staff will also be engaged in the lionâs share of work during the implement and evaluate phase (step 4). Because the public is the ultimate audience for reporting, public considerations must also play a pivotal role in establishing measures (step 3). To build support for performance-based planning and for sustainability, transit agency leadership may play a larger role in reporting (step 5).
25 Because sustainability focused exercises are undertaken at the discretion of the transit agency, it is critical that appropriate resources and input are present at every step in order to ensure that the process is successful. Sustainability also informs the way the process is conducted (indicated by the foundational âsustainability principlesâ underlying all other steps in Figure 1). The first time the cycle is conducted, sustainability is likely to be a stand-alone exercise, isolated to a subset of goals and measures within a larger performance management system. In later iterations of the performance management cycle, sustainability may be used to inform all aspects of performance management and be a consideration in developing all performance measures and reporting documents. The transit agency staff responsible for carrying out sustainability tasks is likely to similarly change. When sustainability is first considered, it is common for the process to be overseen by a single individual or by a task force within the transit agency. As sustainability becomes more institutionalized within the transit agency, it is likely to driven by a broader coalition of stakeholders within the agency. While the performance cycle, as it is currently practiced, emphasizes measures that fall under a single transit agency or organizationâs control, transit agencies and other planning organizations should continue to build relationships with outside organizations to develop/support shared goals. This relationship building can result in collaboration between organizations and a set of outcomes that are more meaningful to the traveling public. Many transit agencies establish stakeholder advisory groups to inform decisions such as changes in service. A stakeholder group could be established to inform the transit agencyâs sustainability goals and objectives such as the LA Metro Sustainability Council (see Box 1). Table 9 illustrates the steps of the performance-based planning and programming cycle focusing on a single goal: Improve mobility and accessibility. The objectives that are chosen reflect actions that the transit agency can directly influence that will promote the regionâs mobility and accessibility. Performance measures chosen to reflect the transit agencyâs current conditions and resources, and realistic targets are identified based on best available data. As the transit agency proceeds through evaluation, the agency should consider how to better achieve these objectives The LA Metro Sustainability Council was formed to inform and improve Metroâs sustainability program by developing targets, metrics and strategies to assist Metro in achieving sustainability program goals. The Council objectives include: â¢ To advise in the development of Metroâs sustainability goals, established targets and performance measures, and assist in the tracking and reporting on a quarterly basis â¢ Improve understanding of our constituents and stakeholders of the sustainability-related efforts and opportunities at LA Metro; and â¢ To provide input on the following priorities for fiscal year 2019: 1) Climate Action and Adaptation Plan; 2) Green Procurement Policy; and 3) Resiliency Framework. The Council consists of external stakeholders from business, academia, the public sector, and community groups. Source: LA Metro, n.d. Box 1. LA Metro Sustainability Council
26 and goals, and consider how to revise all aspects of the process. Finally, as the transit agency achieves successes, those accomplishments are used to build support both for the transit agency and for sustainability objectives. Table 9. Sample Performance-Based Planning and Programming Cycle Step Example Key Questions Set Goals Improve mobility and accessibility What do our stakeholders care about? Determine Objectives Improve transit access for mobility-impaired riders How can the transit agency most meaningfully contribute to this goal? What modes are relevant, and what geographic scale is appropriate? Who needs to be at the table to ensure that we understand what resources are necessary to achieve this objective? How much capital is required in order to make the improvements? How will the projects be financed? What projects cannot be financed if these projects are prioritized? When should the transit agency aim to complete the necessary capital improvement projects? Establish Measures 1. Percent of stations/stops ADA accessible 2. Percent of vehicles ADA accessible 3. Percent of riders who are mobility-impaired by mode 4. Mobility-impaired rider overall satisfaction score What mobility data do we currently track? How many stations/stops and vehicles are currently ADA accessible? Which stations currently require improvements? How many vehicles must be purchased or retrofitted in order to be made accessible? What targets should be set? Implement and Evaluate 1. 100% of stations/stops ADA accessible after five years 2. 5% increase in vehicles that are ADA accessible 3. Mobility-impaired ridership increases 25% Is the transit agency observing an increase in mobility-impaired riders? What other obstacles prevent mobility-impaired riders from using transit? As we revisit this goal, what other performance measures should be considered to improve transit access for mobility-impaired riders? Report CEO ribbon-cutting ceremony for the last station to be made ADA- accessible How can success be used to both communicate the value of transit and build support for sustainability efforts?
27 Step Example Key Questions Notification on transit agencyâs social media sites Inclusion in annual performance reporting documents Each of the five steps is described in more detail in the following sections. 4.1 SET GOALS Sustainability goals should be reflective of the âtriple-bottom lineâ of social, economic, and environmental issues, and focus on the long-term rather than the short-term (NCHRP 2014). However, the practice of sustainability goal-setting is evolving within transit agencies. The process is largely driven by public demand and, while common themes have begun to emerge, continues to be customized to each new planning context. Goals should be reflective of sustainability outcomes that fall directly under the control of the transit agency as well as broader sustainability outcomes that are meaningful to transit users. A self-assessment tool may be useful to determine the transit agencyâs level of progress toward implementing a sustainability policy system (NCHRP 2011). When a sustainability framework is implemented for the first time, the stakeholders establishing these goals will likely be made up of a subcommittee or single individual within the organization. As both goal-setting and institutional support for sustainability grows, the group of stakeholders should expand to include representatives from other key groups within the transit agency, including leadership. Resources such as the Performance-based Planning and Programming Guidebook recommend that goal- setting occur in conjunction with both internal and external stakeholders (FHWA 2013a). This ensures that goals are connected to outcomes that are important to all of the transit agencyâs stakeholders. Another tool that can be used to identify which topics are most relevant (or material) is a materiality assessment, defined in detail by the GRI Standards (GRI 2017). See Error! Reference source not found. for an example of how TransLink used the GRI Standards to conduct a materiality assessment. Goal-setting often evolves depending on a transit agencyâs level of experience with regard to sustainability. Many organizations beginning the process of incorporating sustainability will develop stand-alone processes that specifically address principles of sustainability. In this case, there may be sustainability goals and sustainability performance measures that feed into a specific sustainability report, such as the award-winning report produced by SEPTA in 2011 (SEPTA 2017). See Box 3 for more information on SEPTAâs goals related to social and economic sustainability. As organizations gain experience, sustainability is often included throughout a performance-based planning and programming approach. Good examples of this include LA Metro and TransLink; both agencies have begun to incorporate sustainability considerations into all aspects of planning, and no longer consider sustainability in isolation from other planning efforts.
28 Box 2. Incorporating Public Input into Goal Setting The GRI Standards provides guidance to transit agencies on conducting a materiality assessment to identify topics that may be important to consider in sustainability reporting. In 2017, TransLink, the transportation authority responsible for public transportation in the Metro Vancouver Regional District, completed a two-year materiality assessment (in accordance with the previous version of GRI, the GRI G4 Guidelines) that used feedback gathered by survey to determine the transit agencyâs priorities. Both internal and external stakeholders were questioned on issues of governance, economic, environmental, and social responsibility. This engagement effort resulted in 17 topics being identified as highest-priority. The following list shows the top 30 items identified through the assessment: 1. Safety and security of transit customers 2. Investing in infrastructure 3. Satisfaction of transit customers 4. Air pollution (from transit operations) 5. Employee safety and wellness 6. Minimizing overcrowding on transit services 7. Minimizing crime on TransLink property 8. Ensuring transit fares are affordable 9. Accessibility for mobility, hearing and visually impaired customers 10. Employee training and education 11. Data privacy and security 12. Energy consumption 13. Climate change/greenhouse gas emissions 14. Increasing transit ridership 15. Satisfaction of employees 16. Public disclosure of data and targets 17. Planning for extreme weather events 18. Managing sustainability risks 19. Keeping TransLinkâs operating costs low 20. TransLink Senior Leadership oversight and input into sustainability strategy 21. Waste diversion 22. Investing in innovations (new bus technology) 23. Stakeholder engagement programs 24. Environmental screening of suppliers/contractors 25. TransLink Board oversight and input into Sustainability 26. Satisfaction of non-transit customers 27. Water consumption and reuse 28. Green buildings 29. Social Screening of suppliers and contractors 30. Investing in the community (sponsorship/donations) Source: TransLink 2017, GRI 2015 While the environmental goals associated with a sustainability framework may be more familiar to transit agencies, social and economic goals may not be as familiar. The goal elements included in the APTA Recommended Practices: Social and Economic Sustainability for Transit Agencies (2018) are a good starting point for transit agencies to develop more specific goals that reflect their individual priorities. APTA recommends the following six goals with regard to social and economic outcomes (APTA, 2018b, p. 2) (see also Section 1.1):
29 1) Community Building and Engagement: Engage diverse groups to improve transit service, create hospitality in customer service, and demonstrate goodwill through engagement techniques and commitment to good design in the public realm. 2) Economic Impact: Support the economic growth of our regions and the nation. 3) Employees and Workforce: Create a conducive and supportive environment for all transit employees. 4) Financial: Ensure the reliability of transit services through financial stability. 5) Mobility and Accessibility: Make it easier for people of all abilities to affordably and reasonably access different goods and services to meet their daily needs. 6) Safety and Emergency Preparedness: Ensure that operations are safe and do not compromise the well-being of riders, staff or the public. While goal-setting typically precedes establishing performance measures, it is useful to establish goals, objectives and measures together. Performance measures can help ground and validate the goals and objectives. This was observed during workshops conducted by NCDOT (Maurer et al 2013). Several objectives were revised or removed if the workshop participants were unable to define reasonable measures to track an outcome. However, NCDOT elected to retain several more aspirational objectives such as economic development with the assumption that emerging strategies would support development of measures in the future (Maurer et al 2013). The Southeastern Pennsylvania Transportation Authority launched its first award-winning five-year sustainability plan in 2011, and recently released its second-generation plan: SEP-TAINABLE 2020. Two of the transit agencyâs goal areas specifically relate to social and economic outcomes. All of the agencyâs goals address the triple-bottom line of sustainability, and the complete list is included below. Natural Environment â¢ Goal 1: Reduce SEPTAâs Carbon Footprint â¢ Goal 2: Improve Energy Efficiency â¢ Goal 3: Improve Water Efficiency â¢ Goal 4: Reduce Stormwater Runoff â¢ Goal 5: Reduce & Reuse Waste Healthy Communities & Workforce â¢ Goal 6: Integrate with Livable Communities â¢ Goal 7: Improve Access to Local Food Via Transit â¢ Goal 8: Develop a Highly Skilled, Healthy & Versatile Workforce â¢ Goal 9: Support Regional Business Equity Economic Vitality â¢ Goal 10: Increase Ridership â¢ Goal 11: Improve Operating Expense Performance â¢ Goal 12: Institutionalize Environmental Management Practices â¢ Goal 13: Financial Value of Sustainability Source: SEPTA 2017 Box 3. Goal Setting: SEP-TAINABLE 2020
30 4.2 DETERMINE OBJECTIVES Where goals are broadly defined, objectives are action-oriented. Todd Litman characterizes objectives as âspecific ways to achieve goalsâ (Litman 2016, p. 10). Where goals may be generally formed to encompass a broad range of outcomes, the objectives provide a clear path for the transit agency to follow in support of a goal. This does not necessarily imply that transit agencies must be solely responsible for objectives â during objective-setting, agencies may choose to engage outside organizations (including the metropolitan planning organization or regional planning commission, counties, health organizations, and others) to develop shared objectives. An example of this is the objective shared by TransLink and the City of Vancouver that âhalf of all trips be made either by walking, biking, or transit by 2040.â See Box 4 for another example of developing a shared objective. Establishing an objective is effectively a bridge between the goal to the performance measure, and they are distinct from goals in that they are specific and measurable â criteria that later also come People experiencing homelessness often shelter in public transit facilities and vehicles. This can result in real or perceived safety issues to transit passengers and employees. TCRP Synthesis 121âTransit Agency Practices in Interacting with People who are Homeless (2016)âdescribes a continuum of transit agency responses to this issue: 1. Not our problem 2. Riders are unhappy â do something! 3. Enforcement 4. Partnerships AND Enforcement In 2012, SEPTA partnered with the City of Philadelphia and Project HOME to open the Hub of Hope during the winter as a walk-in engagement center located in the concourses under Two Penn center in SEPTAâs Suburban Station. Project goals included: 1. Transition people experiencing homelessness into permanent housing 2. Provide easy, centralized access to integrated health care services and connect people to ongoing primary care; and 3. Deepen the understanding of necessary, strategic, and effective tools and methods to better assist and end homelessness for people who were living in the subway concourses. While no specific objectives were identified during the literature review, the agencies could develop shared objectives such as: âReduce the number of individuals experiencing homelessness that live in the subway concourse by 10 percent by 2022.â Since its opening, there have been over 36,000 visits to the Hub and 1,429 placements into shelter and treatment programs. In 2018, the Hub of Hope was established as a permanent facility offering a safe place for individuals experiencing homelessness to have a cup of coffee, take a shower and wash laundry, and speak with case managers to find permanent housing. The Hub continues to facilitate opportunities to connect individuals to treatment, medical care, and housing. Sources: TCRP 2016, Project Home 2018 Box 4. Example Shared Objective: Addressing Homelessness
31 into play when evaluating performance measures. Many of the same criteria that are used to evaluate performance measures are also used to craft clear objectives, such as the âSMARTâ criteria (Specific, Measurable, Agreed, Realistic, and Time-Bound) (FHWA 2013a). The Performance-Based Planning and Programming Guidebook (FHWA 2013a) outlines a useful process for establishing objectives. FHWA acknowledges that transit agencies may initially develop more general objectives, but refine them over time to be specific, measurable and time- bound. A transit agency could start with a general objective, but as familiarity with sustainability and performance management increases, later develop a SMART version. For example, a general objective might be to âincrease the number of requests for proposals (RFPs) that include sustainability criteria.â A SMART version would modify that to include a target and a date: âinclude sustainability criteria in 80% of RFPs by 2020.â In developing objectives, transit agencies may find it useful to consider the following (FHWA 2013a): â¢ Key focus areas: What aspects of the goal are most important? â¢ Purpose of travel: What type(s) of travel are affected (e.g., commute, educational, leisure, errands)? â¢ Mode of travel: Should the objective be specific to one or more mode(s)? â¢ Geographic scale: How does location affect the development of the objective? As with many other parts of the performance management process, evaluating objectives alongside goals (or through an iterative process) will ensure that goals are well-grounded, and that objectives are focused on appropriate goals. Considering performance measures alongside objectives will ensure that objectives have a clear path to implementation, and that objectives have appropriate targets. Using an iterative process that highlights the relationship between the steps of the performance management process is useful to ensure that the transit agencyâs goals can be implemented effectively. Objectives included in the APTA Recommended Practices: Social and Economic Sustainability for Transit Agencies (2018) speak strongly to social and economic outcomes. APTA recommends twenty-one objectives that link to six goal elements. The complete list of objectives is included in Table 10. The APTA objectives are not specific and measurable because they are intended to apply to a wide range of transit agencies, however, this list of objectives provides a useful starting point for transit agencies to tailor objectives to their specific situation and goals. For example, DART established a goal related to safety and security that is similar to the APTA goal of Safety and Emergency Preparedness, but established a detailed specific objective that follows the APTA objective of security (see Box 5). Table 10. APTA Goals and Objectives Goal Objective Community Building and Engagement â¢ Collaboration and Partnerships â¢ Community Engagement â¢ Good Design Elements â¢ Rider Engagement
32 Goal Objective Economic Impact â¢ Measure and Communicate Economic Benefits of Transit â¢ Politically Leverage Economic Benefits â¢ Understand Distributional Effects on Specific Areas or Groups â¢ Extend Economic Reach of Public Transportation Employees and Workforce â¢ Employee Recruiting â¢ Employee Retention â¢ Organizational Culture and Workforce Engagement Financial â¢ Fiscal Responsibility â¢ Procurement Strategies â¢ Sustainable Investments Mobility and Accessibility â¢ Access â¢ Affordability (housing and transportation) â¢ Multimodal Connectivity Safety and Emergency Preparedness â¢ Emergency Preparedness â¢ Health and Wellness â¢ Safety â¢ Security Source: Adapted from APTA 2018b 4.3 ESTABLISH MEASURES A transit agencyâs performance measures, taken as a whole, provide a framework for evaluating progress toward goals. The right assembly of performance measures will provide meaningful evidence of progress to a diverse group of internal and external stakeholders with differing needs of the transit agency. Performance measures should also be useful to provide valuable feedback on the transit agencyâs efforts with regards to other aspects of the performance-based planning approach. Performance measures can be used to clarify the definition of goals, provide a reference for target-setting and a baseline for planning efforts, inform policy decisions as transit agencies consider alternatives, consider priorities for the next planning cycle, and communicate the value of transit to the public (FHWA 2013a). However, transit agencies must consider performance measures relative to the transit agency context, the resources available for sustainability efforts and performance management, the type of performance measures needed, the quality of both Goal: Improve customers' sense of safety and security at facilities and on vehicles through improved awareness, enhanced design features, and increased visibility Objective: Keep our riders safe by maintaining security incident rates equal to or better than that of the communities in which we operate Performance Measure: Percentage of customers reporting they feel safe riding DART Source: Shelton 2018 Box 5. Dallas Area Rapid Transit Social Sustainability Objective
33 individual performance measures and the assembly of performance measures, and the best way to prioritize measures. The process is not insubstantial, and continues to evolve as organizations move further in the direction of transparency and accountability to stakeholders. The process involves selecting, refining and evaluation measures. 4.3.1 Select Measures A main focus of this project has been to develop a list of social and economic sustainability performance measures that can be used track a transit agencyâs performance toward achieving social and economic outcomes. Section 3.3 and the companion ExcelÂ® database provide a list of social and economic performance measures related to public transit as well as transit service measures that agencies can use as a starting point. The ultimate list of performance measures that is appropriate for a transit agency may be drawn from that list based on agency characteristics, requests from key stakeholders, or data availability. For sustainability-oriented efforts, an important factor to consider is the level of effort and level of expense required for each objective. Unlike federally required NTD measures, objectives related to sustainability are left to the discretion of transit agencies. However, some of the NTD measures support social and economic sustainability, such as those related to financial performance, safety, and transit asset management. For example, transit agencies must now report asset inventory, condition assessments, performance targets, and a narrative report to the NTD (FTA 2017). Performance measures that track the percentage of revenue vehicles that have exceeded the useful life benchmark demonstrate a commitment to a transit agencyâs financial sustainability. Measures related to transit convenience and availability (e.g., travel time, hours of service, average travel delay, etc.) are important indicators of transit accessibility, and could be used to evaluate goals related to improving/expanding transit access. In the example below, DART uses on-time performance to support a social sustainability objective of improving reliability for passengers (see Box 6). Certain transit agencies that receive federal funding must also report safety performance measures in accordance with the recently published Public Transportation Agency Safety Plan (PTASP) Final Rule, which goes into effect in July 2019 (FTA 2018). This rule requires transit agencies to develop Safety Management System (SMSs) that includes safety performance targets. Transit Goal: Grow ridership by improving rider experience through reliable, inclusive, universally accessible, comfortable, and convenient transit services. Objective: Improve reliability to help all riders get where they need to go on time. Performance Measure: On-Time Performance by Mode: â¢ Bus = 90% of better â¢ Paratransit = 92% or better â¢ Rail = 95% or better Source: Shelton 2018 Box 6. Dallas Area Rapid Transit Example Using Transit Service Measures
34 agencies should consult the requirements and guidance related to this new rule when developing safety performance measures. Transit agencies must be mindful of available resources to ensure that the process of performance management is itself sustainable and can be maintained over a period of years. It may not be desirable to select a long list of measures, but instead focus on a smaller subset that can be meaningfully tracked over a period of years. If certain measures are already being tracked, it may be useful to prioritize these measures as the foundation of the performance-based planning and programming approach. Additional national reporting requirements that track social and economic sustainability performance measures include OSHA and the U.S. Department of Labor Equal Employment Opportunity (EEO) reporting requirements. Some transit agencies may also use voluntary reporting frameworks such as the GRI Standards, the Community of Metros (CoMET) benchmarking group and the Nova Group of Metros benchmarking group. When assembling a suite of potential performance measures, an important consideration is the type of performance measures (e.g., input, process, output and outcome) (See Section 3.2, Table 7) (Litman 2016). The different measures are useful in that certain types of measures tend to be more meaningful to the traveling public and local communities (outcome measures), where other measures may be more reflective of transit agency actions. Most organizations, including transit agencies, primarily (or exclusively) report outcome measures. Outcome measures are easier to measure and are more understandable to internal and external stakeholders. However, outcome measures simply report a result and cannot be improved or influenced. Input or process measures, while more challenging to measure, are easy to influence and can âserve as a catalyst for changeâ that support improvements in social and economic sustainability outcomes (Blair and OâToole 2010). Box 7 provides an example of how different types of performance measures can be used to improve safety. Goal: Reduce accidents at rail crossings Output Measures: Total number of at-grade crossings Outcome Measure: Number of accidents at rail crossings Input Measures: â¢ Number of at-grade crossings eliminated â¢ Investment in additional at-grade rail crossing safety measures â¢ Number of near-misses reported Process Measures: Process in-place to report near-misses Source: Authors. Refer to the Transit Advisory Committee for Safety (TRACS 2017) 16-02 Final Report: Safety Data and Performance Measures in Transit for more information on the benefits of using multiple types of performance measures. Box 7. Improving Safety Performance Using Different Types of Performance Measures
35 4.3.2 Refine Measures for Agency Context When appropriate measures have been selected, performance measures can be assembled to reflect key characteristics of the transit agency. All of the performance measures included in Section 3.3 or the companion ExcelÂ® database may be re-formulated based on the count, measure, qualifier, and normalization factor. This process can be used to reflect the transit agencyâs institutional capacity, the publicâs desired frequency of reporting, and other factors. Examples of performance measures broken down according to this schema are included in Table 11. Table 11. Examples of Performance Measure Count, Measure, Qualifier, and Normalizing Factor Count Measure Qualifier Normalization factor Number of planning studies led or collaborated on per year Percent of transit access intersections that are ADA compliant (none) Percent increase in value of land near rail station areas relative to other areas Number/percent of interviewers who received anti-bias training (none) Total and percent revenue (none) by type (e.g., capital, operating, etc.) and by source (e.g., fare, local, state, federal, etc.) (Dollars) Public transportation expenditures (Public transportation) per capita (Measured in dollars or miles funded) Sidewalk connections, bike facility connections, pedestrian safety improvements included in project planning and design (none) The APTA Recommended Practice Quantifying and Reporting Transit Sustainability Metrics (2012) includes several normalizing factors, which can be used to provide critical context for each transit agency. Selecting an appropriate normalizing factor is critical to track transit agency performance over time, or in order to compare performance between among transit agencies. Table 12 defines each normalization factor identified in the APTA Recommended Practice.
36 Table 12. APTA Normalization Factors Normalization Factor Description Purpose Passenger Miles Traveled (PMT) The cumulative sum of the distances ridden by each passenger To measure service consumed in a way that considers productivity and operational efficiency and captures effects to increase ridership or vehicle occupancy rates Vehicle Revenue Hours (VRH) Hours traveled when the vehicle is in revenue service To measure service provided in a way that accounts for operational efficiency and captures efforts to reduce deadheading and roadway congestion. Vehicle Miles (VM) Miles traveled from the time a vehicle pulls out from the garage to go into revenue service to the time it pulls in from revenue service, including âdeadheadâ miles without passengers to the starting points of routes or returning to the garage. To measure service provided in a way that accounts for facility, vehicle and fuel efficiency and captures efforts to purchase and retrofit vehicles and facilities with cleaner and more efficient technologies. Vehicle Revenue Miles (VRM) Miles traveled when the vehicle is in revenue service To measure service provided in a way that accounts for both operational efficiency as well as facility, vehicle and fuel efficiency, capturing efforts to reduce deadheading and roadway congestion, as well as efforts to purchase and retrofit vehicles and facilities with cleaner more efficient technologies. Unlinked Passenger Trips (UPT) or Boardings The number of times passengers board vehicles, no matter how any vehicles they use to travel from their origin to their destination and regardless of whether they pay a fare, use a pass or transfer, ride for free, or pay in some other way. Also called boardings. To measure service consumed in a way that considers productivity and operational efficiency and captures efforts to increase ridership or vehicle occupancy rates. Unlike PMT, will not account for the relative amount (distance) of service provided to each passenger on each mode.
37 Normalization Factor Description Purpose Produced Seat Miles (PSM) The total seating capacity of vehicles in revenue service multiplied by total vehicle miles. To measure service provided in a way that considers seating capacity of vehicles, which can be useful for comparing systems utilizing different capacity vehicles; overlooks standee capacity as a component of overall vehicle capacity. Revenue Vehicle Length (RVL) The size of a vehicle based on the distance from the front of a vehicle to the back of a vehicle. To measure the relative efficiency of vehicle washing systems in a way that accounts for the relative size of vehicles and captures efforts to reduce the amount of water required to clean each vehicle. Per Capita in Service Area of Operation The number of people with access to transit service in terms of population served and area coverage (square miles), based on definitions contained in the ADA Act and reported to the NTD. To measure service provided in a way that accounts for total extent of operations and captures measures to increase transit mode share and to reduce the number and distance of trips taken by private automobiles within the service area of operation. Source: Adapted from APTA 2012, pp. 5- 6 4.3.2 Evaluate Measures Once appropriate performance measures have been selected and refined, it is important to use evaluation criteria to ensure that the measures are appropriate and achievable. This process also enables transit agencies to prioritize measures, identify the most critical items to track and, in the face of limited resources, determine which measures out of a long list should be the object of attention. The Performance-Based Planning and Programming Guidebook focuses on âSMARTââ criteria (FHWA 2013a). In Well-Measured, Todd Litman recommends a similar list of criteria but includes several additional factors that should be considered specifically for sustainability-oriented performance measures (Litman 2016). A list of guiding questions that can be used to form evaluation criteria well-suited to sustainability performance measures are included in Box 8. A particularly useful question for transit agencies newly embarking on performance management is the question of scale. For example, a transit agency may not be able to realistically track all expenditures on local suppliers â but could focus on tracking specific products/services. This type of criteria can be used to reframe performance measures to make them more actionable and relevant. The text box to the right includes a comprehensive list of evaluation criteria that synthesize elements of SMART criteria with other feedback collected over this research effort.
38 In addition to evaluation criteria that should be used against individual measures, the performance management system at large should be evaluated for many of the same factors, but also for completeness. This is particularly important to capture all aspects of sustainability. When evaluating the entire suite of performance measures, the following questions can be used to evaluate the strength of the approach and commitment to all aspects of sustainability: â¢ Are there measures here reflective of all aspects of sustainability â including environmental, social, and economic outcomes? â¢ Do some measures explicitly include all transit riders, including mobility-impaired, the elderly, low-income, minority, and other transportation-disadvantaged populations? â¢ Do these measures, taken as a whole, reflect the goals chosen by the transit agency? â¢ Do these measures meaningfully contribute to our sustainability objectives? The two lists of questions provided above provide a starting point for transit agencies that may be refined based on individual transit agency needs and resources. The list of evaluation criteria, like the list of performance measures, should not be so extensive that it becomes a cumbersome tool. Using the final list of evaluation criteria to prioritize performance measures can be accomplished in several ways: two of the most common ways include a survey or a workshop. If it is possible to bring many groups of stakeholders into a single location for a day, a workshop approach is useful for building collaboration within the organization and developing institutional supportâboth for performance management in general and sustainability in particular. Workshops are useful for assembling diverse stakeholders and views in a short period of time, which would enable the transit agency to develop measures that reflect a diverse set of interests. Furthermore, a workshop is useful in developing organization culture around sustainability outcomes and enabling organizations to deepen their understanding of sustainability and performance measurement through conversation (Lane et. al. 2014). Is the objective specific enough to be measured? Does the transit agency have the data necessary? Do responsible parties have the resources necessary to track this measure? Is this measure targeted at a particular outcome? Is it realistic for the transit agency to be held accountable to this target? What is the appropriate level of scale (e.g., financial, geographic) for this measure? Is this measure understandable? Will this measure enable us to compare our progress relative to peer agencies? Is the measurement time interval appropriate? (Recall that not all performance measures need to be tracked at the same interval in order to be relevant.) Is this performance measure useful in establishing targets for agency operations? Box 8. Guiding Evaluation Criteria for Individual Measures
39 4.4 IMPLEMENT AND EVALUATE While earlier steps involve extensive public involvement, a key feature of this step involves close coordination between technical staff and the individual/individuals responsible for evaluation. Much of the day-to-day work of executing plans and tracking progress is carried out by technical staff. As capital improvements are made and services expanded, this staff will be tasked with gathering the relevant data on the performance measure and evaluating the transit agencyâs progress. Internal collaboration across different departments within the transit agency is necessary in order to identify stakeholders already engaged in tracking data that can be used to measure performance. It is important to understand what data is already being collected, in what format(s) the data available, and what is being done with the data. When more than one individual or department is responsible for a measure, it is critical to confirm that the same definition is used for data collection. For example, bus drivers must identify customer injuries in the same way as train drivers. Collecting data related to transit agency environmental sustainability is generally more straightforward in terms of what should be measured and how (e.g., gallons of diesel fuel consumed). Social and economic sustainability is more challenging to quantify and may not have well established forms of measurement. For social and economic sustainability measures, internal data on operations will often be supplemented by data gathered from public sources, or through passenger surveys. A list of potential data sources for social and economic performance measures is included in Box 9. Many of those on this list can be publicly accessed. American Community Survey (ACS) American College of Sports Medicine (ACSM) Health & Fitness Journal APTA Transit Statistics Bureau of Economic Analysis (BEA) Bureau of Labor Statistics (BLS) Bureau of Transportation Statistics (BTS) County Tax Assessors Census Transportation Planning Products (CTPP) Housing and Transportation Affordability Index (HTAI) Internal agency data Integrated Public Use Microdata Series (IPUMS) Longitudinal Employer-Household Dynamics (LEHD) Local Travel Surveys (LTS) Metropolitan Planning Organization (MPO) National Household Travel Survey (NHTS) National Transit Database (NTD) National Establishment Time Series (NETS) Transit Boardings Estimation and Simulation Tool (TBEST) State Department of Education State Department of Transportation Surveys (e.g., paratransit, onboard, household) â internal and external United States Census Bureau One key tool in obtaining data to support social and economic sustainability performance measures is through employee and passenger surveys, such as the annual LA Metro survey (2018) (see Box 10). Currently, surveys are generally developed by individual agencies. Transit agencies would Box 9. Potential Data Sources for Social and Economic Performance Measures
40 benefit by using a standardized survey tool to provide uniformity to transit agencies seeking to compare their progress on social and economic measures, and reduce the barriers to entry for transit agencies seeking to use a similar tool. This data must be gathered and summarized in order to make it meaningful. Box 10. LA Metro Customer Satisfaction Survey (June 2018) Once a year, LA Metro distributes paper surveys on board buses and train in order to track the transit agencyâs performance and demographic trends of transit users. In the spring of 2018, bus users were asked the following questions: â¢ Generally speaking, I am satisfied with Metro bus service. â¢ This bus is generally on time (within 5 minutes). â¢ I feel safe waiting for this bus. â¢ I feel safe while riding this bus. â¢ This bus is generally clean. â¢ This busâs stops are generally clean. â¢ Did you receive a discount on your fare? â¢ If yes, what type of discount did you receive? â¢ What type of fare did you use? â¢ Do you own a (select: smart phone, cell phone, neither)? â¢ How often do you use mobile applications (on your phone or tablet) such as "Go Metro", "Transit", "511", & "Google Maps" to get traffic information? â¢ Do you ride Metro Bus/Rail primarily for (select: work/school, errands/recreation, both equally)? â¢ How did you get to the FIRST bus or train of THIS trip? (select: walked, dropped off, drove, biked, skateboarded, other) â¢ How many minutes did it take you to get to the FIRST bus or train of THIS trip? â¢ How many minutes did you wait for that FIRST bus or train? â¢ Do you have a car available to make THIS trip? â¢ Will you have to transfer to complete THIS trip? â¢ If yes, is your transfer scheduled to arrive within 15 minutes? â¢ In the past 6 months, while riding Metro, have you personally experienced any of the following types of sexual harassment? (select: non-physical, physical, indecent exposure) â¢ Regarding the number of fare enforcement personnel on-board Metro buses, do you think there are: (select: far too many, too many, right amount, too few, far too few) â¢ Regarding the number of police officers onboard Metro buses, do you think there are: (select: far too many, too many, right amount, too few, far too few) â¢ How many days a week do you usually ride Metro? â¢ How many years have you been riding Metro? â¢ What language did you complete the survey in? â¢ What is your ethnicity? â¢ What is your gender identity? â¢ What is your age? â¢ Householdâs total annual earnings? Source: LA Metro 2018. A key consideration for any performance tracking system is the question of handling data. Transit agency officials report that obtaining key pieces of data at regular intervals is a substantial challenge, and maintaining a relevant database requires grappling with issues of data integrity.
41 Managing the data required for this process is not an insignificant undertaking. Establishing protocols for data and conducting regular quality control is critical. For some measures, it may be worth the investment to develop robust databases that can intelligently track and report data. As work products are completed, transit agencies will evaluate progress. At regular intervals, transit agencies should evaluate whether performance results indicate progress toward achieving a goal, consider what other obstacles may prevent success, and whether additional performance measures should be considered to achieve goals. This feedback should be considered in future iterations of the performance cycle to inform changes throughout the process. In the example provided in Box 11, Rogue Valley Transit District (RVTD) developed performance measures for use in scenario-level evaluation, project level evaluation, and performance monitoring. For each measure, RVTD indicated why the measure was being tracked, if the data is already collected by RVTD, and which tool(s) are used to obtain data. RVTD developed measures for use in one or more of three different situations: (1) scenario-level evaluation, (2) project-level evaluation; and (3) performance monitoring. Measures are identified as either Tier Iâ required measure based on adopted RVTD, Rogue Valley MPO, or Statewide plan or policyâor Tier IIâ RVTD on-going measure or local agency plan supported measure. A draft version of performance measures related to RVTDâs Community Goal: Connect the region, focusing on increasing equitable access to transportation and improving quality of life is provided below. Measure Scenario- level Evaluation Project- Level Evaluation Performance Monitoring Already Collected by RVTD Tool Tier Ridership X X X X TBEST, Farebox I Ridership per capita X X X X TBEST, Ridership, Population Data I Percentage of current and future mixed-use/multi- family zoned land with Â¼- mile of a transit stop X X TBEST II Low-income pop. within Â¼-mile of transit stop X X TBEST I Minority population within Â¼-mile of transit stop X X TBEST I Percentage of regional employment within Â¼-mile of transit route X TBEST, JEMnr II Aged population within Â¼- mile of transit route X X TBEST I Disabled population within Â¼-mile of transit route X X TBEST I Source: Adapted from Draft RVTD Goals (used with permission) Box 11. Example: Rogue Valley Transit District Performance Measurement
42 4.5 REPORT Reporting measures will likely require support from a different group of stakeholders again than are involved with implementing the performance management system. Reporting a transit agencyâs progress with regard to sustainability outcomes is a useful technique for building consensus within an organization around both sustainability and performance management, and may help to create a sense of ownership by others in the organization. This step should not be overlooked, as reporting progress toward social and economic sustainability performance measures may be an effective method for increasing confidence in the transit agency, and can be used to make the case for additional funding. Reporting can be an effective tool to both communicate the impact transit has on social and economic sustainability and provide accountability to stakeholders. Some transit agencies may find it beneficial to produce a stand-alone Sustainability Report in order to call attention to the transit agencyâs sustainability efforts. Transit agencies may also choose to integrate sustainability into other reporting documents, such as financial and accountability reports, but should be explicit about the measurementsâ connection to sustainability outcomes. Mapping performance measures against other reporting standards (e.g., GRI, UN Sustainable Development Goals, and International Association of Public Transport) may also help to draw connections between the transit agencyâs efforts and other measures of sustainability that may be familiar to different audiences. In 2017, TransLink began integrating sustainability reporting into their annual Accountability Report. Transit agencies should consider their intended audience(s) when determining how to present information on sustainability performance measures. Tables, figures and graphics can be particularly effective ways to communicate progress towards social and economic performance measures. Data can be presented over time or for a specific reporting period. Figure 2. TransLink Safety and Security Trends Source: TransLink 2017, p. 29 (used with permission)
43 Figure 3. SEPTA Workforce Development Data Source: SEPTA 2018, p. 35 Reporting can also yield feedback from internal and external stakeholders that can help transit agencies further refine goals, objectives and measures to support continuous improvement. Additional Resources A few key resources related to performance-based planning are included in Box 12, and additional resources can be found in Appendix A â Publication Review Summaries. â¢ FHWA Performance Based Planning and Programming Guidebook (2013a) â¢ NCHRP 708: Sustainability Performance Measures for Transportation Agencies (2014) â¢ Well Measured: Developing Indicators for Sustainable and Livable Transport Planning (Litman 2016) â¢ TCRP Transit Capacity and Quality of Service Manual (2013) Box 12. Key Resources for Transit Agencies