Benchmarking and Comparative Measurement for Effective Performance Management by Transportation Agencies (2019)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

1 Project Overview Research Objectives and Approach

Effective Performance Management by Transportation Agencies Project Overview Benchmarking and Comparative Measurement 2 Introduction Transportation is entering a period of massive change. Autonomous cars are a high-profile example of sector-wide shifts that infrastructure leaders and practitioners must navigate, along with perennial challenges like keeping assets in good repair. Less visible, but equally transformative, is the rush of data that is reshaping many aspects of our lives, including transportation. Google’s Eric Schmidt claims that humans took 10,000 years to generate the first five exabytes of data, but we now generate that every other day. Benchmarking is a process that empowers departments of transportation (DOTs) to harness the power of data rather than be overrun by it. Benchmarking allows DOTs to uncover success stories buried in data, share those stories with their peers, and use the lessons learned to help navigate challenges and become better stewards of our shared transportation system. Benchmarking sometimes raises fears about unfair comparisons, but most transportation practitioners agree that their agencies share overlapping characteristics: customers always care about reliability, safety, mobility, and convenience; agency leaders seek similar broad perspectives on the work they guide; and all agencies share a 24/7 commitment to investing in the right mix of people, knowledge, and resources to ensure transportation works as seamlessly as possible. Benchmarking provides a tested way to identify and spread the use of the most effective strategies for providing reliable and efficient transportation infrastructure. Surprisingly, however, benchmarking is not widely practiced among DOTs today. This report provides a roadmap to help change that. “The AASHTO community is more ready than it’s ever been to do benchmarking.” Carlos Braceros, Utah DOT CEO “The appetite for benchmarking is greater than ever. People are looking for state-to-state comparisons. This is a better time than ever to do this project.” Tim Henkel, Minnesota DOT, Assistant Commissioner Definition: benchmark ben(t)SHmärk The ongoing process of comparing outcomes and practices to those of similar organizations, with the intention of continuously improving quality and performance.

Effective Performance Management by Transportation Agencies Project Overview Benchmarking and Comparative Measurement 3 Research Objectives As shown in Figure 1, the research objectives for this project were threefold: 1. Produce research-based general guidance for transportation practitioners to use in deploying benchmarking; 2. Create a companion digital platform to help agencies identify DOTs most similar to their own and demonstrate the value of having easy access to comparative data; and 3. Conduct benchmarking pilots in the areas of nonmotorized transportation and environment using the principles described in the benchmarking guidance. Although the primary intent was to have the platform and pilots showcase and support the guidance, these elements also helped to inform the guide’s development as a result of lessons learned through these two implementations of benchmarking principles. Figure 1. Relationship of the three primary elements of the project

Effective Performance Management by Transportation Agencies Project Overview Benchmarking and Comparative Measurement 4 Research Approach This project had four major work components. Conduct Literature Review and Interviews The overall project direction was informed by information gathered from the following activities: • A literature review that included National Cooperative Highway Research Program (NCHRP) comparative studies, the organization and practices of successful benchmarking initiatives, and digital tools similar to the envisioned benchmarking platform; • Interviews with DOT executives and practitioners about the particulars of bringing benchmarking to DOTs; and • Interviews with benchmarking experts to learn about their benchmarking processes and which elements are most vital to success. Based on the literature review and interviews conducted with the agencies and organizations shown in the box below, the research team identified two benchmarking methods that shaped the direction of the project: 1. Independent benchmarking—An individual staff person or agency can practice benchmarking to monitor its performance against peers without much cost or upfront buy-in required. However, because other agencies are not committed to the endeavor, obtaining nonpublic data can be a challenge, and discovering noteworthy practices from high performers requires initiative and persistence from the benchmarking individual agency. 2. Benchmarking via networks— Benchmarking networks are organized groups committed to continual learning and improvement, often coordinated by a dedicated facilitator. Members agree on common reporting measures and definitions, and facilitators promote formal benchmarking activities; informal peer interactions provide additional value. Transportation Agencies • Colorado DOT • Delaware Valley Regional Planning Commission (Philadelphia MPO) • Michigan DOT • Minnesota DOT • Missouri DOT • Seattle DOT (Seattle, WA) • Utah DOT • Virginia DOT Benchmarking Initiatives • American Bus Benchmarking Group • Blackbaud (benchmarking firm) • Canadian Water and Wastewater Benchmarking Initiative • Council of State Governments • Florida Transportation Information System • Hospital Benchmark Generator • International Bus Benchmarking Group • International City/County Management Association 1

Effective Performance Management by Transportation Agencies Project Overview Benchmarking and Comparative Measurement 5 Develop Guidance The centerpiece of this report is a step-by-step guide on how transportation organizations can start and sustain a benchmarking initiative, whether within their own agency or among a network of peers. The guidance builds on the two- pronged approach to benchmarking that was identified by the research team in its literature review and research on the use of benchmarking by transportation organizations. Initial drafting of the guide relied heavily on the research done at the start of the project, particularly on the insights gained during interviews with benchmarking practitioners who had grappled with benchmarking challenges and transportation practitioners who discussed the realities of benchmarking at a transportation agency. A working draft guide was presented to the project’s panel and several additional transportation practitioners at a two-day invitational workshop in order to discuss the guide and direction for the remainder of the project. Feedback from this workshop provided the bulk of changes to the next iteration of the guide. Further changes were incorporated based on the progress of the two benchmarking pilots. As these trial benchmarking networks ran into roadblocks, achieved success, or played out differently than originally envisioned, the guide was updated to reflect the realities experienced by the research team. Build a Comparative Benchmarking Platform As a companion to the guidance, the research team created a web tool where agencies could find appropriate peers, access performance data, input their own data, compare performance across selected measures, and, most importantly, connect with other practitioners to discover practices for performance improvement. The team presented initial ideas for how this tool might work and solicited feedback from the project’s panel. Certain elements required significant discussion, such as who the tool would be accessible to, whether all users would have access to all data, and whether it was acceptable to publicly rank each agency’s performance. The project team implemented the vision that came out of storyboarding and discussions to create the fully functioning tool: the Transportation Comparative Benchmarking Platform. After several rounds of testing and updates by the research team and project panel, the final version of the platform was transferred to the transportation performance management (TPM) web portal on the American Association of State Highway and Transportation Officials (AASHTO) website for continued hosting. Detailed functionality of the final site is outlined in Part 2 of this report. New users can explore the site and register for access at http://benchmarking.tpm-portal.com/ 2 3

Effective Performance Management by Transportation Agencies Project Overview Benchmarking and Comparative Measurement 6 Conduct Benchmarking Pilots Two pilots provided real-world tests of the theoretical approach presented in the research study’s guidance on how to conduct benchmarking networks among peers. The intention was to test the principles and steps offered in the guidance and subsequently update the guidance based on what was learned in these trial implementations. The broad topic areas of environmental and nonmotorized performance were specified for the pilots in the project’s instructions. Narrowing the performance areas to something more specific took several rounds of research, memos, discussion at the project workshop, and consultation with subject area experts. The research team ultimately pursued wildlife–vehicle collisions and bicycle and pedestrian connectivity as the pilot performance areas. Recruiting efforts resulted in six to eight agencies willing to participate in each pilot, which reflected the team’s goal—based on early research—of five to ten participating agencies. The team led these practitioners through the steps of the benchmarking process outlined in this guide. The results of the pilots are detailed in the final section of this report. 4 Environmental Pilot Wildlife–Vehicle Collisions Percentage reduction in wildlife–vehicle collisions after project construction Nonmotorized Pilot Bicycle and Pedestrian Connectivity Route directness index

Report Organization Part 1: Benchmarking Guidebook Step-by-step guidance on how to start benchmarking now. Part 2: Transportation Comparative Benchmarking Platform A visual tour of the companion benchmarking web tool. Part 3: Benchmarking Pilot Results A review of the processes, outcomes, and lessons from the two benchmarking pilots. 7 Effective Performance Management by Transportation Agencies Project Overview Benchmarking and Comparative Measurement Part 1. Benchmarking Guidebook Part 3. Benchmarking Pilot Results Part 2. Transportation Comparative Benchmarking Platform