Management and Use of Data for Transportation Performance Management: Guide for Practitioners (2019)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Introduction

Introduction • Foundation • Reporting • Insight • Cases Why This Guide? Transportation Performance Management (TPM) is an established and maturing practice at state and local transportation agencies. Recent federal legislation has established requirements for agencies to set performance targets and report on safety, pavement and bridge conditions, transit asset state of good repair, system performance, freight, and air quality. These requirements have resulted in increased visibility and attention to TPM and increased awareness of the importance of data within that process. Transportation agencies are recognizing that the value of performance management goes far beyond simply meeting federal requirements—it provides a way to make visible progress in meeting agency objectives such as reducing fatalities, improving asset condition, eliminating bottlenecks, or speeding project delivery. Meanwhile, with the emergence of new collection technologies, data sources, and private-sector data product and service offerings, seismic shifts are occurring in the data landscape. These changes are causing transportation agencies to re-examine their approach to data management. TPM Defined This guide uses the term “TPM” to mean a process that involves 1. Measuring current transportation system performance, 2. Setting goals and targets for performance improvement, 3. Allocating resources and planning work to achieve the desired improvement, and 4. Monitoring results achieved to adjust plans and programs and update targets as needed. Performance Measurement: Understanding current performance and tracking progress toward meeting objectives. Performance Management: Using performance measurements and other information to drive decisions. Management and Use of Data for Transportation Performance Management: Guide for Practitioners 2

Introduction • Foundation • Reporting • Insight • Cases Data are essential throughout the TPM process. Good quality data allow agencies to understand current performance, assess risks, and plan and prioritize improvement actions. Reliable data can be used to pinpoint trouble spots, identify where to intervene, and illuminate where investment is most beneficial. It helps agencies to maximize the effectiveness of available funding and staff resources, and it provides the strong, credible foundation for decision making that is essential for building and maintaining public trust. For many agencies, the problem is not a lack of data; it is a lack of capability to transform available data into useful information. This requires deliberate effort at all stages of the data life cycle, from specification through analysis, to ensure that data are of sufficient quality and that it can be integrated, visualized, and used to provide insights. Having people with the right skills and experience to carry out these activities is essential. Careful attention to assessing data needs and implementation of proven data management practices can provide the foundation for a successful TPM process—helping agencies to make wise transportation investments that save taxpayer dollars, reduce congestion, encourage economic growth, improve the quality of life, and even save lives. Lack of skillful data management creates inefficiencies due to duplicative effort and manual, error-prone processes. It also creates risks ranging from failed data collection efforts to lack of progress on improving performance. Most transportation agencies are not starting from scratch with TPM; they will have already established a set of transportation performance measures and data sources used for internal and external reporting. However, there are many scenarios that may cause an agency to seek ideas for improving performance measures and associated data programs: • Current data are not sufficiently reliable in calculating performance measures that support funding and resourcing decisions. • Current performance measures are not helpful in identifying how and where to make improvements. • Agency performance measures were established based on what data were available at the time, and now there are better data sources available. • New analysis or visualization tools are available that would provide value, but require new data or changes to data formats. Target Audience This guide is intended for transportation agencies—state departments of transportation (DOTs), Metropolitan Planning Organizations (MPOs), or local agencies—seeking to improve the efficiency and effectiveness of how they collect and use data for TPM. It can be used to review and assess existing data-related activities. The target audience includes • Staff with responsibility for TPM and/or individual performance areas (e.g., safety, pavement, mobility) who are seeking ideas for making effective use of data; • TPM staff seeking to gain a broader understanding of data sources and management methods; • Managers seeking to communicate the importance of data sharing and coordination to improving performance and strengthening accountability; • Managers seeking materials to help change their agency culture so that people see data as an enterprise asset; and • Data analysts seeking to understand the entire data life cycle and how what they do fits into a bigger picture. Introduction 3

Management and Use of Data for Transportation Performance Management: Guide for Practitioners Introduction • Foundation • Reporting • Insight • Cases • New data management practices are being implemented agency-wide that offer opportunities to re-think how performance data are collected and processed. • Changes to organization or staffing of TPM functions have taken place, and the new manager is looking to make improvements. • New or modified federal or state requirements for performance reporting are issued. While this guide draws many examples related to the federally defined TPM areas (safety, pavement, bridge, and system performance), it does not provide official guidance for MAP-21/FAST Act target setting or reporting. It is intended to provide a framework for assessing current data management practices and a source of ideas for practice improvement. Its purpose is to promote practices that will enable agencies to go beyond meeting reporting requirements and to get valuable insights from data that can be used to boost agency results. Research Report Available This guide is a product of NCHRP Project 08-108, “Developing National Performance Management Data Strategies to Address Data Gaps, Standards, and Quality.” A final research report for this project is available that includes a detailed literature review and practitioner interview results that were used to shape this guide. 4

Introduction • Foundation • Reporting • Insight • Cases Relationship to Other TPM Guidance This guide provides an overview of data management practices for TPM. It is not a comprehensive reference on working with TPM data and does not go into depth on individual data sets or technical management processes. It is intended for use in conjunction with other available references. To understand how the material in this guide fits with the bigger-picture TPM processes, the FHWA TPM Toolbox provides a useful framework, which is illustrated in Figure 1. This guide builds on two components of the FHWA TPM framework: C-Data Management and D-Data Usability and Analysis. It also acknowledges the essential role of organizational and cultural factors in successful TPM, illustrated by the outer circle (component A) that encompasses all other elements of TPM. Institutional barriers are often much harder to overcome than technical barriers in making improvements to data management and utilization. It is important to have strong leadership and buy-in across the organization for any improvement initiative. The FHWA TPM Toolbox includes a maturity assessment that agencies can use to gauge the state of their organization and chart a future course. See the end of this chapter for a link to the TPM Toolbox and additional reference material on TPM and transportation data assessment and management. Scope of the Guide This guide provides a high-level view of data management practices for TPM. Readers are encouraged to consult the references provided at the end of each section for further information on topics of interest. 5

Management and Use of Data for Transportation Performance Management: Guide for Practitioners Introduction • Foundation • Reporting • Insight • Cases Figure 1. The FHWA TPM framework. 6

Introduction • Foundation • Reporting • Insight • Cases Framework for Improving Data Utilization in TPM The framework for this guide is based on a cyclical model of a generic data management process, depicted in Figure 2. Figure 2. The data management cycle. This cycle begins with specifying and defining data requirements, and then proceeds to obtaining the data, storing it in one or more repositories, and processing it as needed to support use. Then, data are shared in various forms, analyzed and used for decision making, and communicated to different audiences. Introduction 7

Management and Use of Data for Transportation Performance Management: Guide for Practitioners Introduction • Foundation • Reporting • Insight • Cases Figure 3 shows the organizing framework for this guide. It extends this cyclical model of data management to illustrate the process of utilizing data for TPM. Figure 3. Framework for improving data utilization for TPM. The left side of Figure 3 shows how data can be used within TPM. This provides the motivation and the requirements for the six data management processes shown on the right side of the diagram. Every investment in data—whether it is collecting new data or improving reporting tools—should be evaluated based on how it helps the agency to better identify needs and solutions, prioritize projects, allocate funding, manage real-time performance, enhance accountability, and/or meet reporting requirements. The bottom row of Figure 3 identifies three fundamental processes for making use of data for TPM: Foundation, Reporting, and Insight: • Establish a data Foundation—defining performance measures, identifying data requirements, selecting data sources, and obtaining the data. 8

Introduction • Foundation • Reporting • Insight • Cases • Set up Reporting processes—establishing databases and reporting systems and publishing data and reports. • Gain Insights for decision making—analyzing performance data to understand trends and explanatory factors/root causes, developing predictive capabilities, setting performance targets, and communicating the agency’s “performance story” to different audiences. The numbered columns of Figure 3 show how each of these processes are broken down into more specific steps to specify, obtain, store and manage, share, analyze and use, and communicate data for TPM. Establish a Data Foundation Step 1: Specify & Define Data. Determine what types of data are needed, how data will be used within TPM business processes, and— based on this—specify attributes, scope, level of spatial and temporal granularity, and frequency of updates. Step 2: Obtain Data. Acquire the data needed for calculating performance measures and for understanding trends and root causes of performance results. Set Up Reporting Processes Step 3: Store & Manage Data. Set up data repositories either within the agency or “in the cloud”; load, validate, clean, and integrate the data; document the data for both technical and business users; and manage access to the data—to both protect it from unauthorized use and to ensure that it is accessible to those who need it. Step 4: Share Data. Put the infrastructure in place to produce data products (e.g., reports, maps, interactive portals) and share data across business units within the agency, with partner agencies, and/or with the general public. Gain Insights for Decision Making Step 5: Analyze & Use Data. Configure and use various data analysis tools to understand trends, predict future performance, and formulate performance targets. Introduction 9

Management and Use of Data for Transportation Performance Management: Guide for Practitioners Introduction • Foundation • Reporting • Insight • Cases Step 6: Present & Communicate Data. Translate data and analysis results into information that can be understood and used by different data customers. It is important to note that getting data right for TPM can take time and experimentation. While these steps are numbered and generally follow a linear order, they may be carried out in parallel, in a different sequence, or in an iterative fashion. For example, in the process of creating data visualizations (Step 6), one might identify needs for additional data (Step 2) or improved data cleansing (Step 3), or perhaps creation of an additional performance measure (Step 1). Common feedback loops are illustrated in Figure 3 with arrows connecting different steps along the top: • Steps 5 and 6 are often accomplished iteratively—analysis of data is conducted in preparation for presentation, but the process of presenting and communicating data may lead to further analysis to answer questions that arise. • Data analysis in Step 5 may identify needs for changes in how data and performance measures are specified. • Review of data after it is acquired may also uncover gaps or limitations that were not anticipated, and they may lead to the need for changes in data or performance measure specification. Nevertheless, this simplified framework can be used to consider all of the steps needed to make effective use of data for TPM and identify areas for improvement. The remainder of this guide provides guidance for each of the six steps and presents a set of case study vignettes that illustrate various aspects of data utilization for TPM. Note that this guide does not attempt to be comprehensive in its treatment of data management or TPM practices given the extensive available literature on these topics. It focuses on key points that agencies should keep in mind when considering practice improvements. Guidance for each step includes • A discussion of what is involved in implementing each step and some of the critical choices to be made • A synthesis of key points in the form of “Do’s and Don’ts” • Checklists that can be used to assess agency capabilities and identify opportunities for improvement 10

Introduction • Foundation • Reporting • Insight • Cases o For each step, checklists for “basic” and “advancing” capabilities are provided. The intent is that agencies focus first on getting the “basic” capabilities in place, and then seek to further advance their practices. A set of pull-out worksheets are provided at the end of this guide with these checklists that can be duplicated and used to assess agency capabilities. • References that provide further information related to the step For more information... 1. FHWA TPM Toolbox Guidebook: https://www.tpmtools.org/guidebook/ Maturity Assessment: https://assessment.tpmtools.org/ 2. NCHRP Report 866: Return on Investment in Asset Management Systems and Practices (TRB, 2018) https://www.nap.edu/catalog/25017/return-on-investment- in-transportation-asset-management-systems-and-practices 3. Applying Archived Operations Data in Transportation Planning: A Primer (FHWA, 2016) https://ops.fhwa.dot.gov/publications/fhwahop16082/ 4. NCHRP Report 814: Data to Support Transportation Agency Business Needs: A Self-Assessment Guide (TRB, 2015) http://www.trb.org/Main/Blurbs/173470.aspx 5. Performance-Based Planning and Programming Guidebook (FHWA, 2013) https://www.fhwa.dot.gov/planning/performance_based_pla nning/pbpp_guidebook/ 6. Benefit-Cost Analysis of Investing in Data Systems and Processes for Data-Driven Safety Programs (FHWA, 2012) http://safety.fhwa.dot.gov/rsdp/downloads/bcareport.pdf Introduction 11