National Academies Press: OpenBook
« Previous: Summary
Page 10
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 10
Page 11
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 11
Page 12
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 12
Page 13
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 13
Page 14
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 14
Page 15
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 15
Page 16
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 16
Page 17
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 17
Page 18
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 18
Page 19
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 19
Page 20
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 20
Page 21
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 21
Page 22
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 22
Page 23
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 23
Page 24
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 24
Page 25
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 25
Page 26
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 26
Page 27
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 27
Page 28
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 28
Page 29
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 29
Page 30
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 30
Page 31
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 31
Page 32
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 32
Page 33
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 33
Page 34
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 34
Page 35
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 35
Page 36
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 36
Page 37
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 37
Page 38
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 38
Page 39
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 39
Page 40
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 40
Page 41
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 41
Page 42
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 42
Page 43
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 43
Page 44
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 44
Page 45
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 45
Page 46
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 46
Page 47
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 47
Page 48
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 48
Page 49
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 49
Page 50
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 50
Page 51
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 51
Page 52
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 52
Page 53
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 53
Page 54
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 54
Page 55
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 55
Page 56
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 56
Page 57
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 57
Page 58
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 58
Page 59
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 59
Page 60
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 60
Page 61
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 61
Page 62
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 62
Page 63
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 63
Page 64
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 64
Page 65
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 65
Page 66
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 66
Page 67
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 67
Page 68
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 68
Page 69
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 69
Page 70
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 70
Page 71
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 71
Page 72
Suggested Citation:"Volume 1 - Research Report." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 72

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

V O L U M E 1 Research Report

C O N T E N T S V O L U M E 1 Research Report I-15 Chapter 1 Introduction I-15 1.1 Project Purpose I-15 1.2 Project Approach I-15 1.3 Organization of the Report I-17 Chapter 2 Performance-Based Resource Allocation I-19 2.1 Current Practice Among State DOTs I-22 Chapter 3 Performance Targets I-22 3.1 The Role of Targets in Performance-Based Resource Allocation I-28 3.2 Factors Influencing Target-Setting I-35 3.3 Approaches for Target-Setting I-45 3.4 Role of Economic Models and Management Systems in Target-Setting and Tradeoff Analysis I-54 3.5 Topic Areas for Volume II Guidance I-55 Chapter 4 Data Stewardship and Management I-55 4.1 Introduction I-56 4.2 Elements of Effective Data Stewardship and Management I-58 4.3 Organization and Governance I-62 4.4 Data Sharing I-64 4.5 Documentation and Reporting I-65 4.6 Technology I-70 4.7 Relationships to Target-Setting and Resource Allocation I-72 4.8 Summary of Success Factors and Obstacles I-74 4.9 Future Research

I-15 1.1 Project Purpose The overall scope of NCHRP Project 8-70, “Target-Setting Methods and Data Management to Support Performance- Based Resource Allocation by Transportation Agencies” is designed to assist public sector transportation agencies develop and improve performance management practices through the following three key objectives: • To provide an overall description of performance-based resource allocation (PBRA); • To provide a comprehensive description of the process and methods by which targets are set for use in PBRA; and • To provide a comprehensive description of the data, infor- mation systems, and institutional arrangements needed to support PBRA decision-making. 1.2 Project Approach The report proceeded through three phases. In Phase 1, the research team accomplished the following objectives: • Describe the purpose, desired outcomes, and essential elements of PBRA; • Develop a list of public and private organizations reviewed that use all or part of the performance management process, including examples of how organizations apply each element but with a focus on target-setting; • Identify the fundamental differences and similarities between objectives for a public sector agency and a private sector organization; • Develop the criteria that will be used for selecting case stud- ies and the form that case study documentation will take; and • Prepare the questionnaire that will be used in case studies. Building from this foundation, Phases 2 and 3 were con- ducted simultaneously, providing a detailed description of the case studies selected in Phase 1 and focusing on the following: • Performance Targets: – The role of targets in PBRA; – Factors influencing target-setting; and – Approaches to target-setting. • Data Stewardship and Management: – Elements of effective data stewardship and management; – Institutional issues related to data stewardship and imple- mentation of data governance; – Data sharing, documentation, and reporting; – Key technical considerations that guide setting data requirements and data management system development; and – Summary of success factors related to effective data management systems and institutional relationships to support PBRA. 1.3 Organization of the Report This report is organized in three volumes. Volume I is the research report that focuses on the target-setting and data man- agement elements of the performance management frame- work. Volume II provides the actual guidance for target-setting and data management and is a freestanding document. Vol- ume III, which will be published on the TRB website as NCHRP Web-Only Document 154, is the case studies that provide the resource material upon which the study and guidance are based. Volume I This volume provides findings and conclusions from all three phases of NCHRP Project 8-70. It describes a Performance Management Framework within which both state Departments of Transportation (DOTs) and Metropolitan Planning Organi- zations (MPOs) can develop and implement a PBRA decision process. It focuses on two elements of the framework—target- setting and data management systems. Chapter 2 describes the Performance Management Frame- work which includes Goals and Objectives, Performance C H A P T E R 1 Introduction

Measures, Targets, Resource Allocation, Reporting Results, and Data. It also describes several emerging trends in U.S. transportation industry practice. Chapter 3 describes the role of targets in PBRA in both the public and private sector. It outlines the factors that influence target-setting and methods by which targets are actually set. Chapter 4 summarizes the ways data management systems and organizational units within a DOT are used to ensure the use of accurate, timely, high-quality data for decision-making purposes. Two dimensions are involved in this process. The first focuses on the technical challenges associated with data systems and the second focuses on the institutional and orga- nizational challenges associated with data stewardship and data governance. Volume II Volume II of this report provides guidance for state DOTs and MPOs on setting targets and establishing data systems to support PBRA. Chapter 1 provides a process that can be used by transporta- tion agencies for developing and evaluating performance mea- sure targets. The process consists of the following six steps: 1. Establish a Performance Management Framework, 2. Evaluate the Factors Influencing Target-Setting, 3. Select the Appropriate Approach for Target-Setting, 4. Establish Methods for Achieving Targets, 5. Track Progress Towards Targets, and 6. Adjust Targets Over Time. Chapter 2 explains how transportation agencies can use data management and governance to strengthen existing perfor- mance measurement and target-setting programs. It applies the research conducted in Phase 3 into practical guidance for transportation agencies. The Guide is organized under the following headings: • 2.1 Establishing the Need for Data Management/Governance; • 2.2 Establishing Goals for Data Management; • 2.3 Assessing the Current State of Data Programs; • 2.4 Establish Data Governance Programs; • 2.5 Technology for Data Management; and • 2.6 Linking Data to Planning, Performance Measures, and Target-Setting Processes. Volume III This volume, which is published on the TRB website as NCHRP Web-Only Document 154, provides a description of the case studies to which the guidance in Volume II is linked. The list of organizations interviewed and provided as case stud- ies in Volume III is shown in Table 1.1. The case studies serve as specific examples which illustrate the use of performance targets and data support systems for resource allocation within the Performance Management Framework. I-16 Organization Organization Type City of Coral Springs, Florida Hennepin County Public Works U.S. Army ARDEC GASB Statement 34 U.S. DOT Japan Ministry of Land, Infrastructure, and Transport Atlanta Regional Commission Metropolitan Transportation Commission Orlando-Orange County Expressway Authority Port Authority of New York/New Jersey Kansas State Department of Education ABC Logistics Corporation X Do-it-Yourself Retailer Multinational Conglomerate RCG Information Technology Florida DOT/State Government Minnesota DOT Ohio DOT Washington DOT/State Government Maryland DOT (State Highway Administration, Motor Vehicle Administration, Maryland Transportation Authority) Local Government Local Government Federal Government Federal Government Federal Government International MPO MPO Toll Authority Port/Transit/Other Education Private Private Private Private Private State DOT State DOT State DOT State DOT State DOT/Transit/Other Table 1.1. NCHRP Project 8-70 case study organizations.

I-17 Responding to trends in the 1990s that placed greater empha- sis on public-sector accountability for more effective per- formance, state and local transportation agencies initiated a number of programs such as transportation asset manage- ment, performance-based planning, management and budget- ing, and maintenance quality assurance, among others. These efforts promoted a policy-driven, performance-based approach to managing and developing infrastructure that encour- aged identification of alternative solutions, use of economic methods (e.g., minimization of life-cycle costs and benefit/ cost analysis), explicit consideration of tradeoffs in program resource allocation, use of quality information through for- mal data collection programs, and applications of automated decision-support systems. They also promoted definition of performance measures and targets based upon stated policy objectives and input from public outreach. These efforts coin- cided with innovations in planning, programming, and more flexible funding in Federal surface transportation legislation beginning with the Intermodal Surface Transportation Effi- ciency Act (ISTEA 1991), as well as growing interest in different approaches to solving transportation problems through greater understanding and use of transportation operations solutions and associated intelligent transportation systems (ITS) technol- ogy. Documents such as the Federal Highway Administration’s (FHWA) Asset Management Primer; American Association of State Highway and Transportation Officials’ (AASHTO) Transportation Asset Management Guide [a product of NCHRP Project 20-24(11)]; NCHRP Report 545: Tools for Asset Management Tradeoff Analyses; and NCHRP Report 551: Perfor- mance Measures and Targets for Transportation Asset Manage- ment reinforced these approaches among practitioners. These studies laid out the concepts, procedures, and infor- mation needed for a performance-based approach to resource allocation and many state and local agencies have benefited from improved decision processes resulting from their adop- tion and implementation. At the same time, a number of other agencies still find it difficult to adopt such an approach in their transportation programs. As a result, recent and ongoing research shows considerable variability in agency resource allocation capabilities and practices nationwide. Some agencies have the tools needed for performance-based asset management, e.g., well developed statements of policy objectives and performance targets, modern infrastructure management systems, data collection and processing proce- dures to support both performance monitoring and manage- ment system application, and an organizational culture that supports strong communication and coordination horizontally and vertically. In contrast, other agencies lack even basic ele- ments, such as inventories of assets and defined performance measures. Furthermore, a vicious cycle ensues: agencies that lack the data and analytic tools needed to show a declining con- dition and performance of their assets find themselves unable to justify spending more (in staff effort as well as dollars) to improve their asset management processes that would enable them to do better, PBRA. In spite of uneven implementation among state depart- ments of transportation, performance management has been evolving steadily into an effective business process that links organizational goals and objectives to resources and results. Performance measures, and their attendant targets, are the lynchpin in this process. They are the link connecting goals to specific investments. The methods, including underlying data support systems, by which the measures and targets are estab- lished, play a critical role in the overall success of a public agency or private company. PBRA takes place within an overall Performance Manage- ment Framework, depicted in Figure 2.1, which is comprised of six basic elements described in the following paragraphs. Establish Goals and Objectives. PBRA decisions are anchored in a set of policy goals and objectives which identify an organization’s desired direction and reflect the environment within which its business is conducted. For example, many state DOTs have well-defined goals for the transportation system, C H A P T E R 2 Performance-Based Resource Allocation

including infrastructure condition, level of service and safety, as well as goals reflecting economic, environmental, and com- munity values. Likewise the private sector frequently establishes policy goals to guide production of products and services while defining the environmental and community context for its investment decisions. Select Performance Measures. Performance measures are a set of metrics used by organizations to monitor progress towards achieving a goal or objective. The criteria for selecting measures often include the following: • Feasibility, • Policy sensitivity, • Ease of understanding, and • Usefulness in actual decision-making. Identify Targets. Targets are a quantifiable point in time at which an organization achieves all or a portion of its goals. These points set a performance level for each organizational measure, such as achieving a 25 percent reduction in highway fatalities by 2030. The methods used to set such a target include the following: • Establish a Performance Management Framework, • Evaluate the Factors Influencing Target-Setting, • Select the Appropriate Method(s) for Target-Setting, • Establish Methods for Achieving Targets, • Track Progress Towards Targets, and • Adjust Targets Over Time. Allocate Resources. The allocation of resources (time and money) is guided by the integration of the preceding steps into an organization’s planning, programming, and project devel- opment process. To the extent possible, each investment cate- gory is linked to a goal/objective, a set of performance measures, and a target. Specific investment proposals are defined in rela- tion to specific targets. Measure and Record Results. The data for each per- formance measure must be regularly collected and period- ically analyzed. The analysis should indicate how close the organization is to achieving its targets and identify the actions necessary to improve results. Many public and pri- vate sector organizations have tracking systems in place to monitor performance allowing senior staff to make periodic budget adjustments. Create Data Management Systems. “Good” data is the foundation of performance management. Effective decision- making in each element of the performance management framework requires that data be collected, cleaned, accessed, I-18 Goals/Objectives Performance Measures Target Setting Evaluate Programs and Projects Allocate Resources Budget and Staff Measure and Report Results Actual Performance Achieved Quality Data Figure 2.1. Performance Management Framework.

analyzed, and displayed. The organizational functions that produce these requirements are called data management sys- tems. There are two key dimensions to creating and sustain- ing these systems. The two areas are equally important and must be synchronized within an organization to ensure the generation and use of accurate, timely, and appropriate data. The first area centers on the technical challenges associated with data systems, including development and maintenance of hardware and software, and the specifications for data col- lection, analysis, archiving, and reporting. The second area focuses on the institutional issues associated with data stew- ardship and data governance. 2.1 Current Practice Among State DOTs Within transportation the application of performance mea- sures has an especially long history. These historical roots are likely due in part to the fact that transportation programs deal with engineered facilities and ongoing operational services that have been supported by well-established funding mecha- nisms, a strong research culture, and extensive statistical reporting. What has changed today, however, is the context in which performance monitoring and reporting are con- ducted, and to what purposes. There is a much stronger emphasis today on the need to demonstrate responsiveness to customer needs, accountability for program expenditures, and provision of satisfactory levels of service or results. Well- executed performance-based management and reporting helps an agency to maintain credibility with executive and legisla- tive bodies in justifying requested budgets and demonstrating wise use of public funds. This current study is at the focal point of the following emerging trends in U.S. transportation industry practice: • Increased use of performance measurement in policy, plan- ning, and programming. DOTs and other transportation agencies are learning how to apply performance measures to their resource allocation decisions and to understand the elements and best practices needed to implement per- formance measures successfully as part of their day-to-day management. Nonetheless, while applying performance measures is conceptually straightforward, there are many details. The simplicity of the performance measurement idea belies the challenges and the levels of effort and commitment that are actually involved in putting the idea into practice. • Formalization of asset management concepts and principles drives agencies towards a policy-driven, performance-based approach to resource allocation and utilization, explicit con- sideration of alternative solutions, analyses of tradeoffs in terms of cost versus performance, and project and program evaluation methods and criteria that, through the use of performance measures, are consistent with policy objec- tives and targets. • Development and application of maintenance quality assur- ance programs, which entail explicit maintenance levels of service that, like performance measures, can serve as both indicators of the current condition/performance of main- tained features in the transportation system and expressions of target values for improvement. Several DOTs have successfully moved concept into practice by incorporat- ing maintenance levels of service within performance-based budgeting tools for their maintenance programs. • A renewed focus on analytic tools and other information technology resources that are needed for performance- based management. Data collection and processing, making needed information accessible to all organizational levels across the agency, implementing useful management sys- tems and other analytic tools, and aligning systems and data with an agency’s business processes are critical to effective decision support for policy-making, planning, and resource allocation. Cost-effective data collection procedures and technology, updated organizational responsibilities for managing shared data, taking the best advantage of the sub- stantial investment in legacy systems while providing new capabilities where needed, and ensuring the quality and accessibility of data to a diverse set of agency and stakeholder users are some of the key challenges that are now being dealt with by DOTs in this area. • An increased focus on the development of a risk manage- ment plan as part of a performance measure program. While there was some consideration of the need for a formal risk management plan before the Minnesota bridge event, assess- ing risk across the enterprise has become a higher priority.1 Ways in which agencies have responded to these trends using performance-based management are summarized in a recent report by AASHTO’s Performance-Based Highway Program Task Force titled A Primer on Performance-Based Highway Pro- gram Management: Examples from Select States.2 The report pro- files the experiences of DOTs in 11 states: California, Florida, Maryland, Michigan, Minnesota, Missouri, Montana, New York, Ohio, Virginia, and Washington. Agencies’ functions are described in one or more of the following management areas: • Policy development and long-term planning; • Programming and budgeting; I-19 1Eighth National Conference on Transportation Asset Management (http:// pressamp.trb.org/conferences/programs/program.asp?event=486). 2A Primer on Performance-Based Highway Program Management: Examples from Select States. American Association of State Highway and Transportation Officials, Washington, D.C., January 2008. Accessed March 13, 2008. http://www. transportation.org/sites/quality/docs/PeformanceBasedHighwayProgram_Jan 2008.pdf.

• Program, project, and service delivery; • System operations; and • Monitoring and reporting results. The ways in which performance measures are engaged by these functions involves the following elements of comprehen- sive performance management: • To select appropriate measures to gauge agency perfor- mance in critical program and service areas; • To track and report actual results and, where appropriate, to compare to stated objectives, targets, or benchmarks; • To analyze results to identify key factors influencing per- formance and opportunities for improvement; • To allocate resources and operate transportation systems to drive better results; and • To continue to monitor and report progress. The AASHTO Primer also notes that performance measure- ment is not a silver bullet and does have inherent limitations. For example, not all aspects of transportation system perfor- mance are under the control of an owning/operating agency. Performance data may signal a change in some aspect of trans- portation system condition or operation but do not necessar- ily indicate the cause of that change. Some important aspects of performance are not easily measured. Even where perfor- mance outcomes can be measured, that information may be only a part of what decision-makers need in their deliberations. Nonetheless, the value of performance measurement as part of the total package of information available to an agency has been well validated in the experiences of several of the DOTs that have been described in the AASHTO Primer. Profiles of each of the 11 state DOTs are presented in the AASHTO Primer in this context. The AASHTO Primer observes that the 11 DOT profiles demonstrate the following benefits of comprehensive performance-based management: • More efficient allocation of increasingly scarce resources; • Development and justification of budget and project pro- posals; and • Accountability to road users and the general public for deci- sions in funding, constructing, maintaining, and operating the highway system. Given the objectives and perspectives of this study, these DOTs that appear to have gone the furthest in applying performance-based thinking to investment decisions, pro- gramming, and resource allocation illustrate several elements of best practice: • The application of performance measures throughout the agency that are integrated vertically, horizontally, and among processes. Process-related integration implies top- to-bottom consistency of performance measurement con- cepts, tools, and measures throughout several key functions: e.g., policy formulation, data collection/analysis, long- range and short-term planning, programming/budgeting/ resource allocation, program/project delivery, and system monitoring/feedback. Horizontal integration implies consis- tency in performance measurement across agency divisions and business units. Vertical integration implies consistency in performance measurement (though at different levels of detail) among levels of management. Several agencies described this approach as holistic. • The application of performance measurement in a system- atic, documented way. Performance measurement is embod- ied in a number of documents that represent a progression of thinking. Each document marks the successful completion of findings for a particular function and provides guidance for the subsequent function. Long-term plans, short-term components, work programs, STIP documents, approved budgets with project lists, and performance monitoring/ tracking reports are examples of a systematic approach that enforces the consistent, integrated processes of the previous bullet. • Strong executive/managerial support and involvement in performance reviews and decisions on reallocating resources, in central and district offices, as well as among program and key business unit managers. • Recognition that performance measurement can involve a culture change within the agency, with steps taken to focus on the positive aspects of this change while mitigating the potentially negative aspects. • Transparency of performance results and their implications for transportation customers and stakeholders, as well as the owning/operating agency. State DOTs take different approaches to this need—e.g., some favor devices such as dashboards, report cards, or score cards, while others pre- fer a more narrative or descriptive communication. All of these approaches appear to be successful in the context in which they are used. The important point is the recogni- tion that technical data that characterize transportation network conditions, operations, work needs, services, and programs need to be translated into a form that different audiences understand and can respond to. • Several agencies look at organizational performance, as well as transportation system performance, and a subset of these links the two concepts: i.e., transportation system perfor- mance influences the performance evaluation of agency business units and employees. This concept has existed for some time in the private sector but now is being considered by public sector DOTs. NCHRP Project 20-60 considered performance measures and targets in the context of asset management within state I-20

DOTs.3 Asset management is by definition a policy-driven, performance-based process that stresses, among other attri- butes, the use of quality data and analytic tools, including predictive models. Management principles, methods, and decision criteria that are rooted in asset management thus per- mit a long-term view of options and their consequences, in addition to programming and budgeting in the medium- and near-term. Performance measures and targets are critical to asset management applications in policy formulation, public outreach, planning, programming, budgeting, resource allo- cation, program and service delivery, operations and mainte- nance, transportation system monitoring, and assessment of needed updates in policies and priorities. NCHRP Report 551 tabulates examples of performance measures in use by state DOTs and summarizes previous work on performance mea- surement, as well as prescribing a method for target-setting. Ultimately, performance measures are used for many pur- poses in state DOT, including the following: • Provide a foundation for policy formulation and systemwide planning. • Issue a report card: “How are we doing?” • Track progress towards public and explicitly stated policy goals and agency priorities. • Support investment decision-making in resource alloca- tion, performance-driven investment decisions, formalized performance-based budgeting, and strengthened internal program management. • Provide the basis for quantification of program benefits and other impacts as part of investment decision-making, analysis of tradeoffs, and communication to stakeholders in support of program investments. • Demonstrate accountability and responsiveness to stake- holders, ensuring “wise use of tax dollars.” • Assess the status of a program, evaluating its cost- and performance-effectiveness. • Meet or respond to Federal and state legislative mandates and reporting requirements. • Guide improvement of delivery of services, focusing on desirable outcomes and alternative methods of delivering these results. • Engage an agency within a comprehensive, statewide per- formance initiative aimed at broad-based improvement in government delivery of services and accountability. • Improve communication within the agency itself as well as with transportation system users, political leaders, other stakeholders, and the public at large. While many DOTs and their state governments apply one or more of these aspects of performance-based management, only some apply PBRA specifically, including target-setting. I-21 3Cambridge Systematics, Inc., PB Consult, Inc., and Texas Transportation Insti- tute. NCHRP Report 551: Performance Measures and Targets for Transportation Asset Management. TRB, National Research Council, Washington, D.C., 2006.

I-22 3.1 The Role of Targets in Performance-Based Resource Allocation Introduction Performance management is a business process that links organization goals and objectives to resources and results. Per- formance measures and their corresponding targets are the lynchpin in the process. They provide the direct link between the stated goals of an agency and the effectiveness of its invest- ment decisions in reaching those goals. Performance measures, used along with well-defined and well-communicated tar- gets, provide transparency and clarity to the resource allocation decision-making process. Targets provide the critical context for evaluating the effectiveness of investment decisions. For example, a performance measure will define how an invest- ment decision will be evaluated in terms of its impact, in absolute terms; in fact, performance measures are often referred to as evaluation criteria. The corresponding target provides the perspective for evaluating the impact of the investment decision in relation to the desired end-state, i.e., how significant is a particular investment in helping an agency attain a particular goal. Targets provide the means in which the relative effec- tiveness of a particular investment decision can be clearly communicated. Because targets play such an important role in PBRA, this study focused on the factors that influence target selection and the approaches by which targets are actually established. Towards that end, the case studies included validation of the seven-step process for setting targets found in NCHRP Report 551. The steps are as follows: • Step 1—Define Contexts and Time Horizons. This ini- tial step involves developing explicit statements about how targets will be used and what time horizons they will cover. • Step 2—Select Scope of Measures for Targets. This step involves identifying the performance measures that are suitable for target development. Some measures may not lend themselves to quantitative targets; others may not have sufficient baseline or trend information available for the agency to be comfortable with establishing a target. • Step 3—Develop Long-Term Goals. A distinction is made between long-term goals about desirable performance lev- els and short-term targets that represent the best that can be done given resources. • Step 4—Consider Funding Availability. This step involves creating realistic estimates of future resources that can be used as the basis for financially constrained performance targets. • Step 5—Analyze Resource Allocation Scenarios and Tradeoffs. The performance implications of different resource allocations are analyzed both within and across program categories. Use of analytic tools that project future performance as a function of investment level is fundamental to this activity. • Step 6—Consider Policy and Public Input. The process is supported by two-way communication between the agency which provides easily understandable information about the implications of different resource levels and stakeholders providing their feedback on desired performance levels and priorities across different measures. • Step 7—Establish Targets and Track Progress. This final step involves selecting target values for performance mea- sures and putting the procedures in place to track progress towards achievement of targets. While these basic steps were validated by many of the case studies, the actual use of the steps varied among organizations. In addition, several factors used in setting targets, which are discussed in the next chapter, need to be explicitly linked to the seven steps. This linkage is made in Volume II—Guide for Target-Setting and Data Management. C H A P T E R 3 Performance Targets

Public Sector Experience As evidenced by many of the case study results summarized in this report, the use of specific targets by agencies using PBRA is still somewhat limited. When it does exist, not all of the steps listed in the previous section are taken as part of the process. The robustness of the approach is largely dependent on the following: • The method used to develop targets—internally developed by agency staff or developed via stakeholder outreach/ planning team consensus process; • Amount of time available for planning a comprehensive performance-based approach—the more time available, the greater chance an agency has to think through each of the steps defined in the previous section; and • Support by management for agency staff to conduct the exercise and analyses needed to support a well-thought-out performance-based approach. For transportation agencies at the state, regional, and local level, target-setting is most often seen in relation to asset management systems (e.g., bridge and pavement) where a strong data resource is available for infrastructure condition, collected by almost all state DOTs and local juris- dictions over the last 10 years, in part because of Governmen- tal Accounting Standards Board Statement 34 (GASB34). The amount of data resulting from the GASB34 financial reporting requirements has reinforced performance-based infrastructure management processes and has provided a strong foundation for understanding the relative impact of various resource allocation levels on infrastructure condi- tion over the years. This, in turn, has enabled a strong analytic process in which trends can be reasonably extrap- olated into the future and meaningful performance targets can be set. For many agencies reviewed, the PBRA process involves allocating resources based on how well each potential invest- ment performs in relation to other potential investments, or in a more general sense, whether it provides a positive or neg- ative impact in relation to one or more evaluation criteria. This is often the case, as opposed to evaluating the perfor- mance of an investment in relation to a more discrete target. More often than not, this is due to the financial and staffing resource constraints that most transportation agencies are under, which impact the following: • The ability to collect the data needed to track progress toward meeting targets; • The ability to develop new technical tools to project future performance as a function of varying levels of investment; and • The need to avoid setting and communicating to the pub- lic and transportation stakeholders unattainable targets, in light of significant financial constraints. Where the data and the technical resources exist, trans- portation agencies are using targets as a way to measure the effectiveness of particular investment decisions, in relation to transportation goals, most often as a means to support addi- tional funding for a particular investment type or program, or to justify the cost of investments already programmed for funding. In this sense, target-setting becomes a strong account- ing tool for decision-making authorities. Several of the state case studies provide excellent examples of the role that performance measure targets play in PBRA, espe- cially the selection of projects for the state highway construction program. The Minnesota DOT (Mn/DOT) case study, how- ever, provides one of the best examples because that agency has done the following: • Recently updated its 2003 statewide performance-based transportation plan; the 2009 version contains changes in performance measures and targets which not only illus- trate the evolution of performance management but also provide an example of the significant funding infusion states need to meet their performance targets; • Implemented performance management during three sep- arate administrations; • Developed an annual “Snapshot” which compares actual performance goals to targets; • Adopted a performance-based formula for distributing state and Federal highway construction funding to its dis- trict offices; and • Secured new state funding by describing highway system needs within the context of a performance management framework that sets targets for most of its performance measures. After completion of the 2003 Statewide Transportation Plan, Mn/DOT integrated PBRA into its highway planning, programming, and project development process. Each year every Mn/DOT district, following uniform guidance, identifies investment priorities. These priorities are based on quantifiable performance measures and targets that establish an impartial statewide basis for identifying critical transportation improve- ments for the entire trunk highway system. This process, first identified in 2003 and refined over the last 5 years, is illustrated in Figure 3.1. This five-step investment process, described in the 2009 Statewide Transportation Plan, provides the framework and guidance for developing Mn/DOT district 20-Year highway investment plans. These 20-year investment plans, newly updated in 2009, provide the link between the policies and I-23

strategies established in the Statewide Plan and the capital improvements that are made to the State highway system. Together, the eight district plans constitute a State 20-Year Highway Investment Plan for 2009–2028. The plans were devel- oped in accordance with the following five steps which are described in detail in the case study write-up in Volume III and which differ from the seven steps in NCHRP Report 551. Step 1—Identification of Investment Needs. Investment needs fall into two categories: improvements to address sys- tem performance and improvements to address regional or community priorities. Performance-based needs include investments to meet established system performance targets related to traveler safety, infrastructure preservation, inter- regional corridor mobility, Twin Cities mobility, and Greater Minnesota urban mobility. The analytical models and method- ologies used to calculate the investments to meet these system performance targets are described more fully in the District Plan Summary section of the Statewide Transportation Plan. Regional priorities include a wide range of highway improve- ments to support local business or community development goals, from major highway expansions and new interchanges to intersection modifications, trails, and sidewalks. These regional priorities ($3 billion to $5 billion) illustrate the fact that there are many demands on available transportation funding beyond the investments needed to meet established statewide perfor- mance targets ($62 billion) in the next 20 years. Step 2—Project Future Revenue. Next, revenues were projected based on the trends in state and Federal revenue sources for state highway construction. No new sources of rev- enue were assumed but the increased bond funding for trunk highways enacted by the 2008 Legislature was factored into the projection. Construction cost trends also were analyzed and projected so that investment needs and expenditures could be estimated in year-of-construction dollars. A more complete description of revenue and cost trends and projections is provided in Chapter 5 of the Statewide Transportation Plan. Given the volatility in both costs and revenues and the cur- rent discussion of increased Federal infrastructure funding as an economic stimulus package, the projections assumed in the new plan represent a snapshot in time and will need to be updated annually as long-range investments become programmed in the four-year State Transportation Improve- ment Program. Step 3—Set Goals: A Balanced Program of Investments. The investment priorities reflected in the 2009 update of the District Plans differ significantly from the 2004 plans. At that time, Mn/DOT identified infrastructure preservation as its top priority, and districts were directed to fully fund preservation needs before other priorities, including safety, mobility, and local community priorities. The revenue and costs outlook in 2004 projected sufficient long-term funding to meet not only preservation needs but other areas of need as well. Between 2004 and 2008, revenues have not grown as antic- ipated, and construction costs have increased dramatically. Even with the increased transportation revenues provided through Minnesota law 2008, Chapter 152, the costs to fully preserve bridges, pavements, and other roadway infrastruc- ture during the next 20 years will exceed projected funding. The investment goals for the 2009 district plans reflect Chapter 152 legislative direction, consideration of system performance trends and stakeholder input. While infrastruc- ture preservation continues to be an important priority for Mn/DOT, it cannot be the exclusive priority. The goal for the 2009 District Plan updates is to lay out a balanced program of investments that achieves three objectives: I-24 Step 1 Identify Investment Needs Step 2 Project Future Revenues Step 3 Set Investment Goals Step 4 Develop Investment Plan Step 5 Prioritize Unfunded Investment Needs Investments to Meet Performance Targets Regional and Community Improvement Priorities Revenue Outlook Total Unfunded Statewide High Priority 20-Year Highway Investment Plan STIP 2009-2012 Mid-Range HIP 2013-2018 Long-Range HIP 2019-2028 Investment Goals: Balanced Program Legislative Direction (Chapter 152) System Performance Stakeholder Input Figure 3.1. Mn/DOT highway investment plan development process.

1. Supports the continued development of the statewide econ- omy and livability of Minnesota communities; 2. Represents the optimum allocation of projected revenues among the four strategic investment priorities of safety, mobility, infrastructure preservation, and regional and community improvements; and 3. Results in a consistent level of investment effort across districts towards statewide system performance targets, including the investment directions established in Chap- ter 152 for the rehabilitation or replacement of fracture critical and structurally deficient bridges and other high- way improvements. Step 4—Develop Investment Plan. Given the needs, pro- jected revenues and investment goals, each district developed investment plans for 2009–2028. The investment plans are divided into three timeframes: 2009–2012 STIP (State Trans- portation Improvement Program), 2013–2018 HIP (Mid- Range Highway Improvement Plan), and 2019–2028 LRP (Long-Range Highway Investment Plan). Investments identi- fied for the STIP include projects that have developed scopes and cost estimates. Investments identified for the HIP repre- sent very preliminary cost estimates subject to change as proj- ects are developed. The Long-Range Plan investments in the second 10 years represent general estimated investment levels in various improvement categories. The case study contains a table that summaries investment needs by strategic investment priority and planning period. Step 5—Prioritize Unfunded Needs. With a total esti- mated investment need exceeding $65 billion during the next 20 years and projected revenues of about $15 billion, Mn/DOT’s analysis indicates that almost $50 billion remains in “unmet needs” in present dollar value. To place this level of funding in perspective, every 5 cents on the motor vehicle fuel tax in Minnesota provides just under $100 million per year to the State Road Construction fund. To generate an additional $2.5 billion in revenue over 10 years would require the equiv- alent of a 12.5-cent increase in the state gas tax. The 2009 plan fully acknowledges that future transporta- tion funding will never be increased to meet this degree of “unmet need.” The plan’s policies and strategies, therefore, emphasize a new approach to meeting system improvement needs through stronger partnerships and innovation. This is especially evident in the plan’s vision for mobility in the Twin Cities, calling for more comprehensive and fiscally realistic approach to congestion mitigation. The plan also stresses the need to set priorities. Towards this end, Mn/DOT has identified five percent of the “unmet needs” as high-priority investment options should additional revenue be available during the next 10 years. Additional funding, such as a Federal economic stimulus bill, would likely carry specific eligibility criteria or investment direction. For this reason, the identified high-priority unfunded investments are distributed across all four strategic investment categories. Private Sector Experience Institutional Context for Private Sector Decision-Making Private and public sector goals are analogous in the sense that in both cases, a strategy drives decisions about organi- zation and processes. Organization and process decisions, in turn, drive a need for resources and help set targets. Finally, the gap between performance and the agreed targets is the ultimate basis for re-allocation of resources. Private companies set up transportation as either a profit center, a cost center, or a service center. This form of orga- nization drives the overall policy goals and objectives. A profit center would focus on maximizing profit. A cost center would focus on minimizing cost. A service center would focus on maximizing service. Acquisition, operation, and maintenance processes are organized to support the overriding goals and objectives. Acquisition, often of fleets and terminal equipment, is orga- nized to achieve targets such as return on investment, operating cost, and/or expansion. Transportation companies measure and use return on investment in order to decide whether to buy new equipment, build new hubs, or upgrade systems. Opera- tional processes are geared to meeting either cost or service tar- gets or both. Maintenance processes are usually geared to lowering operating costs. Private sector transportation compa- nies use the Operating Ratio (operating expenses divided by sales) as a primary measure of performance. Underneath this and other profitability metrics, private sector companies almost universally track operating metrics such as on-time arrivals and departures and root causes of delays. Information Technology (IT) processes are geared to sup- port operational processes and usually need to meet return on investment criteria. Radio frequency identification (RFID), Global Positioning Systems (GPS), Transportation Manage- ment Systems (TMS), fleet equipment, and terminals need to support the overall mission of the organization. Environmental regulations regarding noise, air, and water pollution constitute a minimum threshold for the allocation of resources, but private sector companies generally have a relatively ill-defined approach for setting resource alloca- tion decisions to meet social or environmental commitments. Seven approaches provide some insight into the wide range of approaches in current practice, which include: compliance with mandatory regulatory standards; compliance with voluntary industry standards; subjective evaluation; using last year’s per- formance as a benchmark; quotas; linkage to a profit-oriented goal; and increases in brand equity. Industry standards such as ISO 14001 offer companies benchmarks and standards I-25

for environmental stewardship. Subjective approaches can reward social or environmental awareness but rarely commit to any consistent or documented standard. This approach can be very motivational if the evaluation is positive, but it is subject to review, revision, and even reversal depending on the man- ager in charge. As with charitable giving, many private sector companies use last year’s contributions as a benchmark for next year’s level of resources allocation. Following the model of minority hiring, where performance is measured as a per- cent of sales dollars, some companies target earning or spend- ing a certain percentage of their sales dollars on social or political contributions. Many companies strive to find overlap between social and economic goals. The underlying premise is that the elimination of waste is consistent with cost reduc- tion. Therefore, “green” goals also are coincidentally profit- increasing. Finally, some companies participate in socially responsible spending because it increases their brand equity. Brand equity is frequently measured at large private sector com- panies, whether they are in consumer products or in transpor- tation. For example, the brand equity that can be gained by emphasizing that rail transport is “greener” than road haulage can measurably increase the shareholder value and the market penetration of a railroad. Also, there may be a difference between the way the private and public sector DOTs disburse funds, which may result in a difference in the method and the timing of data collection and aggregation and in the timing of allocation of resources. Private sector funds are budgeted in one cycle and released in another. Expenses are released via use of a purchasing card (p-card) or purchase orders (PO). Public sector funds may operate on a different release schedule than private sector. Corporate Planning and Control Processes PBRA is part of a broader corporate planning and control process. This process has four components: strategy, plan- ning, execution, and control and feedback (Figure 3.2). Each component consists of the following subprocesses: • The Strategy-setting process consists of having a mission, understanding the competitive landscape, and developing a strategy to be successful in that competitive environment. I-26 Metrics Benchmarks Targets Operation, including Outsourcing Processes Applications Hardware Backbone Performance versus Metrics Budget Mission Competitive Analysis Strategy Strategy Planning Execution Control and Feedback Source: Boston Strategies International. Figure 3.2. Corporate planning and control process.

• The Planning process consists of agreeing on or developing metrics that help track and sometimes predict success by referencing benchmarks from competitors or best-in-class companies that represent the state of the art in a given func- tion or process (for example, transportation management). • The Execution process includes organization, process man- agement, and information architecture. The organization structure includes a determination of which activities to out- source and which to perform in-house. Process manage- ment includes the definition of key business processes and codified sequences of activities. The IT architecture deter- mines the way in which information supports and/or shapes the organization and the processes. • The Control and Feedback process can consist of many management controls, but the primary instruments are the annual budget and cost accounting. When the feedback is good, the budget usually increases. When the feedback is bad, the budget usually decreases. Evolution and Current State of PBRA in Private Sector Companies The history and evolution of performance-based manage- ment is deep and rich at most private sector companies. Pri- vate sector companies have had metrics in place for as long as most senior managers have been at their companies, which in many cases is over a dozen years. The key incentive for achieving individual objectives is compensation. Another key incentive is the motivation gained from competing among business units in the same company and winning. That being said, not all private sector companies practice all five elements of the PBRA model. While the vast majority of companies have goals and track performance data, many do not set targets and many do not have explicit feedback mechanisms to allocate resources based on varying levels of performance. Here is how the five elements of the PBRA model often play out at major private sector companies: • Goal Setting. Goal-setting is typically initiated by the CEO and his or her top-level advisors such as the CFO. These flow to business unit heads and then on to operational staff. The goals are often, or are driven by, top-level financial impera- tives. These often seem arbitrary to people below the senior management level, who can perceive the goal as indirect or imprecise. • Metrics. Nearly every private sector company chooses met- rics to track. These can be either financial or operational, or a combination of the two. The historical legacy of the met- ric is as important as the precision with which it addresses the desired outcome. Organizations learn to interpret mea- surements over time, and if they are imprecise, they apply subjective judgment to the metric. Sometimes companies use the same metrics as their competitors, or as “best-in- class” companies, so they can compare levels of performance through benchmarking. • Targets. The prevalent practice in the private sector is to foster competition amongst business units so a natural tar- get evolves. An alternative is to set the target according to what other companies achieve. Using external benchmarks requires careful consideration of the similarity of the refer- ence organization to one’s own, as well as to ensure that both companies are computing the metric the same way. • Resource Allocation. Most private sector companies set individual compensation to vary according to the degree to which they reach the target. Other common mechanisms for allocating resources are the annual budgeting process and the capital expenditure authorization process. • Results Monitoring. Private sector companies invariably gather operational and financial data from the Enterprise Resource Planning (ERP) or legacy information systems, and make it accessible to users. Some make it easier than others. Some companies post the results to their intranet and have drill-down query capabilities, while others make users request reports from IT. As users’ computer skills increase and systems become more accessible and menu- driven, results are being monitored by a larger proportion of employees. This democratization of data is enhancing the ability to reach the target. Goals are often prioritized using the Keep It Simple, Stupid (KISS) Principle. They prefer one or two universally recog- nized goals to a dashboard of nuanced goals. Some companies, however, have a simple and intuitive hierarchy—safety, then financial results, then operational excellence. Targets are set on an annual basis in most private sector companies, and these are valid for one year. Although senior managers and the Finance department usually have long- range goals, most targets are set and performance monitored on an annual basis. To link individual performance to long- term goals, companies sometimes use equity as an incentive for employees rather than to share multiple annual targets with them. In the private sector, most companies’ processes operate such that every goal should have a target and every target should align with a goal, at least according to the companies interviewed for this study. Alignment of metrics and goals is so important that most companies prefer to reduce the num- ber of goals and targets rather than risk misalignment or con- fusion between goals and targets. Again, this reverts to what is a recurring theme of simplicity as an underpinning princi- ple of performance measurement and management. Realistic goals are essential to making rewards and penal- ties effective. While “stretch targets” used to be common, I-27

most companies have diverged from this approach. They now set goals that they believe can be realistically attained within the year. With realistic targets, the anticipated gains can be embed- ded in business unit or departmental budgets, and individuals can be more readily held accountable for their performance towards reaching the target. On the other hand, complex, con- fusing, or overly aggressive targets can be used as excuses for non-performance. The most common consequence of failure to meet targets is a less-than-favorable compensation review. This may or may not be coordinated with a performance improvement initiative at either the individual or the group or departmen- tal level. Job loss also is a possible consequence of a failure to hit targets. 3.2 Factors Influencing Target-Setting Public Sector Experience There are multiple factors that lend themselves to the devel- opment of a PBRA process within transportation agencies. These factors include such things as strong agency leadership and crisis situations (e.g., funding shortages or public concerns) that focus attention on agency decision-making processes, and others. These factors have been well documented over the last two decades in performance-based literature. This section focuses on factors most important to target-setting as an element of performance-based planning. Target-setting is a critical, yet often under-utilized aspect of PBRA. The factors influencing target-setting, which emerged repeatedly from the case studies developed as part of this study in Volume III, are summarized in the following section. Political/Legislative Influence Perhaps the most immediate and direct factor influencing target-setting as an element of PBRA is the existence of a commission or other political body to which a transportation agency must report the performance of investment decisions. Political intervention in the process may result from contro- versy, as seen in states such as Florida, Virginia, and Ohio that have said candidly that legislative criticism over project deliv- ery and fiscal forecasting compelled expansion of performance targets and accountability, or the increasing public outcry over transportation services that force political attention on an issue, as is the case in Atlanta, GA, where increasing congestion issues led the Governor to establish a congestion task force which ultimately recommended a congestion reduction target for the region. While political influence of this direct manner can have very complicated repercussions, it has shown to be one of the most positive indicators for implementation of target-setting. For almost every agency reviewed that is using targets as part of their PBRA process, political or legislative intervention provided the initial impetus for establishing discrete targets. Political intervention can be triggered by a number of issues, but the most common is the increasing limitation of trans- portation funding at all levels of government, which has cre- ated more competition for available funds and also made it more important to justify funding requests. Depending on the nature of the political influence, elected officials or legislators may direct agencies to develop a more objective process that includes measures and targets, as was the case in Minnesota, as a result of the Minnesota Legislature and Department of Finance requirement that agencies use per- formance measures in biennial budget documents. It was also the case in Washington State, in which the State’s Legislative Transportation Committee initiated a study in 1991 which resulted in recommendations for a PBRA process which is still in place, having been refined and improved over the last two decades. Political bodies may also establish targets themselves, as seen in California which legislated CO2 reduction targets that the Metropolitan Transportation Commission (MTC) has been addressing as part of its recent regional transportation planning activities or in Atlanta where the Governor’s Conges- tion Mitigation Task Force resulted in a recommendation that a travel time index target of 1.354 be adopted for the region. The latter situation, in which political bodies develop targets themselves, can be difficult to negotiate, if the process is not properly informed by knowledgeable transportation staff who will guide the development of reasonable, attainable targets. While political involvement can be challenging, it also can pro- vide transportation staff the support they need to select proj- ects that are proven to improve performance and therefore should be a priority for funding. This can often be a difficult situation for agency staff, working within any scale of govern- ment, that operate within a highly political arena where pres- sures to fund certain projects, regardless of merit (in relation to stated transportation goals), can be intense. Customer Service Focus Those agencies that have taken a clear customer-service approach to transportation planning, management, and design understand the need to use targets to be able to communicate to the system user—the “customer.” Customer satisfaction is a fundamental aspect of performance for these organizations. I-28 4The Travel Time Index (TTI) is the ratio of travel time in the peak period to travel time in free flow conditions (speed limit). For example, a TTI of 1.35 indicates a trip that takes 20 minutes in free flow conditions would take 27 minutes in the peak (35 percent longer). The TTI can be used as a performance measure to track changes in the regional transportation network and to regularly report to the pub- lic progress being made in improvement of the operation of the network.

They break down and analyze customer satisfaction or dissat- isfaction from broad perspectives that address issues in areas such as social and community impacts and environmental impacts of transportation investments, and in more narrow terms that address issues related to daily personal travel needs. Coral Springs, Florida, provides a very strong example of a municipality that has taken a very direct customer-service approach towards its resource investment process, in which customer input drives the decision-making process. Target- setting is a key component of the Coral Springs process as it provides a direct translation of progress, as feedback for a community that has been actively engaged and empowered as part of the planning process. The Coral Springs focus on customer satisfaction is per- haps the most important factor influencing target-setting, which can be seen most clearly in the private sector examples. As indicated in the ABC Logistics case study, customer service is the overriding objective in its resource allocation process, and its entire process is geared towards customer satisfaction. ABC Logistics exhibits all five elements of PBRA, including target-setting. Each defined goal has a target, with targets stemming from the promises made to its customers in their contracts with the logistics company. A Customer Care pro- gram monitors the company’s performance relative to the tar- gets and customer expectations, with a feedback mechanism to communicate results to the public and to link individual employee performance (and merit) to the performance of the company. Additional factors influencing target-setting in the private sector are described in the following section on “Private Sector Experience.” PBRA History/Evolution in State of the Practice Another key factor in target-setting is an agency’s perfor- mance management history. Agencies that are only at the beginning of implementing a performance-based process gen- erally have less complete and less sophisticated target-setting processes. In general, there is a typical evolutionary path that agencies follow. A corollary to this evolution is the emergence of an agency’s data sophistication. Agencies with a history of PBRA, including target-setting, provided the following insight into how best to incorporate target-setting into the PBRA process: • “Keep it simple!” Agencies that have a history with performance-based planning and application of targets in the resource allocation process understand the need to keep the target-setting aspect relatively simple and easy to understand. When adding the target-setting dimension to the PBRA process, it could easily become a much more difficult system to manage. However, if measures and their attendant targets are kept focused and straightforward and are communicated in terms that decision-makers and the public understand, they become useful tools. Keeping it simple can be viewed from the following perspectives: – Keep the measures and the targets simple. Do not make performance measurement an academic exercise; meas- ure what is important, do not measure everything. Too many measures and targets, with their own data systems and evaluation procedures, can create a cumbersome and unsustainable situation in which an agency is data rich but information poor. – Communicate performance in terms that are readily understood. Communicating targets in a manner that makes sense to the general public seems to be a strong indicator for the success of PBRA and the integration of target-setting. – This principle is exemplified by a number of agencies, including ARDEC, which tracks performance in 16 dif- ferent areas on stoplight charts with red, yellow, and green indicators; the City of Coral Springs, FL, which “rolls up” an extensive series of performance measures into 10 key composite measures, referred to as the city’s Stock Index, that summarize city performance at a glance; and Hennepin County, MN, which uses a Balanced Scorecard approach in which numerous measures are evaluated and tracked in terms of multiple perspec- tives (customer, finance, internal process, learning and growth) and simplified into tables of information provid- ing “warning lights” for areas in need of improvement. The “keep it simple” principle is particularly strong in the private sector. • Integrate performance measurement into daily agency prac- tice. This directs attention to key issues, promotes financial resources, and provides the ability to develop stronger PBRA systems. Some of the agencies reviewed have tied perform- ance of investments to staff-level performance, though it is more common among the private sector cases. The agencies have shown the importance of making sure that each staff person understands his/her contribution to the mission, and that the level of contribution should be part of the staff review process. They have learned to “manage by the measures.” • Start now. Agencies with mature PBRA systems indicate that the only way to have a “PBRA History” and begin the evolutionary process is to get started with PBRA and target- setting and learn by doing. Mn/DOT’s measures and targets, identified in its 2003 Statewide Long-Range Plan, were refined through applica- tion. As a consequence, the following PBRA-related changes were made in the new 2009 long-range plan: • Expanded safety measures by stratifying different modal and vehicle type of fatalities; I-29

• Eliminated most of the land use measures; • Added access to intercity bus service, air service, and airports; • Added transit ridership; • Added Congestion in Regional Trade Centers, Public Tran- sit Service Hours, Transit Coverage, and nonauto commuter trips; and • Added air pollutants, cleaner fuel, and wetland impacts. MTC’s performance-based process also has been evolu- tionary during the development of its last three long-range plans. Numerous advances in introducing a performance- based framework were made even within the last year of developing the most recent RTP and have contributed to the development of more meaningful targets for that agency. Washington State DOT’s (WSDOT’s) experience also con- firms that target-setting requires a solid history of performance data as well as managerial comprehension and appreciation of that data, which comes with time and experience. Managers must have the ability to understand transportation system behavior—i.e., “what the data are saying”—and to discern what they can or cannot control. Commitment to Regular Communication and Reporting Regular tracking of investment performance and report- ing of results to the public and transportation stakeholders serves to focus attention on an issue over time so that it is not lost in political and public discourse as new challenges arise. Regular reporting and communication of progress helps to keep staff and the public focused on the particular challenge, especially when it is tied to agency or even staff- level “merit”/compensation. This helps all involved to under- stand the nature of solving process problems over the long term, rather than focusing on immediate issues (e.g., fight- ing fires) that often distract from the larger mission of an agency. It also supports longer term trend development that is needed to track the performance of investments over time. Regular reporting of performance results is a commitment common to all organizations interviewed. This occurs both internally and externally for public sector organizations, and usually just internally for private sector organizations. The fol- lowing examples illustrate how some agencies support the democratization (i.e., widespread access) of data within the agency, as previously discussed in Results Monitoring. For example, the Orlando/Orange County Expressway Authority (OOCEA) publishes a monthly report entitled “Expressway Travel Time Performance” which documents the morning and evening peak travel times per lane on the designated roads which are under the operation of OOCEA. This report is crit- ical to the OOCEA as a “performance management” tool to assess the effectiveness of managing the travel time for the public on the toll system. The report, as well as weekly data for each plaza, also can be used to determine when and where to add staff at toll stations in order to keep traffic moving safely through the expressway system. Other resource allocation decisions resulting from these data include budget adjust- ments and capital project prioritization and scheduling. Japan’s Ministry of Land, Infrastructure, and Transport (MLIT) management framework incorporates four key steps, the third of which is to “evaluate achievement levels.” Through this process, the Road Bureau uses performance measures to prioritize roadway projects within program areas based on reported results and progress towards annual and 5-year targets (Figure 3.3). The Bureau submits a required report, which com- pares performance to funding, to the Ministry of Finance when the MLIT requests funding, though funding to the Bureau is not tied to the reported performance. For the MLIT, as with the OOCEA, these reporting mechanisms with progress towards targets aid in interfacing with the public and stakeholders and also guide internal decision-making. If MLIT targets are not met for a program, for instance, then a closer review is done to determine how the processes for that program may need to be revised, or if a new program may need to be developed to address those performance needs. Conversely, if targets are consistently met earlier than anticipated, target deadlines or measures are reset to reflect more accurate expectations. Each year since 1994 Hennepin County has produced a Community Indicators report that provides a view of the cur- rent community conditions and trends based on selected quan- titative data. The report is a reflection of the health of the county as a whole. Over time, indicators have been adjusted and updated to reflect the county’s changing interests and priorities. However, many of the original indicators have been retained in order to establish a trend line and track changes in residents’ quality of life. In 2002, Hennepin County government intro- duced a strategic management framework that focuses on results and customers. Consistent with the framework, the indi- cators in each annual report have been grouped to align with the county’s current overarching goals. The regular development of these “products” that are distributed to the public and used by decision-makers helps to maintain staff enthusiasm for the performance-based process and ensures the continued devel- opment of the necessary inputs for the report. Span of Control/Agency Jurisdiction Traditionally, transportation performance measures have been used to gauge the impacts of discrete types of investment strategies in relation to very specific transportation goals, with goals most often tied to a modal aspect of the transporta- tion system, fund source for the transportation investment, or particular transportation function within a particular geography (local, regional, state). This has led to a somewhat I-30

siloed approach towards performance measurement, with measures that provide a more narrow assessment of investment options as opposed to a “systems”-level perspective. While many transportation organizations have begun to develop more comprehensive performance measures that better reflect their diverse planning goals and objectives, measuring perfor- mance over the larger transportation network and across var- ious stages of the transportation planning process remains a challenging endeavor. This topic is being evaluated through NCHRP Project 8-67. The span of agency control, whether it is through funding, modal authority, or geographic jurisdiction, plays a strong role in the development of measures and targets, because it controls the perspective from which each investment is evalu- ated. An agency that manages only highways will have a nar- rower set of measures than does an agency with jurisdiction for multiple modes. States responsible for all roads, rather than only the higher functional classes, face greater data-gathering complexities. This can influence how they set targets and how they use data to measure progress towards those targets. For instance, Mn/DOT has direct control over the quality of pave- ment, but it can only influence transit service provided in Greater Minnesota through funding. In all instances, the level of influence that the department has over a particular measure affects the target that is eventually set. Within DOTs, stan- dard siloing of functions has led to strong asset management I-31 Source: Road Bureau, MLIT, “Outcome-Based Road Administration Management in Japan.” Figure 3.3. Road bureau performance measure and target report for FY 2006 and 2007.

systems for roadway maintenance functions, but this process has not translated as strongly to other DOT functions. The emerging authorization debate includes the possibility of Federal performance measures and targets. Targets must be highly sensitive to the differences among states in terms of their data, analysis methodologies, existing funding, PBRA history, and other constraints. This likely will result in very high-level, broad targets. Targets likely need to be tied to incentives or dis- incentives due to limitations in the Federal government’s span of control over the systems being measured. In the case of the Kansas State Department of Education (KSDE), Federal No- Child Left Behind legislation presented Federal targets for the State to achieve, changing the KSDE’s role and to some extent changing its span of control within Kansas’ educational system. The KSDE ties its own state-level targets, which complement the national targets, to accreditation and limited incentives. Financial Resources No constraint or factor in constraining the PBRA process and affecting target-setting is cited as much as financial resources. Financial resources are intimately intertwined with the resource allocation process, both determining an agency’s ability to implement such a process by influencing other factors such as technical resources, and also potentially being determined by the process itself. For example, performance data has played a key role in biennial state legislative budget allocations for Mn/DOT, and it also has played an important role in the debate for new transportation funding. Mn/DOT quantified its highway per- formance measures and targets in its 2003 State Transporta- tion Plan and concluded that Minnesota was under-investing in its highway program by one billion dollars per year. This performance-based analysis was accepted by the legislature and virtually ended the legislative debate on level of need. The legislative discussion shifted from the question of need to the question of payment. Similar to the influence of financial resources on target- setting, staffing and person-hours of available time can affect the depth of a PBRA program, including targets that an agency can assemble and monitor. Timeframe The timeframe of desired results affects how targets will be set and what they will be. Timeframe is sometimes deter- mined by stakeholder and internal agency needs but also can be dictated by forecasting capabilities. At Japan’s MLIT, annual targets are derived in part from the latest major subjects of policy, planning, and program- ming to emerge from the funding reports from the MLIT and Road Bureau, the Road Bureau’s Mid-term Visioning Report, and the national government’s 5-year Major Infrastructure Development Plan. Longer-term targets (referred to by the Road Bureau as “goals”) match this with a 5-year span. The Road Bureau utilizes the funding reports to estimate future funds, assuming that approximately the same amount of money will be available annually for the next five years. This information is used when determining feasible 5-year goals for the Bureau and results in what is essentially a finan- cially constrained target. The Road Bureau sets and annually updates shorter term annual targets when it prepares its annual Performance Mea- sures Report/Planning Report. At this time, the measures themselves also are reviewed and subsequently modified or new measures are added. This adjustment in the measures themselves may then require an adjustment in the relation- ship between the targets and the measures. Targets for the next fiscal year are based on the possibility of achievement and similar to the 5-year goals, consider financial constraints. In Washington, the primary responsibility for translating long-term goals (dictated by elected officials) to short-term or “incremental” goals, objectives, and targets falls to the Department of Transportation, in consultation with execu- tive and legislative members and staffs. This process centers on how to set and describe these incremental milestones, how to communicate them to the public, and what legal lia- bility the State may incur by promoting these short-term targets publicly. WSDOT managers also may consider alter- natives and adjustments in the engineering solutions to problems, in the methods of service delivery, and in the con- struction materials and techniques to be used in order to address these short-term targets. These options help to achieve stated targets within current funding and other resource constraints and thus maintain consistency between short-term program accomplishments and long-term, aspi- rational goals. Technical Resources The presence or lack of forecasting tools can influence greatly the sophistication of forecasted targets. Agencies that have used HERS, PONTIS, and other tools for forecasting the results of long-term programs have greater insight with which to set long- term targets. Availability of analysis tools to identify performance impacts of projects realistically and efficiently and to track performance in relation to targets will determine what measures and targets can be used. Sometimes agencies develop desired measures and targets, even when data are not yet available, as a means of creating a “wish list” of data sources. Often it is difficult for decision-makers to see the need for data collection for a single performance metric, particularly if it appears to be part of a sin- gle endeavor (e.g., a long-range plan); if it is part of a larger, I-32

comprehensive PBRA process, however, it is often easier to justify additional data needs. The evolution of tools over time makes it very difficult to track progress consistently. Change over time in tools, data, and analysis procedures, as well as differences between agen- cies and jurisdictions, can make it difficult for stakeholders— and even internal staff—to properly interpret the results. Staff turnover also can exacerbate this situation. Agencies must develop ways of maintaining their institutional knowledge base to properly utilize evolving tools and procedures. It is often difficult to make the case to senior management or political decision-makers for funding for data and tools. For the MTC, the development and availability of the StreetSaver pavement management tool has greatly aided that MPO with project prioritization and funding distribution among local jurisdictions and has garnered strong buy-in among those communities. It has made this particular asset management program arguably the most data-driven, performance-based process in the MPO’s planning process and has allowed for more rational maintenance of the region’s highways. However, the MTC’s existing analysis and modeling tools are not equipped for a rigorous performance analysis at the project level for long-range planning, with precision levels lower than desired. It also is difficult to compare performance between modes, or even measuring impacts within non-auto modes, such as capturing benefits from bike networks or transit programs. Typical technical resources available to organizations as well as the influence of these resources on target-setting are elaborated further in Section 3.3. Centralized/Decentralized Organization The organizational structure of an agency affects the struc- ture of that agency’s PBRA program and process as well as the development and purpose of targets. WSDOT is a somewhat centralized organization, allowing the central office to work directly with state government and stakeholders to establish targets. It also allows the organization to handle a somewhat complex and involved target-setting process for capital programming, which must account for broad policy goals and objectives, external reporting, and internal programming needs. The target-setting process is coordinated across several other processes, such as WSDOT’s long-range planning process (LRP), its biennial Strategic Plan, the Governor’s GMAP, the Attainment Report submitted by OFM to the legislature, the Federal Stewardship Report, and the Washington State Qual- ity Award (a Baldrige process). Conversely, the Florida Department of Transportation (FDOT) is highly decentralized, resulting in possibly differ- ent targets for different measures in different districts, but with broader and more flexible measures and targets at the central office. Decentralization means that FDOT’s district offices are essentially DOTs themselves, each with its own secretary, with planning, environment, right-of-way, design, construction, maintenance, and legal offices. The various offices within the districts report to the district secretary and not to their counterparts in the central office. The district sec- retaries report to the Secretary, and also sit on the FDOT Executive Board. Detailed funding allocation decisions are made at the district level, resulting in wide variations in fund- ing priorities and decision-making processes among different districts. This structure requires a strong but flexible per- formance management system to ensure consistency across districts in terms of achieving statewide goals. FDOT’s over- all Business Plan seeks to maintain accountability and trans- parency for processes that may not be standardized across the department. Stakeholder Expectations Stakeholder influence can have a very significant impact on target-setting. When external stakeholders become engaged in the process, they can influence which measures are focused upon. These stakeholders range from the public and elected officials to contractors, the construction industry, state and Federal resource agencies, and public safety agencies, among others. The majority of agencies reviewed with established targets developed them through a committee process that provided for stakeholder input. As such, it provided an opportunity for dialogue about the transportation issues, constraints in fund- ing, and other topics, and as such led to the development of realistic, meaningful targets. This is absolutely critical for state DOTs and MPOs who make decisions in a very litigious envi- ronment. It is critical to communicate, not only the lofty, long- term goals for transportation systems, but the reality of fiscal and political and regulatory constraints so that stakeholder expectations can be managed from the beginning. Without this, public transportation agencies would be wary of setting their agencies up for disaster and perhaps even encountering legal trouble. Transportation is a long-term business, with per- formance typically improved over the long term, but agencies operate in the context of short-term politics, which has a very strong impact on performance-based management. It is often challenging to develop and sustain consistent measurement practices and target-setting to enable meaningful tracking of performance over time. For example, for the MTC, several different levels of stakeholder participation constitute the long-range planning process. Overarching goals and strategies were set at the exec- utive level. A subcommittee of the MTC Planning Commit- tee, an ad-hoc committee consisting of the nine counties and 26 agencies in the region, derived the measures and targets. I-33

The MTC Planning Committee voted on and approved mea- sures and targets. The Commission had the final word in approving the targets. Keeping flexibility in the target-setting process, and using language such as “voluntary” and “interim,” was critical in getting the Commission to approve targets and properly set stakeholder expectations. This approach also was considered prudent from a legal perspective. Internal Support A common theme among many agencies with more devel- oped PBRA processes is internal support and an inside “cham- pion” at a high level. While the MTC’s PBRA process in regards to long-range planning has evolved over the development of the last three plans, the Executive Director championed PBRA with staff and elected officials for the development of the MTC’s most recent plan. The development of the plan started with the development of performance objectives and the setting of very aggressive performance targets. The performance-based system in Hennepin County is championed by the County Administrator as well as the Pub- lic Works Director. The setting of performance targets at Hennepin County is driven by the top-level managers; as a result, these managers understand the importance of acquir- ing relevant data. While these initiatives often begin with strong internal sup- port and an inside “champion,” ultimately processes or proce- dures must be put in place that can transcend administrations or individual staff. Private Sector Experience The different mission and vision that private sector com- panies have from their public sector counterparts results in different priorities and targets. As private companies exist to satisfy their shareholders, the primary driver of share- holder satisfaction is typically corporate profits. Therefore, they typically place little emphasis on public outreach, and they try to avoid or minimize the effect of regulatory burden (versus adhering to and even enforcing regulations, which a DOT must do) in order to streamline operations and mini- mize cost. In contrast, transportation agencies heavily weight public policy issues such as social equity, access to urban areas, and urban planning when formulating their policies, whereas few private sector companies take into account public policy con- siderations when deciding how to allocate their resources or measure their performance. Those companies that factor in public policy to their resource allocation process tend to be in highly regulated and/or capital-intensive industries where public affairs is important to the success of the business, such as rail transportation. Five environmental variables shape the performance man- agement and resource allocation process and consequently the target-setting process in private sector companies’ trans- portation organizations. Organizational structure, owner- ship, history, culture, and planning capabilities can combine to form different approaches to performance measurement and management, as described in the following paragraphs: • Organizational Structure. Companies with multiple busi- ness units often hold up the highest-performing business unit as the benchmark that the others should strive to beat. Companies that are capital-intensive tend to have more sophisticated capital expense authorization processes. The element of competition is often more important than the specific targets, so some companies let their divisions have a broad role in determining which metrics to use. • Form of Ownership (Private Versus Publicly Traded). Privately held companies are willing to say which metrics they track, but are reluctant to divulge their targets since that might give away competitively valuable information. In publicly held companies (traded on stock exchanges), financial goals inevitably drive the operational goals. In contrast, privately held companies often emphasize oper- ational excellence before financial success in the belief that the former will lead to the latter. • History. Companies that have a long history with a specific performance metric work comfortably with targets based on that metric since the organization understands the past, present, and future using that metric as a reference point. • Culture. In companies with a competitive spirit, the atti- tude about the numbers is more important than the num- bers themselves. In companies with an analytical culture, data collection and analysis is revered and viewed as syner- gistic with continuous improvement. • Planning and Forecasting Capability. Companies with strong forecasting capabilities are more apt to have long-term goals and targets, whereas companies with weaker forecast- ing capabilities are more inclined to set annual targets only. Private sector companies take a different approach to PBRA based on the different nature of the resources that they manage. Staff, capital assets, and budgets may have different targets as well as different target-setting approaches due to their different composition compared to their public sector counterparts, as described in the following paragraphs: • Staff. Most private sector companies employ a mix of full- time, part-time, and temporary employees, so attrition and turnover rates are higher, which makes it easier to imple- ment the Darwinian principle of “survival of the fittest” to allocate staff resources. A bell curve is often used for grading performance, and as individuals outperform their I-34

peers, they get rewarded for their results. In the aggregate, high-performing teams get rewarded, teams on high- performing projects get rewarded, and even high- performing business units also get rewarded as the sum of the individual rewards adds up. A culture of turnover is more permissive of outgoers. • Capital Investments. Compared to fleets that are used in the private sector (often standard trucks, tractors, and trailers), public DOTs sometimes have more specialized fleets that involve a different resource allocation approach. This includes customized and expensive vehicles (such as school buses) as well as emergency vehicles (such as snowplows, police cars, and ambulances) that have inherently lower uti- lization rates. While both the private sector and public sector DOTs use infrastructure such as terminals, maintenance depots, IT systems, warehouses, and commercial/industrial real estate and buildings, DOTs also might manage some unique buildings and facilities; for example, they may man- age toll booths, depots, ports (airports, seaports), and high- ways that are more local and involve specialized maintenance and repair. So while private sector capital investments are often allocated on the basis of their utilization and payback, public sector fleets are often purchased to meet the peak demand. • Operating Expenses. While private sector companies frequently outsource many aspects of their operations, including IT system development and maintenance, freight auditing, brokerage, and freight carriage, the public sector may use less outsourced services, which makes the public sector’s labor cost more solidly fixed cost rather than a resource that can be flexed up and down as needed. The flexibility that comes with outsourcing gives private sector companies more flexibility to base resource allocation on customer contracts. Furthermore, public sector DOTs may be more representa- tive of large companies, which are characterized by more for- mal and documented strategies, more timely and predictable planning cycles, more management controls, and longer response times between data gathering and resource decision- making. They have more formal and documented strategies than smaller companies, in part because they work on a longer time horizon (5 years is the norm for large companies). They also have a more defined competitive positioning than smaller companies, which often compete in fragmented, more com- petitive environments in which competitive analysis is less actionable. Finally, they document their business strategy more formally, in part because large companies frequently have an obligation to shareholders and in part because legislation like Sarbanes-Oxley requires disclosure. Larger firms have more timely and predictable planning cycles than smaller companies. Capital budgeting is usually done at the end of the fiscal year so as to have a proper budget for the upcoming fiscal year before it starts, and operating budget variances are tracked monthly. They also have more controls than smaller companies. Larger companies have more metrics and more sophisticated measurements (end-to-end, with clearer definitions, etc.) because there are more people to inform. Finally, large companies have longer reaction time than smaller companies because they have to collect fragmented information from a wide network of many locations and aggre- gated and analyzed in order to determine the appropriate corrective action. 3.3 Approaches for Target-Setting As discussed earlier, one of the three key objectives for NCHRP Project 8-70 is “to provide a comprehensive descrip- tion of the process and methods by which targets are set for use in performance-based resource allocation.” In support of this objective, a number of private sector and public sector case studies were developed to provide an illustration of PBRA practices, with an emphasis on target-setting as an element of the process. Unique to this study is the in-depth review of pri- vate sector applications of PBRA and its best-practice examples of target-setting approaches. Some private sector best practices are applicable to the public transportation sector and can be applied within transportation agencies to improve the public sector application of the PBRA process. A summary of public sector practice, as illustrated by the case studies in Volume III which is being published as NCHRP Web-Only Document 154, is provided in the following sections. A summary of the general PBRA process applied within the pri- vate sector follows, along with a description of common target- setting approaches. Public and private sector approaches are reviewed in the context of the seven-step target-setting approach documented in NCHRP Report 551 and a set of potential revisions to the seven-step approach is provided, based on case study review. Public Sector Approaches to Target-Setting As demonstrated in the public sector case studies in Vol- ume III, there is a wide range of agency implementation of PBRA processes and an equally as diverse range of implemen- tation approaches for target-setting as an element of PBRA. The target-setting approach is determined largely by the ini- tial impetus for developing a performance-based process for resource allocation, whether externally driven (e.g., through political or legislative intervention) or internally driven (e.g., strategically developed as a proactive response to the trans- portation planning and funding environment). Approaches that transportation agencies currently are using mirror, in many respects, the approaches identified in the private sector I-35

examples, with variations occurring primarily in the targets, themselves, and the feedback mechanisms that link actual investment performance to changes in resource allocation. Perhaps the most significant distinction between public and private sector approaches is the private sector’s greater uti- lization and application of targets within the PBRA process, compared to the public sector, where the use of targets is not as common. Approaches that public sector agencies currently use to set targets, either in isolation or in combination with one another, include the following: • Policy-driven targets established by agency executive man- agement or other external political body (i.e., “top-down” approach). This is an approach in which targets are set in a hierarchical fashion by senior agency management or an external political or legislative authority operating outside of the transportation planning arena. Oftentimes this occurs in response to public outcry or growing public discontent over a transportation issue, with direct action from elected offi- cials being called for. Under this approach, agency senior management or a political body defines targets in the con- text of larger transportation goals or policies; staff is tasked with developing a transportation investment plan to meet the target and conducting modeling and technical analysis needed to demonstrate attainment of the target under a future funding scenario. • Modeling is often used by agencies to evaluate progress towards targets, even when modeling was not used to estab- lish the target itself. Project and policy scenarios can be tested using travel demand models, combined with other postpro- cessing tools to include metrics such as benefit/cost to evalu- ate their relative contribution to progress towards targets. • Consensus-based process where targets are established internally through a collaborative planning process and reviewed/approved by outside, typically management/ Board-level sources (i.e., “bottom-up” approach). Under this approach, transportation staff develop targets based on comprehensive analysis of planning context and con- straints on possible investment performance, with input from a variety of transportation stakeholders that have an opportunity to impact the process; targets are typically approved by senior management. • Reliance on formal and informal customer feedback in which the transportation system user is viewed as a “cus- tomer,” and the transportation planning and development process is oriented towards customer-service. Under this approach, direct feedback on system performance and objectives for transportation investment are gathered from the transportation system user through a variety of survey and outreach methods. This feedback is then used by trans- portation agency staff to develop specific measures and tar- gets that are closely aligned with the needs of the traveling public (the “customer”). • Use of benchmarks from peer agencies, with targets estab- lished based on review of similar investment approaches and results experienced by other transportation agencies. Under this approach, criteria should be set for peer group selection and analysis. Once the peer group is set, the analy- sis will provide a relative status of each state’s performance measures and targets. The comparison among states will help guide the final determination of targets within selected performance measure categories. Policy-Driven As noted in Section 3.2, one of the most critical factors in establishing targets can be political or legislative intervention in the transportation planning process. In some examples, a political body, senior transportation agency management (sometimes guided by a technical advisory committee), or higher level agencies (e.g., U.S. DOT) will establish a target as an element of the transportation plan and development process. This is a very direct approach and can be a very posi- tive impetus for performance-based planning. It also can cre- ate a difficult situation for transportation agencies to navigate if practitioners were not involved in the target-setting process and/or if the process was not informed by real-world planning, funding, and technical constraints that may impact attainment of the target. Targets set under this approach may often be developed in a more qualitative manner, with less rigorous up-front analysis supporting the target-setting process or, more specifically, with less attention paid to how a target may actually be tracked by transportation authorities given certain technical and resource constraints. Once developed, the targets tend to be well- communicated and well understood (well publicized) and con- vey a strong message, but success in meeting the target can be very difficult if the agencies and transportation staff responsible for meeting a target are not involved in helping to develop it. FDOT provides a strong example of a policy-driven target- setting process, with targets that were established by the State legislature. In FDOT, resource allocation decisions are driven by a number of output- and outcome-oriented performance targets in the areas of highway maintenance, Strategic Inter- modal System (SIS) funding, and transit funding, as required by Florida statute. These include the following: • Eighty percent of pavement on the State Highway System meets department standards; • Ninety percent of FDOT-maintained bridges meet depart- ment standards; • One hundred percent of the State Highway System meets acceptable maintenance standards; I-36

• Fifteen percent of discretionary capacity funding must be used for transit projects; and • Fifty percent of discretionary capacity funding must be applied to the SIS (the department’s own performance target for this is 75 percent). The remaining primary targets are set by the Executive Board, and secondary measure targets are set by the program offices and the districts (i.e., the “measure owners”). As these targets are mandated by state law, meeting them is the depart- ment’s first priority. Determining the standards against which these measures are applied is the Executive Board’s responsibil- ity, and so the standards are reviewed as one of the first steps in the Program and Resource Plan and Work Plan update process. Figure 3.4 illustrates the relationship between goal and target-setting and performance measures in FDOT’s bridge program. There are additional secondary targets for Preliminary Engineering Consultant levels. When FDOT is building their new Work Program, there is an opportunity to bring for- ward new specialty projects. However, there is political pres- sure to add highway capacity expansion projects with any available resources left over. Consultant targets are set so that the level of engineering activity is maintained and the pro- duction pipeline keeps going. FDOT currently is postponing, reducing the size/cost, or removing projects due to revenue reductions. The first area likely to be cut is capacity expansion projects because main- tenance targets are “sacred.” One would logically expect that consultant levels also would go down, but FDOT is struggling with how to set the target to determine the appropriate level of reduction. FDOT previously used right-of-way (ROW) acqui- sition targets as a measure to keep production going, but the Executive Board chose to do away with them because ROW land resources are decreasing. Historically, FDOT reset engi- neering consultant targets to whatever level is projected in the Work Program for four common years. They perform a trend comparison to construction levels and project what levels of preliminary engineering are needed. The Department is con- sidering alternative methods for setting targets. Modeling Modeling is often used by agencies to evaluate progress towards targets, even when modeling was not used to establish the target itself. Project and policy scenarios can be tested using travel demand models, combined with other post- processing tools to include metrics such as benefit/cost to eval- uate their relative contribution to progress towards targets. Both top-down and bottom-up modeling are used to set tar- gets in many companies. Top-down modeling most commonly drives the target by high-level requirements. Top-down model- ing determines the strategies or funding needed to achieve the target; bottom-up modeling determines what level of perfor- mance is possible, and then uses that to calculate the expected target, as Japan’s MLIT does annually. The exact use of modeling depends to some extent on the way in which an agency is utilizing PBRA. For what part of the investment process is the agency setting targets? For example, are the annual financial targets, long-term targets for a long- range transportation plan, or mid-term targets for a package of projects and programs to be included in a TIP? Many agencies have found innovative ways to incorporate performance-based processes and targets into their planning processes and duties, supported by modeling. The MTC uses its own StreetSaver® PMS to calculate preventative maintenance funding targets for its local jurisdictions; the ratio of “actual ver- sus targeted” determines the jurisdiction’s performance score I-37 Goal: Protect the public investment in transportation Objective: 90% of bridges meet FDOT standards Evaluate data and report percent meeting standards Program and contract to repair/replace deficient bridges POLICY FINANCEPERFORMANCE MONITORING PROJECT DELIVERY Figure 3.4. PBRA for FDOT bridge program.

and is a factor in calculating the amount of funding that will be allocated to that jurisdiction. Consensus-Based In many cases, transportation agencies develop targets through an internal, consensus-based planning process as part of a more comprehensive PBRA exercise. Typically, the use of specific targets derived from an internal process is seen in agen- cies with a more sophisticated and well-developed PBRA sys- tem that has developed over several iterations. Staff resources and time permitting, an internally developed process can lead to very meaningful and effective targets within an agency’s PBRA process. The MTC provides an excellent example of an externally mandated (policy-driven) intervention that supported, rather than dictated, an internally managed, consensus-based, target- setting approach. In this case, the California state legisla- ture had required PBRA to be a part of transportation plan development, and for some critical areas, established targets itself (e.g., CO2 reduction). This political intervention served to strengthen the role performance assessment played in the plan- ning process and allowed real leadership to emerge in this area, in MTC’s case, at the Executive Director level where the oppor- tunity was provided for PBRA to be championed with staff and elected officials. As part of the ongoing 2035 RTP, targets have been established for three principal focus areas: Economy, Environment, and Equity. Environmental targets, for example, include a 40 percent reduction in CO2 below 1990 levels, 10 per- cent reduction in PM2.5 below 2006 levels, 45 percent reduction in PM10 below 2006 levels, and 10 percent reduction in VMT per capita below 2006 levels. MTC staff have recently com- pleted an exercise to test how different system expansion strate- gies contribute to achieving these predefined performance targets. Because the MTC is in the early stages of conducting plan analysis against performance targets, there is no direct impact on resource allocation that can be determined or is documented yet. For those targets that the MTC developed, an internal planning team (bottom-up) approach was used. This type of approach is common for transportation agencies using PBRA. Goals and objectives were established at an executive level, and a subcommittee of the MTC representing the various juris- dictions in the region derived the measures and associated tar- gets through a more technically informed process. The MTC Board formally adopted the targets as part of the larger PBRA system. Note that the MTC emphasized that keeping flexibility in the target-setting process, by using language such as “volun- tary” and “interim” in regards to Board adopted policy, was critical in getting targets approved. As noted in Section 3.1, it is important for transportation agencies to keep in mind the litigious environment in which they work and not set them- selves up for failure in regards to not meeting targets, in light of so many success factors being outside of their control. Customer Feedback As stated in Section 3.2, those agencies that have taken a clear customer-service approach within the resource allocation decision-making process understand the need to use targets that communicate to the system user, the “customer,” the return on their investment. Customer satisfaction is a funda- mental aspect of performance for these organizations and per- meates the process for how potential investments are evaluated and selected to receive funding. Coral Springs, Florida, provides a very strong example of a municipality that has taken a very direct customer-service approach towards its resource investment process, in which customer input drives the decision-making process for a wide range of services, to include transportation. Target-setting is a key component of the Coral Springs process as it provides a direct translation of progress as feedback for a community that has been actively engaged and empowered as part of the plan- ning process. Community feedback is gathered through a vari- ety of methods to include, but not limited to, an Annual Citizen Survey, public hearings, a city blog, regular visioning exercises and focus group discussions, a complaint tracking system, and employee surveys. The Coral Springs customer input process is extensive, formal, iterative, and continuous and impacts, to a significant degree, the target-setting approach. With regard to transportation, three targets have been established directly from customer feedback which indicated growing concern over traffic mobility and safety issues. These include a 10 percent speed reduction on traffic-calmed street, reducing intersection crashes to no more than 165, and increasing city transit rider- ship to 125,000 annual trips. Hennepin County, Minnesota, also provides an example of a customer-oriented approach to target-setting. Hennepin County is the largest local government in Minnesota, located within the Twin Cities Metropolitan Area, and has been rated among the best managed large counties in the country. Beginning in 2002, Hennepin County introduced a strategic management framework that focused on results and cus- tomers. Hennepin County uses the Balanced Scorecard (BSC) approach as a management tool, which helps to align county daily work with the county vision and goals in the context of community needs. The “customer” is one of the four perspec- tives that the BSC approach is viewed from as part of the PBRA process (the other three being: Finance, Internal Process, and Leaning and Growth). The Hennepin County Public Works Line of Business has six functional areas, including (1) emer- gency services, (2) environmental, (3) energy, (4) housing and public works, (5) transportation, and (6) administrative ser- vices. These functions are at varying stages in setting perfor- I-38

mance measure targets. For some measures targets have not yet been set. However, the Business Line is on track to develop a complete menu of measures and targets for all functions in the near future. In the transportation service area, a number of specific targets already exist and include targets related to bridge and pavement sufficiency ratings, reducing crash rates, completion of the Bicycle System Plan, and project delivery standards. Benchmarking Benchmarking as a target-setting approach provides a transportation agency with the means to establish targets in a relatively quick and efficient manner that can be realistically achieved. Benchmarking is often handled by a scan of peer agencies that have attempted to reach similar goals through similar types of investment packages. It allows peer agencies to observe realistic performance outcomes for various types of transportation improvements and investment levels, and therefore supports a realistic target-setting approach. In the case of the Maryland State Highway Administration (SHA), for example, targets are developed with input from appropriate managers and data owners, using outside data, trends, and best practices to assist when possible (e.g., bench- marked against peer states). Performance measurement has been integrated into daily agency operations and elevated as a planning and technical necessity. Targets are established by the agency performance measure “lead” who is respon- sible for maintaining and reporting data for a particular measure and ensuring data accuracy. Note that by identify- ing a measure “lead,” the SHA also is facilitating a sense of ownership and responsibility for performance measure- ment and integrating the practice into an overall agency cul- ture of performance. While most public sector agencies do not formally use benchmarking to set performance targets, there is nonetheless an emerging interest in comparing the use of measures and targets across jurisdictions. The AASHTO Standing Commit- tee on Quality, recently renamed the Standing Committee on Performance Management, conducted a study in 2007 which identified best practices for highway project delivery times and cost. The committee also has identified best practices for smooth pavements. The current movement to establish a more performance- based Federal-aid highway program also underscores the importance of establishing measures and targets that can be uniformly applied across states and MPOs. This interest is con- sistent with trends in New Zealand and Australia. The Aust- roads National Performance Indicators (NPI) system includes dozens of indicators in 11 broad groupings, covering safety, asset management, environmental impacts, system capac- ity, user satisfaction, and project management, among other things. The NPI data are noteworthy not only for the breadth and sophistication of the indicators used but also because they present consistent and comparable data across a trans- portation system managed by nine separate agencies in two countries, allowing unprecedented benchmarking possibili- ties. Specifically, benchmarks against other jurisdictions also are being introduced into VicRoads’ process. MTC developed a pavement management system called StreetSaver® that is used by nearly all local jurisdictions in the Bay Area. Local pavement condition data is used by MTC at the regional level to generate a regional pavement condition sum- mary and PCI index, which they then use to predict regional needs for pavement maintenance as part of their regional transportation planning process and which can be used as a benchmark among local governments. While there are five basic approaches to setting targets, the policy-driven approach tends to dominate the target-setting methods used by the agencies reviewed, followed by consensus- based approaches. This makes sense in light of the significant transportation funding issues that have impacted all transporta- tion agencies at the state, regional, and local level throughout the country over the last few years. These funding issues have created a situation in which competition is increasing for ever-decreasing transportation dollars and in which more accountability is being demanded of elected officials and transportation authorities in how limited dollars are spent. This creates a situation in which performance-based processes flourish and one in which decision-makers may establish investment mandates or requirements quickly to show action being taken. While the policy-driven approach to target-setting is the most frequently used today, the advantages and disadvantages of all four approaches need to be considered as practice matures. These advantages and disadvantages are summarized in Table 3.1. Private Sector Target-Setting Approaches Organization accountability in the private sector is strong and direct. Organizational hierarchies are critical in deter- mining reward and penalties for performance versus target. The classic pyramidal hierarchy still often drives behavior, despite the existence of numerous more modern organiza- tional structures that divide businesses by strategic business unit (SBU) or product line. Matrix structures also have become popular, but practically speaking, reporting rela- tionships are typically characterized by one dominant boss. The implication for transportation management is that accountability to that one boss is critical, and performance is usually judged by one person with abundant data. This may contrast with organization accountability in the pub- lic sector. I-39

The Budget-Setting Process Budgets play a critical role in decision-making. Whereas in the public sector a balanced budget may be an option, in the private sector it is the only way businesses work. The fiscal year’s budget must balance. In addition, budget variances are tracked, and monthly variances are addressed promptly. Large variances usually result in some sort of realignment of resources—especially headcount, job responsibilities, or equipment allocation. In addition, IT systems and processes are often re-engineered to achieve the budget figures. Key Performance Indicators (KPI) are prevalent and are usually prominently posted in managers’ offices. These reflect the measures for which individuals are responsible and often to which compensation is tied. The budgeting process begins with functional and business unit heads agreeing on key performance indicators that accu- rately reflect their unit’s success. The metrics typically fall into five categories: cost, quality, service, speed, and delivery. The determination of metrics that are important to the organization as a whole includes identifying the hierarchy of metrics, including especially which ones are important enough to be on the executive dashboard. Usually, this consists of a set of metrics that are common to all subsegments of the business. It should also include a determination of which metrics should be visible at different levels of the organization. Usually, these consist of the metrics that are important to specific subdivi- sions with common geographic areas, types of infrastructure, or financial structures. The selection of metrics should reinforce the objectives of the company, which usually include at a minimum: (1) Fixed asset minimization, both in terms of the number of units, as well as the value per unit; (2) Operating cost minimization; and (3) Service level maximization, including flexibility (the ability to respond quickly to temporary changes in circum- stances) and agility (the ability to adapt to structurally changed circumstances). Transportation cost, in its various permutations such as cost per shipment, cost per mile, cost per order, cost by traf- fic lane, and others, is often separated into several different categories: • Total unit cost—the cost per unit with the numerator being cost and the denominator being a volume metric such as miles, stops, or orders or a resource metric such as the number of vehicles or drivers • Transportation cost separated from product costs— transportation budget or actual total transportation cost, tracked rather than imputed as an “adder” as a percent of rev- enue, which is a common practice among many companies • Landed cost—transportation plus other costs incurred in moving product through a supply chain such as cus- toms, insurance, interest, and storage costs. Landed cost is most often used in importation since duties and inven- tory carrying can represent substantial costs compared to transportation. • Cost avoided—money that would have been spent but does not have to be due to a change, for example, in processes. • Quality-adjusted cost Profitability also is reported in several different ways: • Revenue—If transportation is a profit center for the company. • Operating ratio—operating cost divided by operating revenue. I-40 Target-Setting Approach Advantage Disadvantage Approach to Balancing Advantages with Disadvantages Policy-Driven • Linked to PBRA elements • Political process may ignore PBRA • Establish PBRA with legislative participation to ensure support and understanding Modeling • Defensibility. • Better understanding of future performance. • More time and money intensive. • Models change over time. • Continue to refine modeling techniques. • Use hybrid approach. Consensus-Based • Insures broad understanding and acceptance within agency • May flounder in effort to be inclusive • Appoint internal champion to lead effort to identify the “critical few” measures and targets Customer Feedback • Insures more transparent process • May be confusing to discuss technical measures with public • Describe measures and targets in the simplest terms possible Benchmarking • Provides a peer group comparison • Can be misused for comparative rankings • Continue to refine comparative analysis techniques Table 3.1. Managing risk of public sector target-setting approaches.

• Customer profitability—profit measured in various ways, such as revenues minus expenses or percent profit margin at the customer level. This metric is used by companies that have sophisticated enough information systems to track and attribute transportation cost to specific deliveries and customers. Return on investment is measured by relatively few compa- nies (as it applies to transportation operations). It is mostly applied on a case-by-case basis to specific investments in equip- ment or warehousing. However, the following three new trends are making return on investment a more popular performance measure: • The drive to remove assets from the balance sheet through off-shoring and outsourcing; • The evaluation of investments in off-shoring have gained popularity recently, especially as much sourcing shifted overseas to China, and the value of the U.S. dollar has fallen so sharply as to cause many of those same companies to evaluate moving the production back onshore; and • Intense interest in public-private partnerships (PPP). In such models, each party lobbies for their share of public money while trying to give up as little of their own private capital as possible. Beyond profitability and return on investment, shippers and carriers use a wide variety of important performance metrics that often measure their ability to fulfill customer orders reliably and quickly. These include metrics related to volume, velocity, total supply chain costs, quality of customer interface, volatility, and security. Then, through a process that may be automated to varying degrees, Production, Procurement, Operations, and Logistics all share important information that is used to facilitate the con- trol and feedback loop. The actual metrics depend greatly on the company itself and on the mission, vision, objectives, and tar- gets set in the Planning process. To use cost as an example, the following departments provide cost feedback: • Production (if a manufacturing company) shares its direct costs and its indirect costs: – Direct costs are based on labor consumed, material consumed, and net of any byproducts created that have market value. – Indirect costs, often called Manufacturing Overhead, are allocations of fixed costs such as plant and equip- ment that is used to produce multiple product types or serve multiple channels, and labor that is dedicated to planning and control. • Procurement shares its acquisition costs. Acquisition costs include the cost of purchasing an item or service, plus the cost of transportation, duties and taxes, shipping, handling, and insurance. • Operations shares its direct operating and maintenance costs for the equipment that Procurement buys. • Logistics shares its transportation, warehousing, and pack- aging costs. In addition, Logistics may track inbound freight costs for the material it acquires from external suppliers, since it is part of the overall supply chain. Accounting, usually a part of the Finance department, often compiles the budget and the actual cost data, and produces variance reports that are shared among line managers who have profit and loss (P&L) responsibilities, as well as executive managers to whom those P&L heads report. Where there is an ERP system or other executive dashboard, these metrics may auto-populate and be accessible to authorized managers via a corporate intranet. Executive management interprets the variance reports in light of a wide array of factors, including the following: • External market conditions influencing demand for the company’s products and/or services; • Competitive conditions, including price and non-price competition; and • The pace of internal ongoing research and development (R&D) programs, cost management initiatives, and other corporate programs that could affect the managers’ ability to meet their budget targets. Based on the review and assessment, executive manage- ment may exercise considerable discretion in reacting to updates. Good variances may elicit no response, a pat on the back, a resetting of the budget targets, a promotion, or a raise. Depending on the seriousness, bad variances could elicit no response, a root-cause analysis, a review, or resetting of the individuals’ targets or budget allocation, a demotion, or even termination. Organizational and Motivational Approaches to Target-Setting Private sector companies take a variety of different ap- proaches to governing the target-setting process. The follow- ing four different approaches can be seen in companies, including a hybrid approach: • Edict. Don’t over analyze. Just state the goal and have every- body try to hit it. The underlying principle of this approach is that success in hitting the target is entirely a function of motivation and execution and that planning is a rela- tively minor part of reaching the target. This approach also is called “Ready-Fire-Aim.” The advantage of the I-41

Edict approach is that the target is unequivocal and well- understood throughout the organization. The challenge is that the approach is not inclusive or consultative; it is more characteristic of old-fashioned hierarchical leader- ship. For example, the choice of metrics and targets is made by senior management and is not subject to discussion. • Benchmarking. There are three basic varieties of bench- marking: Best-in-Company, Best-in-Industry, and Best- in-Class. In all the cases, benchmarking means identifying a reference point or points and using them as the basis for one’s own target. The internal target can be higher or lower than the benchmark, but in either case the reference point is used to determine the target. Benchmarking also can be qualitative. For example, strategic benchmarking can iden- tify similar companies’ strategies as a basis for setting one’s own strategy. Organizational benchmarking can be either qualitative or quantitative. One common form or organi- zational benchmarking is measuring staff levels used to service a given level of activity. – Best-in-Company benchmarking fosters competition between operating units on the basis of the key metric(s). – Best-in-Industry benchmarking analyzes the perfor- mance of companies in the same industry or segment and highlights the best in the group as the benchmark, even if its activities are not directly comparable to the subject company. – Best-in-Class benchmarking analyzes the performance of a broad range of entities, including some with unrelated activities, and highlight the best in the group as the bench- mark, even if it is in a different industry than the subject company. • Modeling. Both top-down and bottom-up modeling are used to set targets in many companies. Top-down modeling most commonly drives the target by high-level financial requirements. Top-down modeling determines the opera- tional effectiveness needed to achieve the financial target and sets the operational target around that. Bottom-Up Model- ing determines what level of operational performance is pos- sible and then uses that to calculate the expected financial result. • Hybrid. Hybrid approaches use some combination of the other approaches. Different approaches may be more or less effective in different departments or business units with different management styles as well as over time as the needs of an organization change. Hybrid approaches can be extremely effective because of their versatility and dura- bility. Their drawback is that they can dilute the strong message that might come from the other approaches, espe- cially the edict approach. Positive execution versus targets can result in increased allocation of resources for upstream infrastructure such as terminals and warehouses but usually after a lag. Moreover, the prime driver of such an increase in resources is a pass-through of savings to the customer, who then decides to increase volume through the system due to that company’s (or individual’s) cost savings. Therefore, the initial response to cost savings is often a downsizing—reduction of warehouses, reduction of inven- tory, reduction of vehicles in the fleet, and reduction of sup- pliers. The second-order effect of such reductions is to decrease the cost of distribution; this savings is passed on to customers in the form of lower prices, which can subsequently generate increased sales. These increased sales then require more infra- structure to handle a higher volume of goods moving through the distribution network. Aside from the budgeting process, private sector compa- nies’ approach to resource allocation also includes the use of variable compensation, team competition, wide access to per- formance data, and simple and stable performance metrics. Most companies use compensation. However, they use each lever to varying degrees. The following paragraphs describe these methods in further detail: • Salary and incentive bonuses are powerful motivators and result in targets that are driven by what people can achieve rather than calculated guesses. Also, compensation that varies depending on performance versus the target is widely viewed as an effective way of motivating performance. • Intracompany team competition also is an effective way to motivate performance and establish the right targets based on what can be achieved through competition rather than on a calculated target. Seeing a business unit succeed can stimulate an aggressive response from a competitive peer. The Baldrige winners use internal competition extensively to instill creativity and better results. • Wide and easy access to targets and performance data helps stimulate better knowledge of current performance and performance gaps, hence the ability to improve perfor- mance and to do it more rapidly. In contrast, complexity and waffling will dampen attempts to improve performance, according to most interviewees. Three Baldrige winners— FedEx, Cargill, and Ritz Carlton—plus several Balanced Scorecard users such as CSX, Caterpillar, and BASF, rely heavily on the wide dissemination of performance data inside their companies. • Simple measurements. Complex measurements and multi- faceted metrics dashboards, while intellectually appealing, are counter to common wisdom in the private sector, where one simple and clear target for transportation and logistics per year can achieve significant improvement through align- ment of goals. Companies that use Total Quality Manage- ment (TQM) rely heavily on simple metrics because changes are implemented at the shop floor level where data must be I-42

readily obtained and recorded in order to be credible and immediately actionable. • Stable measurements. Changes in the definition of metrics over time can stifle motivation to achieve the target, as well as raise non-value-added confusion and ambiguity about how to interpret the target or how to interpret performance compared to what it was when historically different metrics were used. Companies that use balanced scorecards need stability of metrics more than most because unstable metrics make the scorecard appear unbalanced. Using each approach to measuring comes with some risks. Table 3.2 outlines the advantages and disadvantages of each approach as well as approaches to mitigating those dis- advantages. To develop the best approach for target-setting in a given environment, the advantages and disadvantages could be quantified on a scale with the disadvantages netted from the advantages and then the highest-scoring result would be the best approach. Application of the Seven-Step Target-Setting Approach Within the Public Sector The seven-step process for setting targets found in NCHRP Report 551 is exemplified in varying degrees for each approach. The following are the seven steps: 1. Define the context for target-setting and establish time horizon(s); 2. Determine which measures should have targets; 3. Develop long-term goals based on consideration of tech- nical and economic factors; 4. Consider current and future funding availability; I-43 Target-Setting Approach Advantage Disadvantages Approach to Balancing Advantages with Disadvantages Incentive Bonuses Strong motivator Inequalities Team or group bonuses Results in a “natural” target Could sacrifice performance on one parameter in trying to maximize another Nonmonetary rewards Intra-Organizational Competition Results in a “natural” target Stimulates sharing of best practices Could sacrifice performance on one parameter in trying to maximize another Stimulate internal competition based on a balanced scorecard of metrics Wide Access to Targets and Data Facilitates more improvement ideas Education and interaction with peripheral users of the data could derail progress Apply varying levels of access permissions Technology to disseminate the information could be costly Use web-based gathering and filtering of input Simple Metrics Focuses effort, resulting in quicker target attainment Could encourage gaming, whereby people sacrifice performance in some areas in order to hit a simplistically defined target Use a set of simple metrics that prevent gaming by imposing tradeoffs Stable Metrics Deeper penetration of targets into processes Slow to adapt to changing external environments Periodic review cycle to ensure metrics are optimal Baldrige Awards Process focus allows for evolution of targets Targets may not be simple or easy to remember Hybrid approach Balanced Scorecard Prevents gaming and suboptimization Complex; diversity of metrics inhibits the progress that often results from focusing on one simple target Hierarchy of metrics Total Quality Management Deep penetration into culture when fully implemented Requires a long-term commitment Hybrid approach Source: Boston Strategies International, Inc. Table 3.2. Managing disadvantages of private sector target-setting approaches.

5. Analyze resource allocation scenarios and tradeoffs; 6. Consider policy and public input implications for target- setting; and 7. Establish targets and track progress. In general, the seven-step process is not used in a formal fash- ion for a policy-driven process. For internally developed processes, there tends to be more leeway and opportunity to allow the full seven-step process to take place, although in most cases only some of the seven steps are actually implemented. For approaches relying on customer feedback, the process is driven more by customer demand as opposed to a comprehensive analysis supported by the seven-step approach, and, similarly, with a benchmarking approach, the seven-step process is not as critical because targets are set based on other agency practices. The application of the seven-step process in the public sector is described in more detail in the following paragraphs. In a policy-driven approach, the motivation for the per- formance-based process and target-setting may be politically driven. Unless guided by a technical advisory committee or a technical stakeholder group, this process is not likely to take into account in real detail the very critical aspect of data and tools needed to develop meaningful targets and to track per- formance of potential investment in relation to the targets (e.g., data availability, data history, data forecasting capabili- ties, performance analysis, etc.). In addition, timeframes to set targets may likely be limited so that policy bodies can show action being taken in an effective manner. With this in mind, Steps 1, 3, 5, and 7 are not often applied as these address the more comprehensive requirements for a complete target- setting approach. However, policy bodies, senior manage- ment, and elected officials often times are able to focus on key measures of interest (as they relate to a particular transporta- tion issue) and can determine which of these need targets to demonstrate progress to the public and their constituencies (Steps 2 and 6). For example, traffic congestion and environ- mental issues such as climate change or air quality that garner significant public interest may help to focus decision-makers on these two steps that address how the performance of trans- portation investment should be evaluated, and how it should be communicated and reported considering public input. With consensus-based processes, where targets are estab- lished internally through a collaborative approach, there is more time to pursue a comprehensive methodology that is more inclusive of each of the seven steps. By default, a consensus-based approach will require more time and greater level of technical and planning effort because of the variety of stakeholders that will be involved in the process, each pro- viding a unique perspective that will warrant attention. As demonstrated in the public sector case studies in Volume III, no one agency included all seven steps in its consensus-based target-setting approach, nor did any agency cite an approach to target-setting that included so many disaggregate steps. In most cases where target-setting was developed through a consensus-based approach, the agency was working within a framework where long-range goals were defined (Step 3); these provided the foundation for the establishment of objectives and performance measures and targets, as appropriate based on technical resources (Step 2); this process was informed by stakeholder discussion of the larger planning context to ensure the development of reasonable targets that are politically palatable and attainable given real-world planning and fund- ing constraints (Steps 1, 4, and 6), and targets were then estab- lished with progress tracked, resources permitting (Step 7). Note that Step 5—“analyze resource allocation scenarios”— was not shown to be a step often taken in the target-setting process, but rather as a part of transportation plan and proj- ect development where potential resource allocations were analyzed in terms of how well they helped the agency meet particular targets. Modeling applies most directly to Step 5, though it is likely done in concert with other processes (such as consensus-based processes) applied to earlier steps. However, when modeling is being used to determine targets, model and data limitations are likely to help determine which measures should have tar- gets (Step 2). Step 4 can directly lead into Step 5 by determin- ing the scenarios to be modeled based on financial constraints; conversely, Step 5 can be used to determine the amount of funding necessary. For approaches based on Customer Feedback, the seven- step process is not as applicable because the primary driver in the process is developing targets that communicate to the cus- tomer the investment performance in terms that the customer cares about, as opposed to communicating performance in terms of transportation planning and development standards. The target-setting process is still informed by and supported by transportation practitioners that provide the appropriate con- text for establishing targets (e.g., what percentage of crashes might be expected given certain funding levels, or how much speed may decrease as a result of traffic calming measures), but ultimately processes oriented towards the customer are driven by the customer perception of what needs to be improved and by how much. As such, most of the seven steps have not shown to be strictly applied. Benchmarking approaches tend to be even less supported by the seven-step process. Targets set under this approach are done so based on experience in peer states and/or given similar types of investment packages to support similar transportation goals. Case study review or scans of peer agency transportation investment performance provide an assessment of what can be expected in terms of impact and performance for various types of transportation improvements and investment levels. Targets are then established based on a relative comparison between peer regions. I-44

Application of the Seven-Step Target-Setting Approach Within the Private Sector A close look at the private sector experience with perfor- mance management might suggest some revisions, particu- larly in the timeframe of targets and in the correspondence of targets to metrics. Three edits would align the 551 process more closely with private sector best practices: • A rephrasing of Step 1 (“Define the context for target-setting and establish time horizon(s)”) might be “Define the con- text for target-setting and establish annual targets.” Private sector experience would suggest that time horizons be kept to a minimum. For example, DIY sets its target annually. Corporation X has short- and long-term targets. None of the companies interviewed had more than two target time- frames. Private sector companies often have a 1-year tar- get and a 3-year target. Also, the actual number of number targets, as they apply to transportation, should be no more than a few in order to achieve organizational alignment around a common goal. • A rephrasing of Step 2 (“Determine which measures should have targets”) might be “Assign a target to each metric or else eliminate the metric.” Private sector companies (MNC is an example) are sensitive to “analysis paralysis,” and therefore prefer to measure only what actually has a target. Measuring things that do not have targets can cloud performance man- agement with ambiguity. Measures without targets increase the number of numbers people study, decreasing from the emphasis on a single core target. It also implies an incom- pleteness of the target-setting process, as if the architects of the performance management process did not go all the way in determining how various metrics are inter-related, and which are more important than others. • Step 5 (“Analyze resource allocation scenarios and trade- offs”) might be better split into two steps: (1) determine the return on capital investment and (2) compute the cost/ benefit ratio of operating expenses. This step has two under- lying and different decision lattices—the first has to do with capital expenditures, and the second has to do with operating expenditures. Private sector firms (ABC Logis- tics is an example) generally allocate capital dollars accord- ing to their return on investment and operating dollars based on their profitability. Most (with the exception of Corporation X) do not consider scenarios, but they should. Return on investment can and should include the costs and benefits to all stakeholders, public and private. There are guidelines for calculating return on investment in public- sector work and in public-private partnerships. The calcu- lation of profitability depends as much as possible on assigning benefits and costs to specific user groups, as in toll road pricing analyses. 3.4 Role of Economic Models and Management Systems in Target-Setting and Tradeoff Analysis As previously indicated, modeling provides a quantitative approach to target-setting. This section provides an initial identification and analysis of the potential roles for economic models and management systems in setting performance tar- gets and in supporting tradeoff analysis. The focus is on the role of economic analysis in these functions. However, since many of the useful procedures which perform some aspects of eco- nomic analysis are termed management systems, this discus- sion also includes those management systems which provide economic analysis and which already are used for or are poten- tially useful for target-setting and tradeoff analysis. The focus of this section is on target-setting and tradeoff analysis at the system or program level, which could apply to the nation, states, metropolitan areas, other regions, or to modal or multi- modal agencies. Economic models and management systems also are useful for determining the impacts of projects on per- formance targets or for doing project level tradeoff analysis. While examples are provided from current practice, and the potential for extensions of current practice are discussed, it is important to keep in mind that economic models and manage- ment systems are by no means sufficient in themselves for setting performance targets or assessing tradeoffs. Usefulness of Current Economic Models and Management Systems in Target-Setting There are current useful and outstanding examples of the potential for the use of economic models and management systems in setting performance targets. In some cases, those procedures are not being formally used to help to set targets. In other cases, they have been used by agencies to set short- or long-term targets or to illuminate tradeoffs. The most often cited examples of economic model systems which can be readily adapted to a positive role in target-setting and tradeoff analysis are the FHWA’s Highway Economic Requirements System (HERS) and the National Bridge Inven- tory Analysis System (NBIAS) which are currently used by FHWA in highway and bridge analysis for the periodic reports on “Status of the Nation’s Highways, Bridges and Transit: Con- dition and Performance,” commonly referred to as the “C&P” reports. The 2008 C&P report, which was recently released in late January 2010 includes many exhibits which illustrate how HERS and NBIAS can be used to determine the relationship between investment levels in highways and bridges and impor- tant performance measures. The transit sections of the report, which are not discussed here, provide similar information relat- ing transit investments to transit performance measures. I-45

Several figures included in this chapter are exhibits taken directly from the 2008 C&P and illustrate a few of the per- formance measures that HERS and NBIAS can relate to lev- els of investment in bridges and highways. The 2008 C&P report contains these and other exhibits which illustrate the impacts of different investment levels on performance mea- sures including user costs, user delays, levels of service, pave- ment condition measures, and the backlog of bridge needs. The use of the HERS for producing the exhibits shown is not straightforward but requires the very sophisticated knowledge of those at FHWA and Volpe who produce the C&P. Many alternative runs of the HERS are needed to pro- duce the results shown in the exhibits. Since most states are not yet familiar with the use of the state version of HERS (HERS-ST) and since the exhibits which are very useful in target-setting require repetitive runs and the post-processing assembly of the results, there will be a great deal of education and training needed to transfer the FHWA capability to use HERS for target-setting to the states. It would be highly desir- able to foster the dissemination of HERS, HERS-ST, and state economic analysis procedures and capacity building at states and MPOs. The bridge model, NBIAS, produces the results which are potentially useful for target-setting more directly by relating investment levels to performance measures. However, the states have not utilized NBIAS. Most states do have the PONTIS bridge management system, which they could use to help inform target-setting and tradeoff analysis. Figure 3.5 taken from the 2008 C&P illustrates the relation- ship between alternative investment levels and overall con- I-46 Annual Percent Change Relative Total Spending Non- Fixed Variable Non- Fixed Variable to Capital Modeled User Rate User Rate User User Rate User Rate User 2006 Outlay in HERS Sources Charges Charges Sources Charges Charges 7.76% $188.9 $115.7 -1.8% -29.7% 7.45% $182.0 $111.5 -0.3% -4.6% -28.3% -33.1% 6.70% $166.5 $102.0 3.0% -0.7% -24.9% -29.2% 6.41% $160.9 $98.6 4.3% 0.6% -23.7% -27.5% 5.25% $140.6 $86.1 8.8% 5.8% -17.5% -20.8% 5.15% $139.0 $85.1 9.3% 6.5% -17.0% -20.0% 5.03% $137.1 $84.0 9.8% 6.9% -16.4% -19.4% 4.65% $131.2 $80.4 11.2% 8.8% -14.6% -17.1% 4.55% $129.7 $79.5 11.8% 9.4% -8.3% -14.1% -16.7% -36.6% 4.17% $124.2 $76.1 13.3% 11.1% -7.3% -12.7% -14.9% -35.4% 3.30% $112.6 $69.0 17.1% 15.5% -5.4% -8.2% -9.4% -32.7% 3.21% $111.5 $68.3 17.5% 15.9% -5.2% -8.0% -9.1% -32.5% 3.07% $109.7 $67.2 18.1% 16.7% -4.8% -7.4% -8.6% -32.2% 2.96% $108.4 $66.4 18.4% 17.0% -4.6% -6.9% -8.1% -31.9% 2.93% $108.0 $66.2 18.5% 17.1% -4.5% -6.8% -8.0% -31.8% 1.67% $94.0 $57.6 22.8% 22.1% -1.6% -2.3% -2.9% -27.8% 0.83% $85.9 $52.6 25.6% 25.4% 0.0% 1.1% 1.0% -25.4% 0.34% $81.5 $50.0 27.2% 27.3% 1.1% 2.8% 2.8% -23.8% 0.00% $78.7 $48.2 28.4% 28.6% 1.8% 4.0% 4.2% -22.6% -0.78% $72.5 $44.4 31.7% 32.2% 3.0% 8.2% 8.7% -20.5% -0.86% $71.9 $44.1 32.0% 32.5% 3.2% 8.6% 9.1% -20.3% -1.37% $68.3 $41.8 33.9% 34.6% 4.0% 10.6% 11.4% -19.0% -4.95% $48.2 $29.5 44.1% 45.9% 8.9% 23.3% 24.9% -10.7% -7.64% $37.9 $23.2 50.4% 53.0% 10.7% 32.0% 34.1% -7.7% Percent Change in Delay on Roads Modeled in HERS Funding Mechanism 2(Billions of 2006 Dollars) 1 Average Annual Funding Mechanism 2 Incident Delay per VMTCongestion Delay per VMTCapital Investment Projected Changes in 2026 Congestion Delay and Incident Delay Compared With 2006 Levels for Different Possible Funding Levels and Financing Mechanisms Figure 3.5. Relationship between alternative investment levels and delay from the 2008 C&P. 1 The amounts shown represent the average annual investment over 20 years that would occur if annual investment grows by the per- centage shown in each row in constant dollar terms. The performance impacts identified in this table are driven by spending modeled in HERS; the figures for ÒTotal Capital Outlay” are included to reflect other spending not modeled in HERS. 2 The funding mechanism used to cover the gap between a particular funding level and current spending will have different impacts on future travel behavior, which will impact the level of performance that would be achieved. Source: Highway Economic Requirements System, FHWA.

gestion delay and incident congestion delay. The table relates projected annual growth rates in constant dollar investments from particular types of fees (non-user, fixed per mile user, and variable per mile user) to the outcomes for the system in terms of the performance measures of the percentage changes in total 2026 congestion delay and incident related congestion delay, compared to 2006 parameters, which provide highly useful information about the impacts of alternative future highway investment strategies. Figure 3.6 provides similar information with regard to pavement performance measures as a function of investment levels, also by various types of source of funds. In this case, the fixed-rate user charges and the non-user charges are so close that although they are numerically different the level of detail I-47 Annual Percent Change Total Spending Relative Capital Modeled Non- Fixed Variable Non- Fixed Variable to Outlay in HERS User Rate User Rate User User Rate User Rate User 2006 Sources Charges Charges Sources Charges Charges 7.76% $188.9 $115.7 $51.4 -23.8% 7.45% $182.0 $111.5 $50.0 $50.2 -22.4% -23.1% 6.70% $166.5 $102.0 $46.2 $46.5 -19.1% -19.4% 6.41% $160.9 $98.6 $45.0 $45.4 -17.5% -18.1% 5.25% $140.6 $86.1 $40.2 $40.6 -11.7% -12.2% 5.15% $139.0 $85.1 $39.8 $40.3 -11.1% -11.8% 5.03% $137.1 $84.0 $39.4 $39.7 -10.5% -11.2% 4.65% $131.2 $80.4 $38.0 $38.2 -8.7% -9.1% 4.55% $129.7 $79.5 $37.7 $37.9 $46.2 -8.2% -8.6% -19.3% 4.17% $124.2 $76.1 $36.4 $36.6 $44.7 -6.3% -6.6% -17.6% 3.30% $112.6 $69.0 $33.6 $33.7 $41.2 -1.9% -2.3% -14.0% 3.21% $111.5 $68.3 $33.4 $33.5 $40.9 -1.5% -1.9% -13.6% 3.07% $109.7 $67.2 $33.1 $33.2 $40.5 -0.7% -1.0% -13.0% 2.96% $108.4 $66.4 $32.7 $32.9 $40.1 0.0% -0.2% -12.5% 2.93% $108.0 $66.2 $32.6 $32.8 $40.0 0.3% 0.0% -12.5% 1.67% $94.0 $57.6 $28.7 $28.8 $35.7 7.9% 7.9% -6.7% 0.83% $85.9 $52.6 $26.4 $26.5 $33.0 12.5% 12.4% -2.6% 0.34% $81.5 $50.0 $25.2 $25.3 $31.5 15.0% 15.1% 0.0% 0.00% $78.7 $48.2 $24.5 $24.5 $30.6 17.0% 17.1% 1.8% -0.78% $72.5 $44.4 $23.0 $23.0 $28.5 20.4% 20.8% 5.7% -0.86% $71.9 $44.1 $22.8 $22.8 $28.3 20.8% 21.2% 6.0% -1.37% $68.3 $41.8 $21.8 $21.8 $27.1 23.3% 23.8% 8.4% -4.95% $48.2 $29.5 $15.7 $15.6 $19.9 41.3% 42.0% 25.2% -7.64% $37.9 $23.2 $12.7 $12.7 $16.0 52.3% 53.1% 37.1% HERS System Rehabilitation 2 1 Funding MechanismFunding Mechanism Percent Change in Average IRI on Roads Modeled in HERS Average Annual Investment (Billions of $2006) -24% -12% 0% 12% 24% 36% 48% 60% $20.0 $30.0 $40.0 $50.0 $60.0 $70.0 $80.0 $90.0 $100.0 $110.0 $120.0 Average Annual Investment Modeled in HERS (Billions of Dollars) Non-User Sources Fixed Rate User Charges Variable Rate User Charges 6002 morf egnahC tnecreP Projected Changes in 2026 Pavement Ride Quality Compared with 2006 Levels for Different Possible Funding Levels and Financing Mechanisms Figure 3.6. Relationship between alternative investment levels and pavement quality from the 2008 C&P. 1 The amounts shown represent the average annual investment over 20 years that would occur if annual investment grows by the percentage shown in each row in constant dollar terms. The performance impacts identified in this table are driven by spending modeled in HERS; the figures for “Total Capital Outlay” are included to reflect other spending not modeled in HERS. 2 The amounts shown represent the portion of spending that HERS directed towards system rehabilitation rather than system expansion, which varies depending on the funding mechanism employed. Source: Highway Economic Requirements System.

of the graph in the exhibit shows them as nearly similar. Both Figure 3.5 and Figure 3.6 show logical relationships between investment levels and performance which can be used directly in informing target-setting. The important caveat is that HERS and all other models are not direct reflections of future agency investments at particular funding levels. Figure 3.7 shows the relationship between bridge invest- ment levels and the backlog measures which are commonly used to relate bridge investment to future bridge conditions. Bridges are either termed to be deficient or not deficient (rather than being measured with a sliding numerical scale of degree of deficiency). FHWA’s analysis for the C&P exhibits and the discussion are fully sufficient to inform any target-setting for highway and bridge investments. However, both the HERS and NBIAS mod- els, and the measures they support, are oriented to those invest- ments which are modeled and to the performance measures which are addressed. These include measures related to phys- ical asset conditions, and for HERS, those measures related to user costs, congestion and delay, and congestion and delay costs. NBIAS covers only bridges, and although it includes estimates of impacts on user costs, the lack of information in NBIAS related to highways connecting to the bridges makes it impossible to estimate the user cost impacts of bridge improve- I-48 Annual 2026 Percent Percent NHS Change Change Bridge in Bridge Relative Total Backlog 2 Backlog to Capital On (Billions of Compared 2006 Outlay Total NHS 2006 Dollars) to 2006 5.15% $139.0 $17.9 $7.7 $0.0 -100.0% 5.03% $137.1 $17.6 $7.6 $1.3 -97.4% 4.65% $131.2 $16.9 $7.4 $5.3 -89.6% 4.55% $129.7 $16.7 $7.3 $6.2 -87.8% 4.17% $124.2 $16.0 $7.1 $10.1 -80.1% 3.30% $112.6 $14.5 $6.5 $18.4 -63.8% 3.21% $111.5 $14.4 $6.5 $19.4 -61.8% 3.07% $109.7 $14.1 $6.4 $20.9 -58.9% 2.96% $108.4 $14.0 $6.3 $22.4 -55.9% 2.93% $108.0 $13.9 $6.3 $22.6 -55.5% 1.67% $94.0 $12.1 $5.6 $34.5 -32.1% 0.83% $85.9 $11.1 $5.1 $42.4 -16.5% 0.34% $81.5 $10.5 $4.9 $46.0 -9.4% 0.00% $78.7 $10.1 $4.8 $48.2 -5.1% -0.78% $72.5 $9.3 $4.5 $54.1 6.5% -0.86% $71.9 $9.3 $4.5 $54.6 7.5% -1.37% $68.3 $8.8 $4.3 $57.8 13.8% -4.95% $48.2 $6.2 $3.2 $78.4 54.3% -7.64% $37.9 $4.9 $2.5 $91.1 79.3% Cost to Maintain:3 $4.7 $50.8 0.0% 2006 Spending:4 $4.3 $57.3 12.8% 2006 Baseline Values: $4.3 $50.8 Average Annual Capital Investment Spending Modeled in NBIAS (Billions of 2006 Dollars) 1 Projected Changes in 2026 Bridge Investment Backlog on the NHS Compared With 2006 Levels for Different Possible Funding Levels Figure 3.7. Relationship between alternative investment levels and bridge investment backlog from the 2008 C&P. 1 The amounts shown represent the average annual investment over 20 years that would occur if annual investment grows by the percent- age shown in each row in constant dollar terms. The performance impacts identified in this table are driven by portion of NBIAS-modeled spending on the NHS. 2 The amounts shown do not reflect system expansion needs; the bridge components of such needs are addressed as part of the HERS model analysis. 3 The amount shown is projected to be sufficient to maintain the economic bridge backlog at its baseline 2006 level. 4 The amount shown reflects actual capital spending by all levels of government on NHS bridges in 2006. Source: National Bridge Investment Analysis System.

ment actions. For example, information on whether or not adjacent highways are improved is needed to estimate whether bridge improvements (for example, widening a bridge to have more capacity) would have an impact on levels of service. For FHWA or for the national level, there are no current economic models or management systems which are used for informing target-setting for the safety, environmental, or freight/economic performance measures. As economic, safety, and environmental modeling methods and procedures are developed in these areas, they can be used for national target- setting and potentially adapted for target-setting at states or MPOs. Table 3.3 identifies the emerging state and local tools. At the state level, current asset management systems for pavements and bridges are used by many states to inform per- formance targets and could be more broadly applied. Usefulness of Current Economic Models and Management Systems in Tradeoff Analysis While a detailed description of tradeoff analysis is beyond the scope of this paper, it is nonetheless important to note that the concept of “tradeoffs” summarizes the main challenge facing transportation agencies; there are more needs then resources available to address them. In this environment agencies must continually make difficult decisions on which areas of the trans- portation network to focus their limited resources. Transporta- tion is often a zero-sum game, so additional investment in one area means that an agency must invest less in another. Guidance on the role of tradeoff analysis in the transporta- tion planning process was initially developed through NCHRP Project 08-36 (Task 07).5 This project report described the use of “what if ” scenarios in helping agencies understand the implications of different funding options. For example, what would be the impact on pavement performance if pavement funds are increased by 10 percent over the next 10 years? What would be the impact on bridge performance if this money was shifted from the bridge program? In the context of the overall performance management framework, this type of analysis can help agencies establish relative priorities, set targets, allocate resources, and better manage stakeholder expectations. As with the discussion of target-setting, this discussion of tradeoff analysis addresses the system or program level rather than the project level. Tradeoff analysis also is useful in select- ing among project alternatives. To conduct system level or pro- gram level tradeoff analysis, an agency must have the results of economic models or management systems or other estimates for the relationships between investments of particular types and the performance measures to be impacted. Few agencies have systematically used the results of their eco- nomic models and management systems for tradeoff analysis. A recent pioneering example is the Detroit metropolitan area’s Southeastern Michigan Council of Governments (SEMCOG) which utilizes HERS, asset management systems, and other sources to develop relationships between investment levels and performance measures for a wide range of programs.6 It then graphically presents the results in a manner which I-49 Goal Area Emerging S tate and Local Tools Safety SHSP process that has emerged over the last few years is an excellent base from which to launch performance analysis, while benefit/analysis can support project - level investment decisions within and across safety related plans and programs, including the STIP, HSIP, Highway Safety Performance Plan (HSPP), Commercial Vehicle Safety Plan (CVSP), and others. System Preservation Asset management systems for pavement and bridge apply benefit to analyze and develop improvements in Federal and some state programs, and are probably the most mature systems among the states for developing investment strategies. Mobility/Congestion The congestion management process that originated with ISTEA for TMAs and was extended by SAFETEA - LU provides an excellent base from which to develop benefit and investment strategies for this goal. It will require integration of congestion management systems with benefit procedures such as are used in the Intelligent Transportation System Data Analysis S ystem (IDAS). IDAS has not been widely used to date. Freight/Economic Growth The Freight Analysis Framework and state - specific freight plans (and freight models) are in an evolutionary stage and are not widely used at this point for investment/analysis, but these could be enhanced towards a more comprehensive freight and economic growth performance framework which utilizes benefit. Environment and Community Environmental Management Systems offer good promise for this goal area but are clearly in an earl y evolutionary stage in regard to investment/analysis. Table 3.3. Emerging state and local tools for development of performance strategies. 5Cambridge Systematics, Development of a Multimodal Tradeoffs Methodology for Use in Statewide Transportation Planning, developed as part of NCHRP 08-36 (Task 07), November 2001. 6Guerre, Joseph and Evans, Jennifer, “Applying System-Level Performance Measures and Targets in Detroit’s Metropolitan Planning Process,” January 2010.

would allow decision-makers to address the tradeoffs between investments that achieve alternative levels of performance across different performance goal areas. This is the type of analysis which helps inform the nation, the states, and the regions on how investments in support of various system and performance goals could be traded off against each other. SEMCOG’s investment prioritization process consists of the following steps: 1. Define measures of effectiveness and assess current performance. 2. Analyze the relationship between funding and perfor- mance within each program area. 3. Develop funding scenarios (each scenario represents a dif- ferent way of splitting anticipated funds across the program areas used in the RTP). 4. Present the results of the analysis to decision-makers in a for- mat that enables them to conduct program-level tradeoffs, with the goal of reaching consensus on long-range funding and performance targets for the region. SEMCOG’s LRP is organized by the following program areas: • Pavement Preservation—There are 22,820 miles of pub- lic roads in the SEMCOG region. The maintenance, reha- bilitation, and reconstruction of these roads falls into the pavement preservation program. • Highway Capacity—The highway expansion program addresses recurring sources of congestion. Work in this area includes widening existing roads. • Bridge Preservation—The bridge preservation program area covers work on the region’s 3,560 bridges. • Safety—The work in this program area focuses on improv- ing high-crash locations. • Transit—The transit program covers the maintenance, operations, improvement, and expansion of the region’s fixed-route transit network. • Nonmotorized—This program covers the maintenance and expansion of the region’s nonmotorized network, which consists of bike paths, sidewalks, roadways that accommo- date bike traffic, and other amenities such as bike storage facilities. • Roadway Operations—This program covers traffic oper- ations, studies, and routine maintenance. The bulk of the budget for this program (82 percent) is used for routine maintenance. To support the new prioritization process, SEMCOG revis- ited its list of existing measures of effectiveness (MOE) and selected a single measure in each program area for analysis (Table 3.4). The next step in the process was to analyze each MOE to determine the relationship between future performance and expenditure level. The analysis was first done separately for each MOE. The results were then combined into AssetManager NT so that they could be reviewed and better understood. AssetManager NT is a visualization tool that enables users to explore the performance implications of various resource allo- cation scenarios. The tool brings together analysis results from multiple decision-support tools (e.g., pavement and bridge management systems) and provides a quick-response “what-if” analysis tool for testing different investment options. Asset- Manager NT was originally developed through NCHRP Project 20-57, “Analytical Tools for Asset Management,” and has subsequently been adopted by AASHTO.7 SEMCOG identified discrete funding scenarios which emphasized different themes for investing resources. Figure 3.8 illustrates the results of this process. For each scenario, the fig- ure shows the percent of available funding allocated to each program area and the resulting performance in 2030. It includes the following four scenarios: I-50 Program Area Measures of Effectiveness Pavement Preservation Percent of pavement in good or fair condition Highway Capacity Hours of congestion delay per 1,000 vehicle miles traveled Bridge Preservation Percent of bridges in good or fair condition Safety Fatalities per 100 million vehicle miles traveled Transit Extent of the transit network (the existing network or the region’s transit vision) Nonmotorized Percent of population and employment within ½ mile of a nonmotorized facility Roadway operations Not applicable Source: SEMCOG and Cambridge Systematics Table 3.4. Measures of effectiveness used in SEMCOG prioritization process. 7Guerre, Joseph and Jennifer Evans, “Applying System-Level Performance Mea- sures and Targets in Detroit’s Metropolitan Planning Process,” January 2010.

I-51 Measure of Effectiveness 2030 Target Funding Split 2030 Target Funding Split 2030 Target Funding Split 2030 Target Funding Split Transit System extent Current System Current System 21% < Current System 12% < Current System 21% Transit Vision 41% Pavement % pavement in good or fair condition 57% 57% 21% 49% 18% 85% 31% 40% 14% Bridge % bridges in good or fair condition 85% 100% 6% 100% 7% 85% 3% 80% 3% Expansion hours of congestion delay per 1,000 vehicle miles traveled 2.9 2.6 10% 2.6 10% 3.0 2% 3.0 0% Safety fatalities per 100 million vehicle miles traveled 0.77 0.74 0% NA 7% 0.73 1% 0.73 1% Nonmotorized % pop. and emp. within ½-mile of nonmotorized facility 13% 44% 1% 100% 5% 44% 1% 13% 0% Roadway Operations NA 41% 41% 41% 41% Projected 2010Program Area 2. Public Opinion 3. Preservation First 4. Transit First1. Current Allocation Source: SEMCOG and Cambridge Systematics. Figure 3.8. Funding scenarios with targets. 0 20 40 60 80 100 0 100 200 300 400 500 600 Annual Budget ($M) Pe rc en t i n G oo d or F ai r C on di tio n Current performance expectation Maximum performance Source: SEMCOG and Cambridge Systematics. Figure 3.9. Pavement performance versus funding. 1. Current Allocation. This scenario represents the funding split from SEMCOG’s existing regional transportation plan. 2. Public Opinion. The funding splits in this scenario are based on the results of a recent public opinion survey in which respondents were asked how they would allocate $100 among the program areas. 3. Preservation First. In this scenario a target of 85 percent in good or fair condition is set for pavements and bridges. The remaining funds are then spread among the other program areas. 4. Transit First. In this scenario the entire transit vision is funded. The remaining funds are then spread among the other program areas. After producing results for each program and scenario and for the alternative funding levels, the information was assem- bled in a manner which illustrated what level of performance results the region would achieve at alternative funding levels for each of the program areas. These results that inform the trade- off analysis are shown in Figures 3.9 through 3.13.

I-52 0 20 40 60 80 100 120 0 25 50 75 100 125 150 Annual Budget ($M) Current performance expectation Maximum performance Source: SEMCOG and Cambridge Systematics. Pe rc en t i n G oo d or F ai r C on di tio n Figure 3.10. Bridge performance versus funding. 0 1 2 3 4 5 0 100 200 300 400 Annual Budget ($M) Ho ur s of C on ge st io n De la y pe r 1 ,0 00 M ile s Tr av el ed Current performance expectation Maximum performance Source: SEMCOG and Cambridge Systematics. Figure 3.11. Delay versus funding.

I-53 0 0.2 0.4 0.6 0.8 1 0 5 10 15 20 25 Annual Budget ($M) Fa ta lit ie s pe r 1 00 M M ile s Tr av el ed Current performance expectation Maximum performance Source: SEMCOG and Cambridge Systematics. Figure 3.12. Fatality rate versus funding. 0 20 40 60 80 100 120 0 10 20 30 40 50 60 Annual Budget ($M) % P op ul at io n & Em pl oy m en t w ith in 1 /2 m ile o f F ac ili ty Current performance expectation Maximum performance Source: SEMCOG and Cambridge Systematics. Figure 3.13. Nonmotorized performance versus funding.

After presenting these graphs, SEMCOG also presented the prepackaged scenarios illustrated in Figure 3.8. Members of the Transportation Advisory Council and Executive Committee were then asked to review the scenarios and to do the following: • Indicate their most preferred scenario; • Indicate their second most preferred scenario; or • Fill out a new scenario with preferred funding targets for each program area. The SEMCOG analysis provides a blueprint for using avail- able economic models and management systems and other procedures to progress through target-setting and through a tradeoff analysis for the types of performance measures that were recommended in the overall performance management framework. SEMCOG provides an excellent example of the use of tradeoff analysis in decision-making; however, future research should likely focus on how agencies could implement trade- offs into their performance management processes. 3.5 Topic Areas for Volume II Guidance The previous sections have described how the public and private sectors identify and use performance targets. Based upon these practices, six steps outlined in Volume II of this report provide guidance for state DOTs on how to set and use performance targets. The steps include the following: • Establish Performance Management Framework; • Evaluate the Factors Influencing Target-Setting; • Select the Appropriate Approaches for Target-Setting; • Establish Methods for Achieving Targets; • Track Progress Towards Targets; and • Adjust Targets Over Time. I-54

I-55 Quality data are the foundation of performance manage- ment. Effective decision-making in each element of the per- formance management framework requires that data be collected, cleaned, accessed, analyzed, and displayed. The orga- nizational functions that produce these requirements are called data management systems. There are two key dimensions to creating and sustaining these systems. The two areas are equally important and must be synchronized within an organization to ensure the generation and use of accurate, timely, and appro- priate data. The first area centers on the technical challenges associated with data systems, including development and main- tenance of hardware and software, and the specifications for data collection, analysis, archiving, and reporting. The second area focuses on the institutional issues associated with data stewardship and data governance. Attention to both of these areas is required to assure solid data management systems. Research for this project included an investigation of the ways data management systems and organizational units within a DOT are used to integrate data for purposes of ensuring the use of accurate, timely, high-quality data for decision-making purposes. The research focused on both technical and institu- tional solutions and best practices. This section summarizes the findings of the case studies in Volume III and includes many specific and relevant examples to demonstrate how data pro- grams are supporting decision-making in many private and public sector agencies. 4.1 Introduction The term data program in this report refers to specific data systems that support a business area of the organization. The “program” usually includes the functions of data collection, analysis, and reporting. In the case of a DOT, some examples of these programs include traffic, roadway inventory, safety, and pavement data. In many organizations, including some DOTs, targets are used to measure how well programs are performing. The process to develop targets is, in many cases, based on historical trends. It is often challenging to develop the most-effective targets for assessing how well a program is performing, and using past performance is a good basis as a starting point. More research is needed to investigate the analytical tools that are available for developing targets that most effectively measure program performance in a specific business area. Establishing targets and developing data programs both rely on one basic component, data. Data must be collected, processed, and distributed through a means accessible to decision-makers at all levels of an organi- zation. Data are the basic pieces of information which when processed through a system are available for analysis. The core data pieces transform into information and decision-makers then use this information to manage business areas across the organization. In order for data to effectively meet the needs of the organization, it should be assessed in terms of accessibil- ity, accuracy, completeness, credibility, timeliness, and asso- ciated risks. A data risk assessment and management plan is often used to identify potential and known risks, assign persons and offices responsible for handling the risks, and for developing risk mitigation plans. Using a risk management plan strengthens the overall data management program within the organization. More information on the assessment process and data management is discussed in the following sections of this report. Within any organization, data serves as the critical link between business areas of the organization. In a DOT, these areas often include operations, planning, and production. Typical office functions within Planning include long-range planning, policy planning, and traffic data collection. Oper- ations often include traffic engineering, safety, and mainte- nance functions. Production refers to construction within a DOT. There are some similarities and differences in the types of data which are used to support each of these areas. Oper- ations, planning, and production functions often also rely on Human Resource type data, including available staffing for C H A P T E R 4 Data Stewardship and Management

particular projects and financial data for projections of avail- able funding. The similarities and differences of the types and needs for data are best managed within a data management framework, which includes data standards, definitions, policies, and pro- cedures for how data are to be collected, processed, and used within the organization. More recently, data and performance measurement pro- grams that support operations and the link between operations and planning in state DOTs are becoming more prevalent. This need is driven by several factors, including increases in conges- tion nationwide, an increasing emphasis on safety and envi- ronmental factors along with a reduction in resources, and the inherent need to keep the transportation system operating effi- ciently. In addition, technology to generate, archive, and ana- lyze transportation data along with the increasing number of private data sources suggests a need to harness and leverage the information. Also recent legislation from the Federal government and state legislatures places an increasing emphasis and respon- sibility on states to be accountable to the citizens for their expenditures. Performance management is one of the ways in which DOTs are responding to this need. A lot of the data and information needed for performance management is available through the planning division of a DOT. This includes data on the extent, condition, and performance of the transportation network in the state. A strong performance-based manage- ment program, such as the one adopted at the Mn/DOT, helps that agency to align its strategic goals with its business and supporting data programs in order to best meet the needs of the citizens of Minnesota. It is this link of performance measurement data which sup- ports planning, operations, and all areas of a DOT that is the subject of the remainder of this chapter. This report will exam- ine the ways in which data systems are linked to performance measures in order to support stronger incentives for agency leaders to invest in their data programs as strategic assets and to allocate resources using a performance-based management approach. The material in this chapter of the report is organized in the same order as the survey instrument used for the case studies. The sections are in the following order: • 4.2 Elements of Effective Data Stewardship and Management, • 4.3 Organization and Governance, • 4.4 Data Sharing, • 4.5 Documentation and Reporting, • 4.6 Technology, • 4.7 Relationships to Target-Setting and Resource Allocation, and • 4.8 Summary of Success Factors and Obstacles. The discussions in each section refer to the case studies found in Volume III. 4.2 Elements of Effective Data Stewardship and Management This section examines the institutional impact of imple- menting data stewardship and governance in an organization. Figure 4.1 illustrates the hierarchical relationship between data management, data governance, and data stewardship. I-56 Data Stewardship (Stewards, Owners, Custodians) Data Governance (DG Board, Stakeholders, DG Maturity Model) Data Management Source: Modified from Figure 1 Data Governance Team, The Data Governance Maturity Model. White Paper, RCG Information Technology, 2008. Figure 4.1. Data management, data governance, and data stewardship.

The examples in this section illustrate how some DOTs and other public or private sector organizations have defined these terms. The data governance role primarily represents the individ- uals responsible for establishing overall policies, standards, and procedures that are to be followed by the organization. The data stewardship role represents the team of individu- als throughout the organization who are responsible for enacting these policies and procedures on a daily basis. The data management program can be considered the umbrella overseeing all activities related to the management of core data systems. Data management can be defined as the development, exe- cution and oversight of architectures, policies, practices, and procedures to manage the information lifecycle needs of an enterprise in an effective manner as it pertains to data collec- tion, storage, security, data inventory, analysis, quality control, reporting, and visualization. Data governance can be defined as the execution and enforcement of authority over the management of data assets and the performance of data functions. The management of data assets for an organization or state DOT is usually accom- plished through a data governance board or council. This role is critical in successfully managing data programs that meet business needs and in supporting a comprehensive data busi- ness plan for the organization. More information on data governance is included in Section 4.3. Data stewardship can be defined as the formalization of accountability for the management of data resources. Data Stewardship is a role performed by individuals within an organi- zation known as data stewards. For example, the Data Stewards as defined by the Kansas State Department of Education (KSDE) are the individuals who manage the definition, produc- tion, accountability, and usage of data. Some of the duties of the data stewards at KSDE include the following: • Provide data analysis related to individual specific program area; • Regularly evaluate the quality of the data; • Identify opportunities to share and re-use data; • Enforce data quality standards; • Communicate business needs for data security; and • Communicate criteria for archiving data. In addition to Data Stewards for individual data programs, such as traffic, roadway inventory, safety, and pavement data, DOT also may have enterprise Data Stewards who are respon- sible for maintaining the enterprise data warehouse. Their roles and responsibilities are similar to those just described, but, on a much larger scale. The RCG Information Technology Company further iden- tifies two categories of data stewards. One is the day-to-day steward who has the responsibility and authority to make deci- sions about data for a given system and the other is the issues/ practices stewards who are usually the supervisors or opera- tional technicians responsible for securing the data or using the data for high-level decision-making. WSDOT has defined roles for Business Stewards and Tech- nical Stewards, as well as other steward roles who ensure that standardization is used in the collection, storage, integration, and reporting of data in order to meet business needs. Using defined roles for business and technical stewards, WSDOT recognizes the importance of building a partnership between the business and information technology segments of the department, which helps to sustain a robust data governance framework beyond changes in administration. In addition to data stewards, there also are Data Work Groups and Communities of Interest groups (COI) associated with the support and/or use of data in an organization. The Alaska Department of Transportation and Public Facilities (ADOT&PF) Program Development Division defines these two groups in the following way: • Data Work Groups (DWG) includes the association of peo- ple who collect and provide data for specific data programs and who establish business process rules for a specific sys- tem. The DWG make recommendations to the data gover- nance council on the development or enhancements to data programs which support business operations of the agency and/or the Program Development Division. • Each Community of Interest represents an association of people with a common interest as users of a specific data system. In the case of Alaska, the COIs include representa- tives from internal business areas and external entities and provide input through outreach and communication with the Program Development Division on the data and infor- mation needs regarding specific data programs. The role of data ownership also is an important part of the data governance framework. Data owners are somewhat dif- ferent from the data stewards. While data stewards tradition- ally have day-to-day responsibility for ensuring quality data is available from a particular system and for managing the definition, production, and usage of the data, data owners have a higher-level of responsibility to ensure the protection of the data and authorize access to various data applications in their business area. In a public agency such as a DOT, the divisions or offices of the DOT may serve as data owners for specific applica- tions supporting their business area, such as planning, traf- fic operations, and engineering. For example, FDOT has an established policy that clearly delineates the responsibilities of the IT division versus the office data owners. Custodians and owners outside of the IT department are responsible for I-57

ensuring that sufficient controls are established within their area of responsibility to ensure the accuracy and complete- ness of data and that data comes from the appropriate source for the intended use. In private sector companies, any group, other than the IT department, does not typically own data. Some private sector organizations also have performance measurement groups whose responsibility includes data ownership. To clarify the definitions and roles and responsibilities for stewardship and data management within a DOT, Mn/ DOT has developed the following draft definitions as part of their overall development of a Data Business Plan for the Department: • Data Management Data management is the development, execution, and oversight of architectures, policies, prac- tices, and procedures to manage the information lifecycle needs of an enterprise in an effective manner as it pertains to data collection, storage, security, data inventory, analy- sis, quality control, reporting, and visualization. • Data Custodians Data custodians are the information technology professionals responsible for the maintenance and security of databases, hardware, and software used to support application systems. • Data Owner Data owner is a role or group who is empow- ered to make decisions about how a data entity can be struc- tured, manipulated, or used. • Data Stewards Data stewards are individuals accountable for the accuracy, integrity, and timeliness of the data as well as for informing users of the appropriate use of data. • Data Stewardship Data stewardship is the formalization of accountability for the management of data resources. • Data Users Data users are the individuals who have author- ized access to retrieve data from information systems to conduct business. A more detailed discussion of how the public and private sector have integrated these roles into their organizations follows in Section 4.3. 4.3 Organization and Governance There are a variety of data governance models which can be used by organizations to develop their own data gover- nance framework. Each model has its own advantages and disadvantages. The main concept to keep in mind is that there is not a one-size-fits-all approach when developing a data gov- ernance framework and each agency should develop a model that best suits the needs of the organization. This section dis- cusses some of the various models that have been used for developing data governance and the associated data steward- ship roles and responsibilities. There are institutional issues, as well as technical chal- lenges, in any organization considering implementing data stewardship and data governance. Using a well-defined data governance model helps to ensure successful sharing of responsibilities with respect to data management functions. A data governance board or council can be established to serve in the oversight role for managing the data governance activi- ties of an organization. The members of this group are usually high-level managers and executives of the agency or company. Likewise, there are corresponding data stewardship roles defined with responsibility to collect, maintain, and use data in accordance with the policies instituted by the board. In many cases, data also are recognized as a valuable asset in both the public and private sector and therefore, agencies allocate a certain portion of their budgets to the proper man- agement of data programs and to securing and protecting the integrity of the data systems within the organization. Protect- ing and securing of data systems includes establishing access to the systems using authorized login and password proce- dures, and establishing back-up and recovery procedures for data applications and databases. Investments in data programs in the private sector are prima- rily judged on a cost/benefit basis. If the cost of an upgrade proj- ect cannot be justified on the savings it will bring, it does not proceed. Departments must carefully demonstrate the return on investment (ROI) to senior executives to gain support for development or continued use of various programs. This also is true in the case of a DOT that must often demon- strate the value of a given program to the state legislature or the public, who sustains these programs through their tax dollars. For example, the underlying data programs that support per- formance measures at Washington DOT were critical in con- vincing the State legislature to continue investments in a fish barrier removal program versus other programs that may have a higher legislative priority. Figure 4.2 illustrates the important link between an agency or company’s strategic mission and goals and the data gover- nance framework that is in place to manage the data programs to ensure that the highest quality data are available in a consis- tent and timely manner to support core business functions in all areas of the organization. The data users and stakeholders as well as the data stewards and custodians each serve a critical role in developing and maintaining data systems, which meet user needs. To have a well functioning data governance structure, there must be open channels of communication between the providers of data and information and the users of the data. Outreach pro- grams from the data providers to the data user community are an effective way to maintain this relationship. The benefits of implementing a data governance framework include improved quality of data collected and reported, which in turn, has a positive impact on decision-making. Figure 4.3 I-58

I-59 Strategic Vision, Mission, Goals for Data Agency Data Programs Data Governance Board Data Users and Stakeholders Division(s) Mission(s) and Goals Data Steward and Custodians Figure 4.2. Overview of a general data governance framework. Source: www.Datagovernancesoftware.com. Figure 4.3. Example data management structure.

shows an example of a potential data governance framework. It shows each of the 10 universal components of a Data Gover- nance program. Regardless of the focus of a data program, it will include each of these components to some extent, although the emphasis of each component will vary according to the program’s objectives. One way to organize these components is by looking at why the program exists, what it is doing, who is involved in the efforts, and how they are performing processes to provide value to the organization. Another way to look at a Data Governance program is to consider the following: • The rules that the program is making (such as policies, requirements, standards, accountabilities, and controls) and the rules of engagement that describe how different groups work together to make those rules and enforce them; • The people and organizational bodies involved in making and enforcing those rules; and • The processes that these people follow to govern data, while creating value, managing cost and complexity, and ensuring compliance. In general, private sector companies do not use data gover- nance protocols or explicit standards covering data parameters, data cleanliness, or data revisions. However, they are often experts at sharing data between business units and across com- panies. There are several different models and combinations of data governance approaches. Some examples along with advan- tages and challenges are discussed in the following sections. Washington State Department of Transportation (WSDOT) Several public sector organizations have been proactive in implementing some type of data governance framework or are in the process of doing so. WSDOT has a Data Council and a Data Stewardship Council to help support data governance at WSDOT. WSDOT also further defines two categories of Data Stewardship: Business Stewardship and Technical Steward- ship. The Business Stewards are executive, managerial, and operational stewards, while the Technical Stewards include the more traditional roles of system architects and database administrators. While these roles may be defined for larger application systems, WSDOT does acknowledge that for smaller applications there may be just one or two people responsible for maintaining a specific data system, and there is no formal stewardship role defined for them. WSDOT Advantages The WSDOT model started with a smaller goal of gaining support at the executive level for a data policy. This eventu- ally led to the support for the development of a data catalog which was followed with the development of a Data Steward- ship program. This proved to be an effective method for beginning implementation of a data governance framework. The role of the Data Stewardship Council is to address data issues across the Department. An additional Data Council also was established to set standards for such components as data architecture, data modeling, and data stewardship. The Department successfully encourages the use of data standards, by linking the funding of various programs, when possible, to the use of those standards within the organization. This is an effective means for gaining compliance with the data management policies and procedures for the Department. Defining distinct roles for Business and Technical Stewards also serves to strengthen this critical partnership and the overall data governance framework for the Department. WSDOT Challenges The WSDOT case study indicates that in some areas there are no formal stewardship roles, and there may only be one or two individuals responsible for supporting a data system. It would strengthen data governance at the Department if all individuals who are performing stewardship roles are clearly identified as such. One method of doing this is by providing their contact information in a Data Catalog, along with the application which they support. Kansas State Department of Education (KSDE) KSDE, like DOTs, must meet Federal reporting require- ments and the development of a data governance framework at KSDE grew out of the need to meet those requirements on an annual basis. There also are clearly defined Federal per- formance measures that each school district must adhere to in order to continue receiving funding and accreditation. Therefore, the data management policies at KSDE were established to direct the data collection, maintenance, and reporting activities of the department to ensure that perfor- mance measures are reported in a consistent, timely manner each year. KSDE also developed handbooks to support data gover- nance at the department. This includes handbooks for a Data Governance Board, Data Request Review Board, and Data Stewards. Each handbook includes the following components for each board: • Mission statement for the board; • Scope and responsibilities of the board members; and • Goals and objectives of the board. I-60

KSDE Advantages There are many advantages associated with the use of the KSDE data governance model. In addition to having a formal Data Governance Board established by an official charter, KSDE also has a Data Request Review Board, which was estab- lished to support the flow of data and information requests. This Board approves, prioritizes, and assigns requests related to data and information systems. When the need arises, this Board elevates issues to the Data Governance Board for reso- lution. There is a clearly defined channel of communication for processing requests and for addressing issues as needed. KSDE also has developed handbooks for the participants in the data governance structure within the organization. These include handbooks for the Data Governance Board, Data Request Review Board, Data Stewards, and all other KSDE data governance participants. This helps everyone to better under- stand their role in supporting the overall data governance framework at the agency. KSDE also involves data stewards in the initial planning stages of system development, as subject matter experts, to ensure that the needs of the organization are met, while the planned project development complies with all standards, definitions, and procedures for use of data at the Department. KSDE Challenges The primary challenge at KSDE was in developing a model that suited their needs. To do this, they researched information regarding data governance and attended webinars and presen- tations to learn more about it. Through this process, they were able to develop a data governance model that best suited their needs. They also have senior-level support for using data gov- ernance at their organization which has proven to be an effec- tive management tool for ensuring that their state education programs remain in compliance with Federal requirements for public education. Hennepin County, Minnesota Hennepin County considers data governance and data stewardship roles to be shared responsibilities within the county. There is an IT Governance Board, established by an official charter, which is comprised of Department Direc- tors, the Deputy County Administrator, and the County Administrator. The county also has an IT Steering Commit- tee that provides advice on how proposed information sys- tems will fit into the current IT architecture at Hennepin County. Each of the departments in the county are expected to work with staff from other departments in the develop- ment of new information systems, while the IT department ensures that the new systems comply with county IT poli- cies and standards. Hennepin County Advantages The Hennepin County data governance model has been successful in large part, due to the strong support from the County Administrator for performance-based management. This agency also uses an official charter to empower an IT Gov- ernance Board and IT Steering Committee. Establishing such a formal structure serves to strengthen the overall data gover- nance structure for the county. While each department in the county can develop new information systems, each office is expected to coordinate with the IT office to ensure that any new systems comply with overall county IT policies and stan- dards. This provides for a streamlined data management pro- gram which effectively meets the needs of all departments within the county. Hennepin County Challenges Hennepin County has developed dashboards and scorecards to make it easier for managers to monitor progress in various county programs on a weekly, monthly, quarterly, and annual basis. Since the development of the scorecards is an ongoing process, some departments of the county have scorecards and others do not. The county has acknowledged that there is a need to have a countywide scorecard, and their challenge is to eventually have the ability to review program performance across all departments for the county. Alaska and Minnesota Departments of Transportation In some DOTs, such as Alaska and Minnesota, the develop- ment of data governance frameworks is just beginning as part of the development/implementation of a Data Business Plan. The roles of data governance and data stewardship will even- tually become part of a more structured approach for manage- ment of data programs within both of these agencies. Mn/DOT also has established a Business Information Council (BIC) to guide the development of their Data Business Plan, which will ultimately include the implementation of a data governance framework for the department. Alaska and Mn/DOT Advantages The advantages of developing a data governance framework at DOTs such as in Alaska and Minnesota are yet to be fully realized, as each of these agencies are in the preliminary stages of defining what their data governance model will look like and how it will be implemented at their respective agencies. Alaska has taken the approach of developing a data gover- nance model for a specific business area, i.e., planning. Mn/ DOT is developing a data governance model for the department as a whole. The commonality between these two approaches is I-61

that they are both being developed as part of a Data Business Plan. Each agency understands the significance of the contri- butions of its data programs in supporting business operations and is engaging in developing data business plans which will be used to manage the current and future investments in their data programs. Alaska and Mn/DOT Challenges Many of the challenges from the Alaska and Mn/DOT exam- ples are shared by other organizations as they begin to develop and implement data governance. There is the need to gain executive-level support for a data governance program. Some agencies are more successful than others in gaining initial exec- utive support. There also is the need to assess the current state of data programs, identify areas where gaps in data and infor- mation exist, and prioritize those needs as part of short-term and long-term investment plans for data programs. Depending on the size of the organization and the number of data systems to be assessed, this process can take several months to complete. Identifying Communities of Interest, which are basically the users of the data systems, also is an involved but necessary process to ensure that any data programs developed actually meet user needs. Ultimately, each DOT will need to design a data gover- nance framework that best meets the business needs of their organization. Port Authority of New York/ New Jersey (PANYNJ) The Port Authority of New York/New Jersey (PANYNJ), while acknowledging that there is not an official data gover- nance board or council overseeing the management of its data programs, does have a governance process for considering and approving potential enhancements to data systems which support core business functions. Any requested changes to an existing system from one line department must be considered for potential impact to the other line departments. Any line department wanting an application developed also must coor- dinate this effort through the IT department. This example demonstrates the importance of having some type of governing or coordinating function which oversees the development and maintenance of application systems within an agency, whether or not the function resides with the IT department, as it has traditionally in the past, or with the estab- lishment of an official data governance board or council. PANYNJ Advantages The data governance example from PANYNJ demonstrates that it is not always necessary to have a complicated data governance structure. If a simpler model meets the needs of the organization and it is just as effective as a more detailed model, then there is no need to overdo the data governance framework. PANYNJ Challenges In the PANYNJ example, the responsibility for data gover- nance basically belongs to the IT department. It is important that the other line departments continue to work with the IT department in the development and implementation of any new systems or redesign of existing systems. While this arrange- ment works well at PANYNJ, it may pose more of a challenge at other organizations. Each of the previous examples illustrates the importance of developing and implementing a data governance structure that suits the needs and size of the organization. Ultimately, since the success of implementation of data governance relies on a strong partnership between the business offices and the IT office of the agency, it is advisable to engage the IT profes- sionals as early as possible in developing the data governance model for the agency and a corresponding strategy for imple- mentation of the model. 4.4 Data Sharing One of the key factors in the success of data management programs is having well-defined procedures, methods, and tools for sharing data both internally and externally. Sharing data provides benefits not only in terms of reducing costs asso- ciated with having multiple offices collect the same data but also in terms of resources dedicated to maintaining duplicate data systems. Data sharing can reduce the risks of providing different responses to the same question when a single source of data is used for reporting and decision-making in a specific business area, such as data that are used for reporting per- formance measures for various programs. In an effort to reduce the cost of data collection, many organizations utilize formal data sharing agreements with exter- nal partners and agencies. An example of this type of arrange- ment is demonstrated in the exchange of data between a DOT and the local police department or division of motor vehicles. The Maryland Motor Vehicle Administration (MVA), for instance, uses Memorandums of Understanding (MOU) with all organizations that it shares data with. Some of the types of data that the MVA obtains from external sources include the following: • National Driver Register data; • Social Security data; • Commercial Drivers License data; and • Insurance data. I-62

The MVA also shares data with the public, including driv- ing records, vehicle data, and title records. Three excellent examples of sharing data in the safety arena come from Alaska. Alaska’s Multi-Agency Justice Integration Consortium (MAJIC) is an active group whose members include twenty key agencies including the Department of Law and Criminal Division, Association of Police Chiefs, Division of Motor Vehicles, Health and Social Services, Department of Transportation, and Department of Public Safety. Each agency signed a Memorandum of Agreement. The mission is “to help agencies more efficiently share complete, accurate, timely information in order to enhance the performance of the crim- inal justice system as a whole.” They have established measures of data completeness, accuracy, and timeliness. The system used in Alaska for sharing electronic citation and collision data is called Traffic and Criminal Software System (TraCS). TraCS is an automated data collection system that includes electronic ticket and collision forms, DWI forms, arrest and incident forms, commercial vehicle inspection forms, and the use of GPS devices and GIS maps. TraCS increases safety by significantly decreasing the amount of time it takes an officer to write a traffic ticket or collect collision report information; greatly improving the accuracy of collision and ticket data that police collect; reducing the time officers spend on paper work, thus increasing their availability for patrol; and reducing dupli- cate data entry by police, DMV, and the courts, which saves time and minimizes errors. The third example of successful sharing of safety data is the Alaska Traffic Records Coordinat- ing Committee (ATRCC) which was created to bring people together who are interested in reducing traffic injuries and deaths by improving the timeliness, accuracy and consistency of traffic crash data. The ATRCC meets at least once each month to discuss ongoing and upcoming safety data sharing projects. PANYNJ also has formal data sharing agreements with local police, Federal, and other state government agen- cies. The Aviation Department of PANYNJ uses data from the FAA databases to integrate with their internal Aviation division applications. Due to the need to have data on extent, performance, and condition of off-state system or local roads to meet Federal reporting requirements, several agencies use data available from MPOs or cities and counties to integrate into the state road network. In the case of the MTC of the Bay Area in Cal- ifornia, there is a requirement that local jurisdictions provide updated pavement condition data to the MTC, or they will not be eligible to receive Federal grant funding. Likewise, each state DOT must submit their annual public road mileage cer- tification to FHWA, as well as the Highway Performance Monitoring System (HPMS) report, or they too risk losing Federal highway funds. An excellent example of data sharing between a state DOT and external entity comes from the OOCEA regarding the sharing of travel time data with FDOT. FDOT routinely accesses the data and traffic monitoring sites used by OOCEA as part of its oversight responsibilities for the road network in Florida. The City of Coral Springs, FL, also participates in cooper- atives to share comparative and competitive data with others in municipal government. They exchange data with a group of 100 cities nationally to benchmark their performance com- pared to other cities of similar size. Through this data sharing agreement, they can design ways to improve performance and services provided by the City. Internal access to data and data sharing is just as critical as having external data sharing arrangements. Internal data are often shared using a data warehouse by using data marts to provide data query, analysis and reporting capabilities. The sharing of metadata to describe the purpose and use of the data also helps to ensure that the correct data are used for making strategic decisions affecting the organization. Some organizations also support data sharing internally with a Knowledge Management (KM) system, which is an electronic repository of all types of information such as the following: • Data standards, policies and definitions for all business application data, including metadata; • Work processes used to support a business program using a specific data system; • Lessons learned regarding use of IT tools, or other similar data systems; • Agency or department policy and standards regarding the use of data; • Reports which monitor the performance of a given data program; and • Data models for critical data systems. The use of a KM system allows an organization to do the following: • Leverage the expertise of people across the organization; • Manage business environments and allow employees to obtain relevant insights and ideas appropriate to their work; • Facilitate and manage innovation and organizational learning; • Make available increased knowledge content in the devel- opment and provision of products and services; • Achieve shorter new product development cycles; • Increase network connectivity between internal and exter- nal individuals; and • Manage intellectual capital and intellectual assets in the workforce (such as the expertise and know-how possessed by key individuals). I-63

The U.S. Army Armament Research, Development, and Engineering Center (ARDEC) has a Knowledge Management office which is responsible for maintaining historical and cur- rent information about armament manufacturing, arma- ment failure reports, and lessons learned. The lessons learned information is very valuable in reducing the learning curve for new engineers. A similar use of a KM system at the Virginia Department of Transportation (VDOT) has proved to be a valuable source of information on lessons learned, thereby reducing the learning curve for employees who may be new to supporting a specific data program. As part of their Data Business Plan implemen- tation, the ADOT&PF Program Development Division also is investigating the ways in which a KM system can benefit the division and regional offices that support statewide planning programs. Data sharing, internally and externally, is often done through the use of published monthly or annual reports. For example, the MTC publishes all available data on the Internet and pro- duces an annual State of the System report that includes infor- mation on congestion, transit ridership, and regional statistics. A combination of dashboards and scorecards also are used to share data with internal and external departments and agencies as well as the public. The KSDE provides public access to the KSDE Report Card which shows how well the school districts are performing according to national and state standards. Similarly, DOTs use such systems as COGNOS to display dash- board information about the performance of the department in such areas, including safety, traffic operations, engineering, maintenance, and financial information on project costs. The VDOT has such a dashboard available for review by the public. COGNOS is described in more detail in Section 4.5. Hennepin County also uses COGNOS to display perfor- mance measure scorecards for departments throughout the county. This allows managers and staff to monitor perfor- mance of vital service programs routinely for the citizens of the county and to address issues as needed when performance is less than the targeted levels. The use of scorecards also is a means to quickly identify where performance targets may need to be adjusted, either up or down, for a given program. Each of the previous examples illustrate how various agen- cies use data sharing methods and tools to reduce costs of data collection efforts and improve the decision-making process by providing data in an easily understandable format such as executive-level dashboards or scorecards, or through the use of published reports for internal and external customers. Sharing of information, such as lessons learned and work processes used to maintain data systems, reduces the learning curve for new staff. This results in increased productivity and shorter project development timelines. Ultimately, this can result in cost-savings to the organization. One of the most important benefits, however, is in improving decision-making at all levels of the organization, when information is shared and is easily accessible to everyone. It also is important to standardize data formats and defini- tions and to facilitate the integration of data from many sep- arate data sources. Having a single source of data to provide information on a weekly, quarterly, or annual basis greatly improves decision-making at all levels. In the case of a DOT, this standard source of information also sustains credibility with external customers, who include the legislature, other state agencies, the Federal government, and organizations in the private sector as well as the general public. The purpose of having effective data sharing practices is to provide the highest quality data, which eventually becomes information and then knowledge. It is easier for managers to make decisions in a timely manner when they have the best available information accessible to them. It also is easier for staff to provide the stewardship responsibilities for maintaining crit- ical data systems which support business operations when they have access to knowledge about how the data systems work and how they support the business needs of the organization. 4.5 Documentation and Reporting There are several ways an organization documents their data systems to provide information and report to managers and policy-makers on-demand. The approaches used for docu- menting and reporting information vary from group to group. A state DOT routinely provides reports to the public and the legislature as well as to other Federal, state, or local agencies. The DOTs, therefore, must invest a certain amount of their budget in maintaining needed documentation and generating standard reports on an annual, quarterly, or monthly basis. Standard documentation is traditionally kept in the form of data diction- aries, metadata descriptions attached to data files, and the use of data catalogs. While this responsibility traditionally resides with the IT experts, it is becoming more common for the responsibility to be shared with business units in the organiza- tion in order to ensure consistent use of data and business terms throughout the organization. Both the ADOT&PF Program Development Division and Mn/DOT are investigating ways to store and disseminate meta- data information to the decision-makers in their agencies. This includes the establishment of metadata standards and the abil- ity to deliver this information so that policy-makers use the right data systems for making business decisions. Another component of internal documentation and report- ing involves establishing mechanisms for tracking changes to data systems. Change tracking mechanisms help to ensure that the prioritization of requested system changes is in accordance with the primary goals and objectives of the agency. It also is advisable to involve all necessary business units in the discus- sions regarding system changes which impact multiple business I-64

areas, prior to designing and implementing those changes, to alleviate potential problems in the future. This is the approach used at the PANYNJ where any department-initiated request for changes to an existing system must be coordinated through the IT Department and must involve a review by other line departments to ensure that all impacts are discussed ahead of time. While determining what type of documentation to maintain and what standards are to be used is important, it is equally important to ensure that the documentation is archived in a repository that is sharable across the enterprise. This is where the use of a KM system, as described in Section 3.4, can provide great benefit to the organization. The use of an enterprise data warehouse also can provide widespread access to data and information to the organization. A data warehouse using Extract Transform Load (ETL) tools mines data from source databases and loads the data into data marts. The data marts can serve not only as a repository for standard reports but also provide query capabilities to allow for analysis of data as needed. It is imperative, in this age of technology, that reports are easily accessible and available in multiple formats such as MS Word, Excel, and PDF and can easily integrate digital images and GIS maps into the reports. The MTC demonstrates this type of flexibility in their StreetSaver program which allows for integration of GIS maps from various external sources through on-line portals. Smaller agencies can use this data to present information on maps to their local city councils in order to request funding for critical projects. While many of the agencies interviewed recognized the importance of having easy mechanisms for executives to run reports on-demand, some private sector companies simply compile books of operational performance metrics and distribute these within their organizations for critique and self-improvement. If agencies and companies do not already have systems sim- ilar to COGNOS for displaying performance data in a dash- board format, many are actively seeking ways to develop such systems. One example of this is from the Maryland Trans- portation Authority (MDTA), which is working to develop an on-line on-demand access for executives to run reports simi- lar to their Finance at a Glance report, which already is in use by MDTA staff. Using a public report card, as demonstrated by the KSDE, also is a powerful motivator for schools to continue striving for excellence in education in Kansas. This allows for a com- parison among school districts and assessments of perfor- mance relative to national standards. Similarly, the “Status of the Nation’s Highways, Bridges, and Transit: Conditions and Performance Report to Congress” issued by the Federal High- way Administration gives a state-by-state comparison of the condition of public roads in each state. States can then assess how well their roadway system compares with other states of similar size and population. Depending upon the size of the organization it is possible to purchase off-the-shelf software to handle the tracking and management of data, as is the case in the city of Coral Springs, Florida. The city uses a web-based system known as Active Strategy to handle the functions for managing their performance data. Just as implementing data governance does not follow a one-size-fits-all approach, each agency should determine their needs for managing their data systems and their documenta- tion and reporting requirements, and invest in systems that best suit their business needs. 4.6 Technology The role of information technology in an organization, such as a DOT, is generally a centralized function, with technical sup- port provided by a distributed network of IT professionals throughout the organization. Sometimes, additional techni- cal support is provided through the use of external outsourced services from the vendor community. If it becomes necessary to outsource IT support, strengthening the internal flow of data and communication of information is even more critical. This is where the use of Business Intelligence (BI) tools is most effec- tive. Several of the available tools are discussed in this section. A part of the institutional arrangement for supporting data management, governance and stewardship also includes estab- lishing policies and standards for collection and use of data within the organization. This is usually a function of the IT office, as well. There are benefits as well as disadvantages or challenges that can occur based on centralizing this responsi- bility with the IT office. Some of the issues are related to the organization and access of data and may include the following: • Flow of data—The flow of data and information needs to be coordinated but not controlled in such a manner that it limits or impedes access to the data by those needing it to support daily business operations. • Ability for different divisions and individuals to query data—Query capabilities for data applications and systems should be allowed from different divisions and individuals in order to enhance their ability to support core business functions across the agency. • Flexibility—There also needs to be as much flexibility as possible designed into the application so that there is poten- tial for future integration of new technology for data collec- tion, processing, and access and reporting from the system. • Stovepiping—The historical approach to developing appli- cations in silos where an application only meets the needs of a particular area of the organization has led to “stove- piping.” This inhibits the integration of data from various I-65

applications and limits the ability to provide consistent, high-quality data from a single source to support decision- making. • Security—Determining what level of security to assign to data systems should be done in consultation with the IT office and the business owners and stewards for the data systems. It is the responsibility of both users and providers of data to ensure that the data systems are protected from unauthorized use or misuse by others internal or external to the organization. • Efficiency—Utilizing the best available technology tools helps to ensure efficiency in the collection, processing, and reporting of data for various systems. • Response to individual user needs—Ultimately the data programs and systems need to be available to respond to individual user needs, whether the user is the front-line employee responding to requests from the public or inter- nal and external offices or agencies, or the user is a senior- level executive needing the most accurate information within a short timeframe. There are many BI tools available which can help benefit the organization in addressing these challenges. Some of these tools and their benefits are described in the following section. Business Use Case Models Business Use Case models are not only used to help identify what data are shared within an organization and external to an organization, but also to identify who is involved in the shar- ing of data. RCG Information Technology Company found that many times business executives were not aware of the relationship of which data systems were used to support key business functions. Developing data business models helps to identify this for executives who also have authority over fund- ing for those data programs. The use of data modeling tools helps the agency to develop enterprise data systems, which best meet the business needs of the organization. These tools are used to develop the data architecture for application systems. The Data Architecture describes the activities required to obtain and maintain data that supports the information needed by the Corporation’s major business areas. Data and information are different. Data is the foundation of information. Data is the raw material that is processed and refined to generate information. Information consists of a collection of related data that has been processed into a form that is meaningful to the recipient.8 Knowledge Management Systems KM systems are used to support the internal sharing of data and information within the organization. Knowledge manage- ment systems are beneficial because they generally accommo- date a wide variety of formats for storing data and information. Reports as well as digital images can be archived and retrieved through the search and retrieval functions of the KM system. A KM system also can be used to store lessons learned which is very valuable in reducing the learning curve for new staff. Some organizations, such as the Virginia DOT have even established an office to oversee knowledge management processes within their agency. VDOT is joined by Alaska, which is now investigating the use of knowledge management at the Program Development Division as part of their Data Business Plan development for the Division. The U.S. Army also is very actively engaged in the use of a KM system at the U.S. ARDEC as discussed in Section 4.4 of this report. Archive Management Systems One of the biggest challenges faced at many of DOTs is managing archived data. Some agencies have invested in the development of archive management systems and others have outsourced the archiving function to other agencies or the pri- vate sector. This is the case with the OOCEA, where most of the data maintenance work that is outsourced is predominantly devoted to managing the archive data. The decision on how to approach archiving of data should be properly scoped to meet the needs of the organization and should take into considera- tion the amount of data to be archived, for what period, and the costs associated with archiving data. Risk Management Risk management has traditionally been used with Asset Management systems and is becoming more prominent in the use with data systems in DOTs as well. The benefits of estab- lishing a risk management program include early detection of potential problems, which provides enough time to develop a strategy to avoid risk. The following are the five basic steps in developing a risk management plan: 1. Identify the risks; 2. Assess/Analyze the risks; 3. Develop a plan to mitigate (avoid) the risks; 4. Assign resources to handle the risks; and 5. Monitor ongoing and potential risks. There also are tools available to help an agency develop a risk management plan. Some of the tools are briefly described in the following paragraphs. I-66 8Implementation of E-Government Principles, AUDIT REPORT, Report No. 05-018, May 2005.

A Risk Register can be used to record information about the risk. This includes a risk identification number assigned to the risk, the threat that may precipitate the risk, the event that will trigger the risk, and the qualitative analysis to be per- formed related to the risk. This analysis can include the prob- ability of the risk occurring (low, medium, high); the impact to the organization if the risk occurs (low, medium, high); and the development of a risk matrix which measures the proba- bility of the risk occurring versus the impact if the risk occurs. Table 4.1 provides an example of what a Risk Register might look like, regarding HPMS 2010 and Real-Time Traffic Report- ing (SAFETEA-LU, Section 1201) requirements, which will impact all DOTs within the near future. This example was developed for the Alaska Department of Transportation and Public Facilities Program Development Division to support their Data Business Plan Development. In addition to the Risk Register, a Risk Impact matrix (Fig- ure 4.4) also can be defined for the data systems.9 A Risk Impact Matrix is a tool which defines a two-dimensional risk universe. In this example, the risk universe describes potential risks asso- ciated with asset management where the asset is bridges. Risk management programs for asset management also are very applicable to data programs, since data are a type of asset. The two dimensions in the risk universe are the (1) proba- bility of service interruption or, in the case of data systems, the probability of lack of access to the system during a 12-month period for instance from the time the risk is identified, and (2) consequence of service interruption or impact to the Divi- sion or Department due to the interruption of access to needed data over the same 12-month period. This matrix was produced using the Lloyd’s Register’s Arivu platform. In the Arivu matrix both the probability and consequence dimensions specifically use an order of magni- tude difference moving from one cell to the next. Geographic Information Systems The advantages of using GIS tools for data integration can- not be overemphasized. Many DOTs have GIS maps for their on-state system but do not have data available at the DOT for the local road networks. GIS tools can be used to integrate the needed off-state system roads from local sources, which facili- tate the agency’s ability to meet Federal reporting requirements particularly in the safety area. Agencies also may require systems to be designed with Service Oriented Architecture and Open Database Connectivity, which provides for a much more flexi- ble system to integrate with future systems with minimal effort. There also are tools available to assist DOTs in the development of risk management plans. Enterprise Database Platforms Twenty years ago, most companies operated a variety of legacy systems that contained various databases that did not relate to one other. For example, customer service had a database of customers and orders, logistics had a database of shipments, production had a database of lots produced, and engineering had a database of specifications. Today, most com- panies have migrated to an ERP platform, where most of this I-67 Identification Qualitative Analysis Priority Status ID# Date Risk Identified Functional Assignment Threat Risk Details Risk Trigger Type Probability Impact 1 Active TDS- 001H 5/01/2008 TDS-HPMS Need to revise HPMS report to comply with 2010 req uirements. All internal data files and programs will need to be reviewed and revised as needed to provide HPMS 2010 report. FHWA mandate for new HPMS 2010 requirements. Schedule Available Resources Cost H H 2 Pending TDS- 001T 1/01/2009 TDS-Traffic Need to comply with Section 1201 – SAFETEA-LU requirements. Need Real-Time Traffic Mgmt. Plan within 2 yrs. of final rule on Sect. 1201. SAFETEA- LU requirements. Schedule Available Resources Cost H H 3 Pending TDS- 001G 9/01/2008 TDS-GIS Need to meet HPMS 2010 GIS requirements. Will need to provide HPMS 2010 report in GIS format. FHWA mandate for new HPMS 2010 requirements. Schedule Available Resources Cost H H Table 4.1. ADOT&PF risk register example. 9Transportation Risk Management presentation, Steve Pickrell, Cambridge Systematics, Inc., July, 2009.

data are stored in a way that it can be accessed from anywhere and related to other data in the central repository. For example, if a quality problem is detected, companies can identify which orders, production lots, shipments, and customers are affected from one database, and possibly with one query. Today, the same companies typically operate a distributed network around a central (or enterprise) database. ERP sys- tems and the utilities and middleware that are associated with them include LAN workstations, servers, data centers, main- frames, and direct access storage devices. The corresponding advancements in the use of hardware may include a variety of peripheral devices that enable communication and execution in field operations, such as wireless devices, including PDAs, printers and scanners, and communications devices such as microphones and headsets. Based on these preliminary case studies, private sector companies are increasingly using technology to support their resource allocation and their target-setting in the fol- lowing ways: • Software is used to generate optimal solutions in the plan- ning phase. Software applications (usually customized) contain embedded decision rules about capacity, expendi- ture, and priority. For example, customer relationship man- agement (CRM) applications determine the priority of a customer order based on the customer’s importance to the company; the capacity management system determines the shift schedule depending on the workload; and the purchas- ing system regulates expenses based on authorization thresh- olds specific to certain organizational levels and even specific individuals. • Enterprise applications are used in the execution of ship- ments and dispatching of fleet and equipment. • Electronic funds transfer is used in the clearing of funds flows. The supply chain has practically become the information chain, with incorporation of information technology in vari- ous stages of the chain. Table 4.2 illustrates the sequential order of the use of technology in the private sector from the planning through execution stages. Global Positioning System There are many similar opportunities for the use of technol- ogy at a state DOT to reduce the costs associated with collecting and distributing data. The implementation of hardware and software in specific cases may help to improve data quality by reducing duplicate data collection and QA/QC procedures, as is illustrated in the case of WSDOT. WSDOT was able to reduce I-68 100% 10% 1% 0.1% 0.01% Pr ob ab ili ty o f S er vi ce In te rr up tio n MEDIUM HIGH HIGH MEDIUM LOW 1 10 100 1,000 10,000 Consequence of Service Interruption Figure 4.4. Example risk matrix.

the cost of roadway data collection by using GPS technology to enhance back data collection activities on routine maintenance activities. The data also was edited at the point of collection, eliminating extensive postprocessing time and costs. This was an efficient way to gather the data needed to update the GIS basemap, which, in turn, is used for many projects and purposes throughout the agency. Many public and private sector agencies also have invested in the use of various types of BI tools to meet their data col- lection, analysis, reporting and archiving needs. “BI tools are a type of application software designed to report, analyze, and present data. The tools generally read data that have been pre- viously stored, often, though not necessarily, in a data ware- house or data mart.” The effectiveness of the use of BI tools is best illustrated in the description of ARDEC’s use of business intelligence tools: “These systems provide the power we need to capture, man- age, and make available workforce knowledge, best practices, processes and procedures, templates and other information to the entire workforce.” I-69 Area Technology 1. Supply Chain Planning Inspection Network design and site location Network optimization Specification of order types, terms, and conditions Trading partner collaboration 2. Transportation Planning Mode selection Routing and scheduling Transportation Management Systems (TMS) implementation 3. Procurement Auctions Contract negotiation e - Auctions Purchasing Cards Request for Quotation (RFx) Supplier management Supplier selection 4. Manufacturing Advanced Planning and Scheduling (APS) Manufacturing Execution Materials management Production scheduling Quality management 5. Order Fulfillment Customer Relationship Management Demand forecasting Enterprise Resource Planning Systems (ERP) implementation 6. Inbound and Outbound Transportation Dispatching Radio Frequency Identification (RFID) Track and Trace, e.g., Global Positioning Systems (GPS) Traffic management 7. Warehousing Inventory management Packaging Packing Picking Receiving, Put - Away, Loading Warehouse Management Systems (WMS) implementation 8. Maintenance Equipment repair and maintenance Facilities maintenance 9. Funds Flows Auditing Billing Claims processing Collections Source: Boston Strategies International, Inc. Table 4.2. Common applications of information technology to private sector transportation performance management and measurement.

Dashboards and Scorecards One of the more commonly used BI tools is COGNOS, which is used as a dashboard for display of information about performance measures related to core programs in an agency. COGNOS is the official management reporting tool desig- nated for use at ADOT&PF, and is widely used in departments throughout Hennepin County. Hennepin County uses a combination of MS SQL server as its database platform and the COGNOS@ Metric Studio tool for managing its Balanced Scorecard Information for each department in the county. Each department has its own score- card for performance measures related to their programs, and managers in each department can easily monitor these on a routine basis. Issues regarding performance can be addressed as appropriate with individual departments and measures can be adjusted when necessary to ensure that all programs are functioning in the best interest of the public. Future Technology In addition to current tools that are available, there are evolv- ing trends in the use of IT technology tools that will become more widespread in the future, such as the use of Extensible Markup Language (XML). XML offers different functionality than Hyper Text Markup Language (HTML). XML basically is used for storing and transmitting data, while HTML is used for displaying data. There are many benefits offered by using XML, but one of the main benefits is that it does not depend on spe- cific types of hardware or software for its use. With the global ability to use mobile phone service for accessing data and infor- mation, this functionality now offers another opportunity for organizations to consider another method for timely transmis- sion of data. In the case of a DOT, use of this technology is very beneficial for real-time traffic management systems. Each of the examples from WSDOT, OOCEA, Hennepin County, RCG Information Technology, ARDEC, ADOT&PF, Mn/DOT, and future technology trends illustrate how criti- cal the use of technology is in providing the foundation for standardized, quality data systems, which are sustainable over time, to meet the core business needs of the organization. The use of technology and formal institutional arrange- ments to support data management provide a strong platform for meeting the business needs of the organization in a timely manner. 4.7 Relationships to Target-Setting and Resource Allocation This research indicates that the approaches used for target- setting and resource allocation are somewhat different in the public and private sector. In the private sector, once a target is set, companies do not universally agree on whether or how to link achievement of the target with allocation of resources. The following four approaches are used in practice: • Allocate resources based on historical levels; • Allocate resources based on achievement of the target; • Allocate resources on the basis of relative financial perfor- mance, such as return on investment for capital expendi- tures, and cost/benefit for operating expenditures; and • Use a hybrid approach. The easiest approach is to allocate resources based on his- torical levels, often including an increment or a decrement for inflation and economic growth. However, this is generally viewed as the least effective way to allocate resources since it can result in years of inertia and resistance to change and can perpetuate wrong resource allocations year after year. Many public organizations and departments allocate resources this way, but private sector companies largely switched from this approach to a zero-based budgeting approach. A second approach to allocating resources is to base the resource allocation on achievement of the target. Those that hit or exceed the targets get more resources. The logic is that the target is a point on a continuum, and the more you exceed the target, the more value you add. Most private sector companies’ personnel bonus compensation plans work this way. Exceed- ing the target is usually viewed as a performance worthy of reward. Those that hit or exceed the targets get fewer resources. The logic is that going beyond the target yields little incremen- tal value and that once a target has been attained resources should be diverted to the next highest-yielding endeavor. A third approach is to allocate resources based on relative financial performance, such as return on investment for capital expenditures, and cost/benefit for operating expenditures. This method raises technical issues such as how to ensure consistent cost/benefit calculations and when to consider expenditures as capital investments versus operating expense—for example, should repaving a roadway be considered a capital expense or an operating expense? A hybrid approach is often the most practical one to take. For example, a hybrid approach might use all four approaches under different circumstances as follows: • Allocate capital expenditures on the basis of ROI; • Allocate operating expenditures on the basis of cost/benefit ratio; • Allocate initial annual resources based, but offer bonuses based on achievement of a target at the end of a year; and • Establish control levels so achievement beyond a certain level is not rewarded, in order to protect against lower optimiza- tion of resources, especially in capital-intensive operations. I-70

The public sector, likewise, may allocate resources based on historical trends, or on the ability of a particular program to meet agency goals consistently. Programs in the public sec- tor that demonstrate consistent success, however, are not likely to receive monetary bonuses for employees, per se, but they are able to receive continued funding from those who make the decisions on how funds are allocated. Many of the companies and agencies interviewed also recog- nized the importance of integrating targets and measures into the job functions listed in each employee’s performance plan, in order to motivate the employees to work harder to achieve or exceed targets. The employees gain a better understanding of the significance and relevance of meeting or exceeding targets, when the targets become part of that employee’s job duties. The KSDE has a Data Quality Certification Program that offers professional development training to its employees. The program recognizes the contribution of individuals who enter statistical data into databases for the school districts by present- ing framed certificates to them upon completion of the pro- gram and recognizing them on the agency website. The motto for the program, “the data pays you and grades you,” is a great motivator for employees to strive to provide the best quality data to the school districts. Another example from the public sector of how target- setting affects resource allocation is from WSDOT. WSDOT was able to gain legislative funding support for continuing the Fish Barrier Removal Program by demonstrating to them the increase in migratory fish population to be gained by removal of the barriers. This fish population is important to the gam- ing and fishing industry in Washington. When the measures and targets were presented to the legislature, continued fund- ing of the program was approved above others that also were competing for similar funding. ARDEC collects performance data on finance-related indi- cators, labor hours, revenues, purchases, human resource information, and others. Its goal is to use this data to eventually manage the status and availability of staff at the work break- down structure level. Even though ARDEC’s priorities are established by others, the organization uses its available data to manage resources. The OOCEA is challenged with managing travel on the expressways in the Orlando/Orange County area. Based on the data collected on travel times and reported in the monthly Expressway Travel Time Performance report, the toll author- ity is able to determine when and where to add staff at toll stations during peak travel times. Travel time performance indicators at OOCEA directly influence resource allocation. One of the strongest examples of performance-based manage- ment comes from Hennepin County. There is strong support for performance management at the county administrative level and the strategic plan for the county includes goals estab- lished for each functional area in the county. The managers in each area are responsible for monitoring the goals, targets, and success rates for their departments and use the balanced score- card method to monitor progress. Depending upon the pro- gram, the indicators may show that additional resources or funding is needed in a particular area and the need is justified, based upon the performance indicators. The City of Coral Springs, Florida, also has a very well- defined set of key intended outcomes (KIO) which are perform- ance measures covering seven strategic priority areas. There also are specific sets of data collected to support the seven areas. Through monitoring the data reflected in the performance measures and targets, the city can make improvements in resources as needed, such as adding more fire trucks or person- nel to respond to calls within a certain amount of time. The City holds itself directly accountable to the citizens of Coral Springs and the performance measures data plays an important role in ensuring that the needs of the citizens are met by the City staff. There also are areas where performance measures data may be lacking and this has a negative impact on the agency’s abil- ity to allocate resources for that program. Without the data to indicate how a program is performing, it is difficult to get the necessary approval for funding and resources. Hennepin County has indicated that while specific depart- ment areas may have excellent performance measures data that can be used to allocate resources, there is a need to have more data available on a countywide basis. Likewise, the MTC has recognized the need to improve the connection between the use of performance measures and those who collect data for the various programs and make funding decisions on data programs. They want to have access to better data for mon- itoring carbon dioxide emissions to comply with the Clean Air Act. They also need better crash data in order to estab- lish performance measures to improve safety and reduce collisions in the Bay Area. Whether all needed data are available to support perfor- mance measures and resource allocation most organizations recognize the advantages of investing in data systems which provide good quality data to assess how their core business programs are performing. In the private sector, having high- quality data may not be a priority compared with other busi- ness goals. Private companies tend to place more emphasis on root cause analysis of problems in a particular area than on data accuracy. This does not suggest that bad quality data are acceptable, but that the incremental cost of getting good qual- ity data may not be worth the additional interpretive value. Many public sector agencies however have been successful in providing timely information based on quality data to the leg- islature or other state and local agencies, as well as the tax pay- ers who have a say in whether funding for a particular program will continue or not. Each of the examples provided in this report illustrate how the methods for defining targets and performance measures I-71

is strengthened through the use of well-organized and well- managed data programs. These programs are, likewise, stream- lined and strengthened through the use of Business Intelligence tools which support the collection, processing, and use of the most reliable and accurate data and information, on behalf of the organization. These data management functions operate most effectively within some type of data governance framework and with the appropriate BI tools. The continued availability of quality data is essential for allocating resources effectively, and for improv- ing the speed at which improvements in business areas can be made. 4.8 Summary of Success Factors and Obstacles Success Factors Several success factors related to effective data management systems and institutional relationships to support PBRA were common to many of the case studies: use of data business plans, identification of a data champion, good coordination between IT and business functions, data governance applied from the top down and clearly defined, achievable goals. Descriptions of the success factors revealed from the case studies are provided in the following sections. Establishing the Need for Data Management/Governance • Demonstrate the ROI to the organization regarding the use of data management and data governance in order to gain buy-in from executives and decision-makers. Demon- strate with specific examples how the use of data gover- nance can meet the goals and targets most important to executives. • Formalize a Business Plan for the agency or department which identifies how each employee’s job is linked to the agency’s mission and goals, thereby, clarifying the impor- tance of their role in the overall success of the department/ office. Assessing Current State of Data Management in Agency • Perform a health assessment of data systems to determine where the most critical deficiencies exist and to develop a strategy for addressing those deficiencies. • Perform a risk assessment of existing data programs to highlight the importance of mission critical programs to management and, thereby, gain continued support for those programs. Planning for Data Management • Start with a smaller achievable goal, when implementing data governance within an organization, and build on small successes to address larger agency goals. • Use a Data Business Plan to strategically manage data pro- grams similar to other strategically managed programs within the organization. • Manage expectations of how data governance can help an organization by explaining the benefits of such models for supporting business operations. • Use Business Models to help executives and managers bet- ter understand the relationship between target-setting and decision-making. • Identify champions from Business and IT sides of an organi- zation to support key systems. Partnerships between both areas are critical to successfully managing data programs. Executing a Data Management Plan • Implement a Data Governance Board or Council to address issues related to development, implementation, and use of data programs which are critical to supporting business functions. • Clearly identify the roles/responsibilities of the staff respon- sible for supporting critical data systems using a Data Gov- ernance Manual or other means. • Collect the right data and the right amount of data, or to put it another way, don’t be data rich but information poor. Pre- sent information to decision-makers in a clear and under- standable way. • Use BI tools (web-based applications, GIS, dashboards, etc.) to allow easy access to data systems and sharing of informa- tion among employees and decision-makers. • Communicate with stakeholders to sustain support for var- ious programs. Continue to provide outreach to all commu- nities of interest to ensure that all needs are addressed. • Develop a business terminology dictionary to align the use of business terms commonly used throughout an organization. This is particularly helpful to staff such as IT professionals who are often responsible for developing applications to meet business needs. • Design data systems so that they can be modified in the future without requiring a complete redevelopment of the software. • Create an annual data file for each data program so that consistent information is provided throughout the year. • Communicate the need for and share metadata with man- agers and policy-makers throughout the organization. Metadata is critical when needing to compare data side-by- side from different data systems. • Establish, update, and enforce polices and procedures to govern data management. I-72

Maintaining Data Management • Use Service-Oriented Architecture (SOA) and Open Data- base Connectivity (ODBC) in the design of new applica- tion systems to enable sharing of data and information across systems. • Manage data as an asset in the organization, through policies governing the collection, maintenance, and use of data. • Use data standards to: – Facilitate establishing targets and measures which meet agency goals. – Reduce the cost of multiple data collection efforts and maintenance of duplicate databases. Strive to collect data once, use it many times. – Facilitate consistent reporting of information. • Use data sharing agreements to reduce costs associated with data collection and maintenance of data systems. • Invest in the staff by providing training in new technology and tools to gain their buy-in for support of data programs and data governance initiatives. Provide professional devel- opment opportunities for staff. • Invest in new technology training for staff. The ease of access to data and the computer savvy ability of staff to utilize tech- nology in creating their own queries and reports has enabled them to more readily attain their goals. The relevance of this particular benefit should not be overlooked at the state DOT level. This investment will ultimately produce beneficial results for the agency. Linking to Planning, Performance Measures, and Target Processes • Use a hybrid approach that employs modeling and bench- marking to establish agency targets and performance measures. • Don’t use a one-size-fits-all approach in establishing per- formance measures and targets. Use the correct metrics for making decisions. Focus on continuous improvement by revising/adding new metrics as needed. • Link the performance measures and targets for a program to budget allocations, improving participation by staff in sup- porting the performance measures and targets. The perfor- mance measure and target-setting process also can be used to motivate employees by linking their performance plans to objectives identified in specific performance measures and targets. • Allow DOT transportation planning staff routine access to other planning offices (regional, district, etc.) and technical resources available in the agency. This strongly enhances a performance-based management process. • Reward business areas which consistently meet targets and goals. Consistent achievement in meeting targets is a power- ful motivator for behavior: success breeds success. • Use external data sources, such as environmental, historic, and other planning agencies for GIS data layers to improve the data used for the performance measurement process when funds are limited to collect this data using internal resources. • Utilize software that is procured or developed internally to automate as much of the performance measurement process as possible. This will allow for more time devoted to the analysis of the performance results. • Revise or stop using targets if performance data are not easily obtainable when a performance target is used. • Programs which do not have a direct link between that pro- gram or project and performance should not be funded. • Identify business units responsible for maintaining current metadata about each performance measure. This facilitates the analysis required for user requested data and informa- tion system changes and enhancements. • Include objectives pertaining to resource allocation in the agency Business Plan. The current Business Plan at the MDTA, for example, has three separate objectives related to resource allocation. These include System Preservation, Implementing and Asset Management System, and Inte- grating MDTA’s financial system with other systems. • Use external data sharing agreements to obtain data for performance measures that the agency does not have. For example, MDTA collaborates with other agencies for sev- eral measures that it needs additional data for, or does not have the necessary equipment to monitor itself. • Establish performance targets through a streamlined process and revisit and revise (as needed) periodically. • Incorporate customer satisfaction as a measure in setting performance targets. • Utilize incentives to facilitate meeting performance objec- tives, including awarding bonuses based upon job per- formance and using quantitative objectives embedded in professional employees’ annual objectives. • Arrange performance measures in a hierarchical order, allowing an agency to translate strategic goals/objectives into operational goals/objectives for each department. The U.S. DOT follows this approach among its various admin- istrations (e.g., FHWA and FTA), allowing it to provide a performance budget that can be related to actual and planned accomplishments for each department. This same scenario would apply to a state DOT, with several divisions, districts, and/or independent offices. The performance in each area then becomes a key basis of resource allocation and budgeting. Obstacles/Challenges Several common obstacles also were identified through many of the case studies. These included the lack of use of BI I-73

tools, and the fact that data programs are not linked. Specific items are described in the following sections. Establishing the Need for Data Management/Governance • From a corporate perspective, the investment in IT systems has to be justified by a benefit, and the benefit of better data quality is often not worth the incremental investment. State DOTs similarly, are faced with the challenge of justi- fying the benefit to the agency in investing in better data quality standards, processes, and policies. Executing a Data Management Plan • Lack of data standards causes adverse impacts on data inte- gration efforts. Maintaining Data Management • Data quality within the organization may not be good enough to support decisions. Need to improve data quality. • Non-integrated data systems limit the sharing of informa- tion and sound decision-making, based on data programs. • Data providers need to learn how to “market” their product. Linking to Planning, Performance Measures and Target Processes • There is often difficulty in identifying what performance measures are needed and how to establish metrics for those measures. • External influences and/or political pressures often influ- ence funding for various programs. While the use of per- formance measures and targets can demonstrate the need for sustaining various data programs, organizations should understand that ultimately, external mandates may repri- oritize organizational goals and targets, and subsequently data programs. • There is a need to address the gap between data supported decisions and data-driven decisions. • Specific performance measures may not be pursued because a baseline cannot be developed using existing data (i.e., per- cent reduction in error in all transactions, etc.). • Not having data centralized is the biggest challenge to target- setting and performance management. Data currently are collected in a variety of formats in a number of different legacy systems. These legacy systems require duplicate entry of data into multiple systems. • Improvements in data quality and quantity would allow for improved performance-based decision-making and management. • The biggest challenge may be one of organizational cul- ture change. Managers and directors need to gain more experience in performance-based decision-making and management. • In the case of the Maryland SHA, the largest data sharing challenge is coordinating with outside agencies that provide performance data for key or legislatively required measures. This is a particular challenge in the area of Safety, where data such as incidents, fatalities, and injuries must be collected from police reports. This information frequently must be derived from hand-written and/or paper records, and is very time consuming. • In a private sector company example, there was a group within the organization that was responsible for data quality, but the group was disbanded because it was per- ceived that they did not add value to the organization. This makes it even more challenging when the company acquires other companies and has to merge data across databases. • Having better, readily available data would make a tremen- dous difference in the ability to measure progress in meeting performance targets. • The biggest gap in data is with trading partners that use different information systems. ABC Logistics relies on solid relationships with its partners to ensure continued attention to improving data quality. State DOTs similarly are faced with the challenge of merging data from separate, silo sys- tems, perhaps from different offices, or divisions within the DOT. Developing a partnership with those offices facilitates the development of processes to integrate the needed data and information into one system. 4.9 Future Research Transportation agencies continuously find themselves faced with challenges from many constantly changing variables out- side their control: inflation, political priorities, and revenues, for example. It is within this environment that transportation agencies must establish a data based planning, programming, and budgeting decision framework. Agencies need to develop short term project budgets and long term financial plans for TIPS, STIPS and LRTPs. Available resources, including money and data, affect the establishment and attainment of perfor- mance targets. Risk Assessment As a result of the uncertainties facing transportation agen- cies, there is an opportunity to apply risk assessment and man- agement to PBRA. However, few examples of comprehensive risk management exist beyond limited application in system preservation and asset management areas such as pavement I-74

and bridge maintenance. For example, NCHRP Report 632: An Asset-Management Framework to the Interstate Highway System recently provided guidance on risk assessment in the context of managing the Interstate highway system; the 8th National Conference on Transportation Asset Manage- ment also provides examples.10,11 There is a need for further research on the role of risk management within the overall performance management framework. Development of an annotated list of specific methods in use, their strengths and weaknesses, and potential applications to PBRA would be a useful next step. Information Technology Tools Data management systems within transportation agencies must incorporate data sharing ability and techniques to effec- tively support target-setting and PBRA. This report and subse- quent guidance includes data sharing topics such as technology for sharing and integrating data. It also discusses how evolving trends such as data exchange formats are affecting data sharing. However, there is a need to expand on this research/guidance to cover issues such as the role information technology plays in supporting data management particularly related to target- setting and PBRA. Specific topics include expanding the knowl- edge base on the use of business intelligence tools which are applicable in the transportation environment; the role of data security/access in supporting or hindering data sharing; and data privacy issues and its impact on data sharing. A primer related to the impact of these data sharing information technol- ogy topics on the success of data management for PBRA should be prepared. I-75 10Cambridge Systematics et al., NCHRP Report 632: An Asset-Management Frame- work for the Interstate Highway System. Transportation Research Board of the National Academies. Washington, DC. 2009. 11Eighth National Conference on Transportation Asset Management (http:// pressamp.trb.org/conferences/programs/program.asp?event=486).

Next: Volume 2 - Guide for Target-Setting and Data Management »
Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management Get This Book
×
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

TRB’s National Cooperative Highway Research Program (NCHRP) Report 666: Target Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target-Setting and Data Management provides a framework and specific guidance for setting performance targets and for ensuring that appropriate data are available to support performance-based decision-making.

Volume III to this report was published separately in an electronic-only format as NCHRP Web-Only Document 154. Volume III includes case studies of organizations investigated in the research used to develop NCHRP Report 666.

NCHRP Report 706: Uses of Risk Management and Data Management to Support Target-Setting for Performance-Based Resource Allocation by Transportation Agencies was released in 2011 and supplements NCHRP Report 666. NCHRP Report 706 describes how risk management and data management may be used by transportation agencies to support management target-setting for performance based resource allocation.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!