National Academies Press: OpenBook
« Previous: Volume 1 - Research Report
Page 73
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 73
Page 74
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 74
Page 75
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 75
Page 76
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 76
Page 77
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 77
Page 78
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 78
Page 79
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 79
Page 80
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 80
Page 81
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 81
Page 82
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 82
Page 83
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 83
Page 84
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 84
Page 85
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 85
Page 86
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 86
Page 87
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 87
Page 88
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 88
Page 89
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 89
Page 90
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 90
Page 91
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 91
Page 92
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 92
Page 93
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 93
Page 94
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 94
Page 95
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 95
Page 96
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 96
Page 97
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 97
Page 98
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 98
Page 99
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 99
Page 100
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 100
Page 101
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 101
Page 102
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 102
Page 103
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 103
Page 104
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 104
Page 105
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 105
Page 106
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 106
Page 107
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 107
Page 108
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 108
Page 109
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 109
Page 110
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 110
Page 111
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 111
Page 112
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 112
Page 113
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 113
Page 114
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 114
Page 115
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 115
Page 116
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 116
Page 117
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 117
Page 118
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 118
Page 119
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 119
Page 120
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 120
Page 121
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 121
Page 122
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 122
Page 123
Suggested Citation:"Volume 2 - Guide for Target-Setting and Data Management." National Academies of Sciences, Engineering, and Medicine. 2010. Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management. Washington, DC: The National Academies Press. doi: 10.17226/14429.
×
Page 123

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Guide for Target-Setting and Data Management V O L U M E I I

C O N T E N T S V O L U M E I I Guide for Target-Setting and Data Management II-5 Chapter 1 Guide for Target-Setting II-5 Step 1—Establish Performance Management Framework II-9 Step 2—Evaluate the Factors Influencing Target-Setting II-16 Step 3—Select the Appropriate Approaches for Target-Setting II-21 Step 4—Establish Methods for Achieving Targets II-24 Step 5—Track Progress Towards Targets II-26 Step 6—Adjust Targets Over Time II-28 Chapter 2 Guide for Data Management II-29 2.1 Establishing the Need for Data Management/Governance II-36 2.2 Establishing Goals for Data Management II-37 2.3 Assessing Current State of Data Programs II-44 2.4 Establish Data Governance Programs II-47 2.5 Technology for Data Management II-52 2.6 Linking Data to Planning, Performance Measures, and Target-Setting Processes

This Guide for Target-Setting outlines a structure that can be used by transportation agencies for developing and evaluating targets. This section describes the actual structure and how an agency might implement it, including examples gleaned through actual agency implementation (these examples reference Case Studies provided in Volume III, which has been published as NCHRP Web-Only Document 154). Target setting must be applied strategically and carefully, with an understanding of the context in which targets will be applied. There is no one predefined, prescribed approach for setting and using targets because their intended use can vary greatly; in fact, no agency currently relies on a single, pre- scribed, systematic approach for setting targets. Using the information from the case studies, as sum- marized in Volume I, the seven-step process from NCHRP Report 551 can be reconstituted and refined within the context of the PBRA framework to create a flexible structure to guide agencies in setting targets. The structure is as follows: • Step 1—Establish Performance Management Framework. Establish the Framework that links organizational goals to resources and results. Performance measures and their atten- dant targets are the link connecting goals to specific investments. • Step 2—Evaluate the Factors Influencing Target-Setting. Ask the right questions about the factors internal and external to the agency that affect target-setting and the approaches that can be used. • Step 3—Select the Appropriate Approaches for Target-Setting. Based on the factors in Step 2, select an approach or approaches for setting targets. • Step 4—Establish Methods for Achieving Targets. Within the context of the Performance Management Framework, identify methods that orient the agency and its resources towards achieving the targets set in Step 3. • Step 5—Track Progress Towards Targets. As part of the “Measure and Report Results” element of the Performance Management Framework, track progress specifically against targets. • Step 6—Adjust Targets Over Time. Based on financial and political realities, ease, or difficulty of achieving targets, and increasing experience in PBRA, use the feedback loop in the Performance Management Framework to reevaluate and adjust targets. Table 1.1 provides a comparison between the seven-step process used in NCHRP Report 551 and the approach suggested in this report. The following sections explain each of the suggested six target-setting steps in further detail. Step 1—Establish Performance Management Framework Performance-based resource allocation (PBRA) takes place within an overall Performance Management Framework. The six elements of the Performance Management Framework are described in Figure 1.1; the Framework is described in greater detail in Volume I, Chapter 2. To II-5 C H A P T E R 1 Guide for Target-Setting

II-6 Guide for Target-Setting and Data Management NCHRP Project 8-70 Approach NCHRP Report 551 Seven-Step Process Performance Management Framework Establish PBRA framework. • Define the context for target setting and establish time horizon(s). • Develop long-term goals based on con- sideration of technical and economic factors. Entire framework, within agency context. Evaluate the factors influencing target-setting. • Determine which measures should have targets. • Consider current and future funding availability. • Consider policy and public input implications for target setting. Target setting. Select the appropriate approaches for target-setting. • Establish targets. Target setting. Establish methods for achieving targets. • Analyze resource allocation scenarios and tradeoffs. Target Setting and subcomponent of allocate Resources. Track progress towards targets. • Track progress. Subcomponent of measure and report results. Adjust Targets Over Time. N/A Part of feedback loop. Table 1.1. NCHRP Project 8-70 target-setting structure compared to NCHRP Report 551 seven-step process and performance management framework. Goals/Objectives Performance Measures Target Setting Evaluate Programs and Projects Allocate Resources Budget and Staff Measure and Report Results Actual Performance Achieved Quality Data Figure 1.1. Performance management framework.

summarize, Performance Management is a business process that links organizational goals and objectives to resources and results. Performance measures and their attendant targets are the lynchpin in this process connecting goals to specific investments. The approaches by which measures and targets are set and the methods by which they are used in investment decision- making play critical roles in the overall success of a public agency or private company. Establish Goals and Objectives Performance–based resource allocation decisions are anchored in a set of policy goals and objectives which identify an organization’s desired direction and reflect the environment within which its business is conducted. For example, many state DOTs have well-defined goals for the transportation system, including infrastructure condition, level of service and safety, as well as goals reflecting economic, environmental, and community values. Likewise, the private sector frequently establishes policy goals to guide production of products and services while defining the environmental and community context for its investment decisions. Through this first step, many of the factors that will affect target-setting will begin to become more evident (see Step 2). Guide for Target-Setting II-7 In the private sector, the processes used by most companies operate such that every goal should have a target and every target should align with a goal. Most companies prefer to reduce the number of goals and targets rather than risk misalignment or confusion between goals and targets. MNC and Corporation X, both large companies, are focused on cost reduction. Select Performance Measures Performance measures are a set of metrics used by organizations to monitor progress towards achieving a goal or objective. The criteria for selecting measures often include the following: • Feasibility, • Policy sensitivity, • Ease of understanding, and • Usefulness in actual decision-making. Companies that use Total Quality Management (TQM) rely heavily on simple metrics because changes are implemented at the shop floor level where data must be readily obtained and recorded in order to be credible and immediately actionable. Also, changes in the definition of metrics over time can stifle motivation to achieve the target; companies that use balanced scorecards need stable metrics more than most because unstable metrics make the scorecard appear unreliable. DIY Company has consistently used one key metric—transportation cost as a percent of gross trade sales. Identify Targets Targets are a quantifiable point in time at which an organization achieves all or a portion of its goals. These points set a performance level for each organizational measure, such as achieving

a 25 percent reduction in highway fatalities by 2030. The steps used to set such a target include the following: • Establish Performance Management Framework, • Evaluate the Factors Influencing Target-Setting, • Select the Appropriate Approaches for Target-Setting, • Establish Methods for Achieving Targets, • Track Progress Towards Targets, and • Adjust Targets Over Time. II-8 Guide for Target-Setting and Data Management Realistic targets are essential to making rewards and penalties effective in the private sector. The anticipated gains can be embedded in business unit or depart- mental budgets, and individuals can be more readily held accountable for their performance towards reaching the target. MNC uses an annual cost reduction target that is widely communicated throughout the organization. Allocate Resources The allocation of resources (time and money) is guided by the integration of the preceding steps into an organization’s planning, programming, and project development process. To the extent possible, each investment category is linked to a goal/objective, a set of performance mea- sures, and a target. Specific investment proposals are defined in relation to specific targets. Measure and Report Results The data for each performance measure must be regularly collected and periodically analyzed. The analysis should indicate how close the organization is to achieving its targets and identify the actions necessary to improve results. Many public and private sector organizations have tracking systems in place to monitor performance allowing senior staff to make periodic budget adjustments. Mn/DOT prepares a one-page “snapshot” with performance measures and red, yellow, and green colored shapes to represent annual progress relative to targets, by state and by district. The snapshot graphically illustrates the trend direction and projects the next year’s forecast. Corporation X uses a combination of hard- ware and software to gather data from its widely dispersed field operations. Create Data Management Systems “Good” data is the foundation of performance management. Effective decision-making in each element of the performance management framework requires that data be collected, cleaned, accessed, analyzed, and displayed. The organizational functions that produce these requirements are called data management systems. The first key dimension centers on the technical chal- lenges associated with data systems, including development and maintenance of hardware and software, and the specifications for data collection, analysis, archiving, and reporting. The second area focuses on the institutional issues associated with data stewardship and data governance. These focus areas will be explored and discussed in Chapter 2.

Step 2—Evaluate the Factors Influencing Target-Setting Asking the Right Questions In order to evaluate the factors affecting target-setting for a particular measure in a public agency or private organization, a practitioner can begin by asking a few simple questions. The recommended approach revolves around a process commonly known as the Five Ws and an H (Who, What, When, Where, Why, and How). WHY Is the Target Needed? Guide for Target-Setting II-9 “Why” requires thinking about the following factors: • Internal support; • Political/Legislative influence; • Customer service focus; • Stakeholder expectations; • Commitment to regular communication and reporting; and • Types of resources to be allocated. This involves developing an explicit understanding about why a target is being developed and the particular performance measure with which it is associated. It provides the link between the target and the larger PBRA process (i.e., the specific measure to which it relates and, in turn, the transportation goal/objective that it supports). This requires specifying the measures suitable for a target. Some performance measures may not lend themselves to quan- tification, while others may support long-term transportation goals but be outside the authority of the agency to influence and therefore inappropriate for defining a prescriptive target. Further, the need for a target related to a specific measure may stem from a need within the agency or from real or perceived needs from elected officials, the public, or other stakeholders. WHO Will Be Using the Targets? “Who” requires thinking about the following factors: • Internal support/culture; • Political/Legislative influence; • Customer service focus; • Span of control/agency jurisdiction; and • Types of resources to be allocated. This involves defining who the end-user is for the target; namely, if that user is internal or external to the transportation agency. An internal user, for example, could be a technical staff person using the target to aid in project evaluation, while an external user might be an elected official, Board member, or member of the public who will be using the target to review invest- ment performance. If the end user includes external parties then this factor involves paying close attention to understanding the policy and public implications of setting particular targets (i.e., what is desired versus what is possible given certain resources).

WHERE in the Process Will Targets Be Used? II-10 Guide for Target-Setting and Data Management “Where” requires thinking about the following factors: • Span of control/agency jurisdiction; • PBRA history/evolution of state-of-the-practice; and • Types of resources to be allocated. This involves defining the point in either the plan development or project delivery process where targets will support the decision-making process (e.g., project evaluation and selection, systems-level review, project design, project delivery, or monitoring of on-the-ground perfor- mance). This is closely related to how an agency has implemented the Performance Management Framework, as well as how much experience an agency has applying PBRA. WHEN Should Targets Be Attained? “When” requires thinking about the following factors: • Timeframe; • Types of resources to be allocated; • Stakeholder expectations; and • Commitment to regular communications and reporting. • Political/Legislative influence This specifies the time horizon for when the target should be met. A distinction should be made between long-term goals about desired performance levels and short-term targets that rep- resent the best that can be done given known resources. HOW Will Targets Be Calculated and Achieved? “How” requires thinking about the following factors: • Span of control/agency jurisdiction; • PBRA history/evolution of state-of-the-practice; • Financial resources; • Technical resources/planning and forecasting capability; • Timeframe; • Political legislative influence; • Organizational structure; and • Internal support/culture. This question can be answered based on two levels of technical rigor. The first, which is less rigorous and perhaps most relevant to the long-range planning aspect of PBRA, is to simply define in broad terms the types of strategies intended to support meeting the target, whether it be by particular project types or investment strategies. The second, which is much more technically rigorous, relies on comprehensive technical analysis that defines how much improvement can be achieved given certain resource allocation tradeoffs and consideration of funding constraints

and various implementation scenarios. The second level of detail, if applied, should feed directly into what the target actually is, as it provides a more detailed perspective on what can actually be attained given certain real-world constraints. However, actual approaches for calculation and achievement of targets depend highly on internal resources and both internal and external influences. WHAT Is the Target? This step is the point where a target is actually established, given that the previous questions have been adequately answered. By asking these questions and considering the typical factors affecting target-setting, the practitioner can select the appropriate approach or approaches for setting targets and arrive at targets that suit an agency’s needs. Typical Factors to Consider There are multiple factors that lend themselves to the development of a PBRA process within transportation agencies. These factors are documented in Volume I and in the Case Studies in Volume III. These are the factors that arise when asking the 5 W’s and an H questions in the pre- vious section. Guide for Target-Setting II-11 When reviewing and refining performance targets, it is important to keep in mind that setting targets typically involves balancing a number of factors, which may vary in importance among different measures or products/services. The driving factor is not always the same. Political/Legislative Influence Perhaps the most immediate and direct factor influencing target-setting as an element of PBRA is the existence of a commission or other political body to which a transportation agency must report the performance of investment decisions. Political intervention in the process may result from controversy or the increasing public outcry over transportation services that force political attention on an issue. While political influence of this direct manner can have very complicated repercussions, it has shown to be one of the most positive indicators for implementation of target-setting, if done prop- erly. For almost every agency reviewed that is using targets as part of their PBRA process, politi- cal or legislative intervention provided the initial impetus for establishing discrete targets. Political intervention can be triggered by a number of issues, but the most common is the increasing lim- itation of transportation funding at all levels of government, which has created more competition for available funds and also made it more important to justify funding requests. The Minnesota Legislature and Department of Finance require that agencies use performance measures in biennial budget documents. Washington State’s Legislative Transportation Com- mittee recommended a PBRA process in 1991 that’s still in place today. Political and legislative influence also could result in an “edict from above,” obviating the need for an agency to define its own target for a measure (MTC, ARC); sometimes the edict is broad enough that there is still room for more refined target-setting within an agency or division (FDOT). This can be difficult to negotiate if the process is not properly informed by knowledgeable transportation staff that guide the development of reasonable, attainable targets, considering planning, technical, and funding constraints for an agency (in terms of both ability to achieve and ability to measure progress towards a target). However, if specific targets already have been set externally, it may be necessary to complete the steps of the overall approach outlined in this

guidance—namely, Establish Methods for Achieving Targets, Track Progress Towards Targets, and Adjust Targets Over Time—and through the feedback loop, better inform the target-setting process through the next planning cycle iteration. While political involvement can be challenging, it also can provide transportation staff the support they need to select projects that are proven to improve performance and therefore should be a priority for funding. II-12 Guide for Target-Setting and Data Management Transportation agencies should guide the development of reasonable, attainable targets by legislatures. However, targets set by external edict may need to be refined through iteration of the target-setting process over time. Customer Service Focus Those public agencies and private organizations that have taken a clear customer-service approach to PBRA understand the need to use targets to be able to communicate to the sys- tem user, the “customer,” the return on their investment. Customer satisfaction is a funda- mental aspect of performance for these organizations, and is therefore reflected in the types of measures selected, the measures that are given targets, and how the targets are created. They break down and analyze customer satisfaction or dissatisfaction from broad perspec- tives that address issues in areas such as social and community impacts and environmental impacts of transportation investments and, in more narrow terms, that address issues related to daily personal travel needs. In Coral Springs, Florida, customer input drives the decision-making process; target- setting serves as a metric for progress, providing feedback for the community. For ABC Logis- tics, targets stem from the promises made to its customers in their contracts with the logistics company. A Customer Care program monitors the company’s performance relative to the targets and customer expectations, with feedback mechanisms to communicate results to the public and to link individual employee performance (and merit) to performance of the company. PBRA History/Evolution in State of the Practice Agencies that are only at the beginning of implementing a performance-based process gener- ally have less complete and less sophisticated target-setting processes. In general, there is a typical evolutionary path that agencies follow. A corollary to this evolution is the emergence of an agency’s data sophistication. Mn/DOT’s measures and targets, identified in its 2003 Statewide Long-Range Plan, were refined through application and adjusted in the 2009 plan. MTC’s performance-based process also has been evolutionary during the development of its last three long-range plans. WSDOT’s experience also confirms that target-setting requires a solid history of performance data as well as managerial comprehension and appreciation of that data, which comes with time and expe- rience. Managers must have the ability to understand transportation system behavior (i.e., “what the data are saying”) and to discern what they can or cannot control. Even imperfect measures and targets, if they are well-established, are well understood and have previous reference points (DIY Company). Commitment to Regular Communication and Reporting Regular tracking of investment performance and reporting of results to the public and trans- portation stakeholders serves to focus attention on an issue over time, so that it is not lost in political and public discourse as new challenges arise. Regular reporting and communication of

progress helps to keep staff and the public focused on the particular challenge, especially as is in the private sector when it is tied to agency or even staff-level “merit”/compensation. This helps all involved to understand the nature of solving process problems over the long term, rather than focusing on immediate issues (e.g., fighting fires) that often distract from the larger mission of an agency. It also supports longer-term trend development that is needed to track the perfor- mance of investments over time. This commitment to transparency will affect the approach an agency selects to define targets as well as the timeframe of the targets. Reporting can be both internal and external for public sector organizations, and usually just internal for private sector organizations (with the exception of financial results for sharehold- ers). Many agencies use regular reporting to drive their own internal resource allocation (e.g., finances), as well as to satisfy external requirements (MLIT, OOCEA, WSDOT, MDOT). Report- ing mechanisms also interface strongly with managing stakeholder expectations and help to make an agency more customer-service focused. The regular development of reports or other “products” that are distributed to the public and used by decision-makers helps to maintain staff enthusiasm for the performance-based process and ensures the continued development of the necessary inputs for the report (Hennepin County). Span of Control/Agency Jurisdiction The span of agency control, whether it is through funding, modal authority, or geographic jurisdiction, plays a strong role in the development of measures and targets, because it controls the perspective from which each investment is evaluated. An agency that manages only highways will have a narrower set of measures than does an agency with jurisdiction for multiple modes. States responsible for all roads, rather than only the higher functional classes, face greater data- gathering complexities. This can influence how they set targets and how they use data to mea- sure progress towards those targets. For instance, Mn/DOT has direct control over the quality of pavement, but it can only influence transit service provided in Greater Minneapolis through funding. In all instances, the level of influence that the department has over a particular measure affects the target that is eventually set. Within DOTs, standard siloing of functions has led to strong asset management systems for roadway maintenance functions, but this process is trans- lating to other DOT functions. Financial Resources No constraint or factor in constraining the PBRA process and affecting target-setting is cited as much as financial resources. Financial resources are intimately intertwined with the resource allocation process, both determining an agency’s ability to implement such a process by influ- encing other factors such as technical resources, and also potentially being determined by the process itself. Financial resources also can be used to create “financially constrained” targets that reflect historical or projected funding (MLIT). Performance data has played a key role in biennial state legislative budget allocations for Mn/DOT, and it also has played an important role in the debate for new transportation fund- ing. Mn/DOT quantified its highway performance measures and targets in its 2003 State Trans- portation Plan and concluded that Minnesota was underinvesting in its highway program by one billion dollars per year. This performance-based analysis was accepted by the legislature and virtually ended the legislative debate on level of need. The legislative discussion shifted from the question of need to the question of payment. In February 2008, the Minnesota Legislature over- rode the governor’s veto and passed a funding bill which provided several billion dollars of new funding for transportation over 10 years. Similar to the influence of financial resources on target-setting, staffing and person-hours of available time can affect the depth of a PBRA program, including targets that an agency can assemble and monitor. Guide for Target-Setting II-13

Timeframe The timeframe of desired results affects how targets will be set and what they will be. Time- frame is sometimes determined by stakeholder and internal agency needs; it also can be dictated by forecasting capabilities. II-14 Guide for Target-Setting and Data Management Although private sector senior managers and the finance departments usually have long-range goals, most corporate targets are set and performance moni- tored on an annual basis. To link individual performance to longer-term goals, companies sometimes use equity to incent employees rather than to share multi- ple annual targets with them. Many agencies have a variety of different timeframes for different planning and programming purposes, with targets for each of the timeframes. At Japan’s MLIT, annual targets are derived in part from the latest major subjects of policy, planning, and programming to emerge from the funding reports from the MLIT and Road Bureau, the Road Bureau’s Mid-Term Visioning Report, and the national government’s 5-year Major Infrastructure Development Plan. Longer- term targets match this with a 5-year span. This information is used when determining feasible 5-year goals for the Bureau and results in what is essentially a financially constrained target. Maryland uses a similar approach with annual targets, 4-year targets, and 5- to 10-year targets, depending on the particular planning document. In Washington, the primary responsibility for translating long-term goals (dictated by elected officials) to short-term or “incremental” goals, objectives, and targets falls to the Department of Transportation, in consultation with executive and legislative members and staffs. This process centers on how to set and describe these incremental milestones, how to communicate them to the public, and what legal liability the State may incur by promoting these short-term targets publicly. WSDOT managers also may consider alternatives and adjustments in the engineering solutions to problems, in the methods of service delivery, and in the construction materials and techniques to be used in order to address these short-term targets. These options help to achieve stated targets within current funding and other resource constraints and thus maintain consis- tency between short-term program accomplishments and long-term aspirational goals. Technical Resources The presence or lack of forecasting tools can influence greatly the sophistication of forecasted targets. Agencies that have used HERS, PONTIS, and other tools for forecasting the results of long-term programs have greater insight with which to set long-term targets. Availability of analysis tools to identify performance impacts of projects realistically and effi- ciently and to track performance in relation to targets will determine what measures and targets can be used. Sometimes agencies develop desired measures and targets, even when data are not yet available, as a means of creating a “wish list” of data sources. Often it is difficult for decision- makers to see the need for data collection for a single performance metric, particularly if it appears to be part of a single endeavor (e.g., a long-range plan); if it is part of a larger, compre- hensive PBRA process, however, it is often easier to justify additional data needs. Evolution of tools over time makes it very difficult to track progress consistently. Change over time in tools, data, and analysis procedures, as well as differences between agencies and jurisdic- tions, can make it difficult for stakeholders—and even internal staff—to properly interpret the results. Staff turnover also can exacerbate this situation. Agencies must develop ways of main- taining their institutional knowledge base to properly utilize evolving tools and procedures.

Development of strong tools and data give agencies the ability to not only calculate targets in a rational way, but the ability to measure progress towards those targets or the resources neces- sary to reach them. These abilities lend credibility to agencies with their stakeholders and the public (MTC). However, tools are often not equipped for a rigorous performance analysis at the project level for long-range planning or comparison between modes. Organizational Structure The organizational structure of an agency affects the structure of that agency’s PBRA program and process as well as the development and purpose of targets. Centralized organization can allow the central office to work directly with the state government and stakeholders to establish targets. It also allows the organization to handle a somewhat complex and involved target- setting process (WSDOT). Guide for Target-Setting II-15 Those with strong forecasting capabilities are more apt to have long-term goals and targets, whereas companies with weaker forecasting capabilities are more inclined to set annual targets only. For example, Corporation X, which has a robust financial modeling system, sets targets for several forecast periods. Private companies with multiple business units often hold up the highest-performing business unit as the benchmark that the others should strive to beat. The element of competition is often more important than the specific targets, so some companies let their divisions have a broad role in determining which metrics to use. Conversely, highly decentralized structures result in possibly different targets (and different resource allocation priorities and decisions) for different measures in different districts, but with broader and more flexible measures and targets at the central office (FDOT). This structure requires a strong but flexible performance management system to ensure consistency across districts in terms of achieving statewide goals. FDOT’s overall Business Plan seeks to maintain accountability and transparency for processes that may not be standardized across the department. Stakeholder Expectations Similar to political and legislative influence, stakeholder influence can have a very significant impact on target setting. When external stakeholders become engaged in the process, they can influence the development of goals, measures, and targets. Typically, the use of specific targets derived from an internal process is seen in agencies with a more sophisticated and well-developed PBRA system that has developed over several iterations. Staff resources and, time permitting, an internally developed process can lead to very meaningful and effective targets within an agency’s PBRA process. Agencies often establish targets through a committee process that provides for stakeholder input. As such, it provides an opportunity for dialogue about the transportation issues, constraints in funding, and other topics, which leads to the development of realistic and meaningful targets. This is absolutely critical for state DOTs and MPOs who make decisions in a very litigious environment. It is critical to communicate not only the lofty, long-term goals for transportation systems but the reality of fiscal and political and regulatory constraints as well; this will allow for stakeholder expectations to be managed from the beginning. Keeping flexibility in the target- setting process also is critical in properly setting stakeholder expectations. Further, transportation

is a long-term business, with performance typically improved over the long term, but agen- cies operate in the context of short-term politics, which can have a very strong impact on performance-based management. Internal Support/Culture A common theme among many agencies with more developed PBRA processes is internal sup- port and an inside “champion” at a high level, such as a top-level executive in the agency (MTC, Hennepin County, Mn/DOT). The champions guide the development and implementation of Performance Management, including provisions which help assure that the new way of doing business transcends administrations and individual staff. As a result, these managers understand the importance of PBRA and are willing to ensure the process has the necessary resources to pro- ceed, such as funding, staff, data, and tools. II-16 Guide for Target-Setting and Data Management In companies with a competitive spirit, the attitude about the numbers is more important than the numbers themselves. In companies with an analytical culture, data collection and analysis is revered and viewed as synergistic with continuous improvement. Corporation X has both budget targets and continuous improvement initiatives, and meeting the operational targets in the continuous improvement programs ensures hitting the budget targets. Leading agencies facilitate a sense of ownership and responsibility for performance measure- ment and integrate the practice into an overall agency culture of performance (Maryland DOT SHA). One way this can be achieved is by designating a performance measure “lead” who is respon- sible for maintaining and reporting data for a particular measure and ensuring data accuracy. Step 3—Select the Appropriate Approaches for Target-Setting There is a wide range of agency implementation of PBRA processes, and an equally as diverse range of implementation approaches for target-setting as an element of PBRA. The target-setting approach is determined largely by the factors influencing target-setting and evaluated in Task 2; the appropriate approaches can be selected with consideration of these factors as shown at the end of this section (Figure 1.2). In practice most agencies will use a hybrid approach (different approaches for different mea- sures but also multiple approaches for a single measure). For example, an agency could use mod- eling combined with customer feedback to arrive at a target that is both analytically grounded (to ensure a connection with predicted outcomes based on resources and existing plans) and sat- isfactory to the public and stakeholders. This often helps to mitigate risks inherent in any single approach. Edict In the private sector, this approach also is called “Ready-Fire-Aim:” just state the goal and have everybody try to hit it (MNC). The underlying principle of this approach is that success in hitting the target is entirely a function of motivation and execution and that planning is a relatively minor part of reaching the target. The advantage of the Edict approach is that the target is unequivocal and well-understood throughout the organization. The challenge is that the approach is not

inclusive or consultative; it is more characteristic of old-fashioned hierarchical leadership. For example, the choice of metrics and targets is made by senior management and is not sub- ject to discussion. Staff members are tasked with developing a transportation investment plan to meet the target and conducting modeling and technical analysis needed to demonstrate attainment of the target under a future funding scenario. While such an approach is some- times used by elected officials or other decision-makers to prescribe a target for an agency, this approach by itself is usually unsuitable for most public sector organizations where trans- parency is expected. Expert Opinion In many cases, transportation agencies develop targets through an internal or external consensus-based planning process as part of a more comprehensive PBRA exercise. Typically, the use of specific targets derived from such a process is seen in agencies with a more sophisticated and well-developed PBRA system that has developed over several iterations. Staff resources and time permitting, such an approach can lead to very meaningful and effective targets within an agency’s PBRA process. Such a process is usually informed by internal staff analysis, but ultimately approved by an agency’s executive management and stakeholder committees. Guide for Target-Setting II-17 Figure 1.2. Selecting a target-setting approach based on influencing factors.

This approach leverages the technical, practical, and local knowledge of members of the agency and stakeholders within the agency’s jurisdiction. Not only does this help to ensure that targets will reflect local and stakeholder priorities but also that they will better reflect “on the ground” reality. Customer Feedback Under this approach, direct feedback on system performance and objectives for transporta- tion investment are gathered from the transportation system user through a variety of survey and outreach methods. This feedback is then used by the transportation agency staff to develop specific measures and targets that are closely aligned with the needs of the traveling public (the “customer”). Those agencies that have taken a clear customer-service approach within the resource allocation decision-making process understand the need to use targets that communicate to the system user (the “customer”) the return on their investment. Customer satisfaction is a fundamental aspect of performance for these organizations and permeates the process for how potential investments are evaluated and selected to receive funding. Agencies can use dozens of different types of outreach tools and then analyze the customer input for trends and priorities (Coral Springs, Florida). Tools include annual surveys, public hearings, blogs, regular visioning exercises and focus group discussions, a complaint tracking system, and employee surveys. Selected processes should be extensive, formal, iterative, and continuous. Hennepin County, Minnesota, utilizes a balanced scorecard (BSC) with the “customer” as one of the four perspectives that the approach is viewed from as part of the PBRA process. In the trans- portation service area, a number of specific targets already exist and include targets related to bridge and pavement sufficiency ratings, reducing crash rates, completion of the Bicycle System Plan, and project delivery standards. The target-setting process is still informed by and supported by transportation practitioners that provide the appropriate context for establishing targets (e.g., what percentage of crashes might be expected given certain funding levels, or how much speed may decrease as a result of traffic calming measures), but ultimately processes oriented towards the customer are driven by the customer perception of what needs to be improved and by how much; the way in which measures are reported reflects this. Essentially, this process is almost always part of a hybrid approach supported by Expert Opinion or Modeling. Benchmarking Benchmarking as a target-setting approach provides a transportation agency with the means to establish targets in a relatively quick and efficient manner that can be realistically achieved. Under this approach, criteria should be set for peer group selection and analysis, such as sim- ilar investment approaches, jurisdiction, span of control, and agency size. In terms of stake- holder expectations, it is often appropriate to select peers that also excel in the specific goal II-18 Guide for Target-Setting and Data Management The MTC sets overarching goals and strategies for long-range planning at the executive level, a stakeholder subcommittee of the MTC Planning Committee derives the measures and targets with staff input, and the MTC Planning Committee votes on and approves measures and targets.

areas being benchmarked against. Once the peer group is set, practitioners should review each state’s performance measures and targets and the degree to which those states are achieving their targets. The comparison among states will help guide the final determination of targets within selected performance measure categories. Agencies sometimes benchmark against other agencies in the region (MDOT, SHA). National datasets can often provide good compilations of data for benchmarking. The AASHTO Stand- ing Committee on Quality, recently renamed the Standing Committee on Performance Manage- ment, conducted a study in 2007 which identified best practices for highway project delivery times and cost. The committee also has identified best practices for smooth pavements. The Austroads National Performance Indicators (NPI) system includes dozens of indicators in 11 broad group- ings, covering safety, asset management, environmental impacts, system capacity, user satisfaction, and project management, among other things. The NPI data system presents consistent and com- parable data across a transportation system managed by nine separate agencies in two countries, allowing unprecedented benchmarking possibilities. In the private sector, the following are the three basic varieties of benchmarking: • Best-in-Company benchmarking fosters competition between operating units on the basis of the key metrics and best-in-company performance levels (DIY Company). • Best-in-Industry benchmarking analyzes the performance of companies in the same indus- try or segment and highlights the best in the group as the benchmark, even if its activities are not directly comparable to the subject company. • Best-in-Class benchmarking analyzes the performance of a broad range of entities, including some with unrelated activities, and highlights the best in the group as the benchmark, even if it is in a different industry than the subject company. Additionally, strategic benchmarking can identify similar companies’ strategies as a basis for setting one’s own strategy. Organizational benchmarking can be either qualitative or quantitative. One common form of organizational benchmarking is measuring staff levels used to service a given level of activity. Finally, benchmarking within the public sector also can include benchmarking against fore- casted targets for other agencies, as opposed to just benchmarking against actual best practice per- formance. The MTC uses several environmental targets that are based on California state goals, but they are not required for the MPO. Modeling Both top-down and bottom-up modeling are used to set targets in many companies. Top- down modeling most commonly drives the target by high-level requirements. Top-down modeling determines the strategies or funding needed to achieve the target; Bottom-up mod- eling determines what level of performance is possible, and then uses that to calculate the expected target (MLIT). The exact use of modeling depends to some extent on the way in which an agency is utilizing PBRA. For what part of the investment process is the agency setting targets? For example, are they annual financial targets, long-term targets for a long-range transportation plan, or mid-term targets for a package of projects and programs to be included in a TIP? Table 1.2 summarizes different tools used for modeling performance and estimating targets. These range from simple interpretations of historical data, straight-line projections (Mn/DOT), and analysis of research results; to more complex (and expensive) travel demand and eco- nomic impact models. Corporation X uses a sophisticated operations and financial model that Guide for Target-Setting II-19

calculates the impact of seven operational variables on three key business unit and corporate financial metrics. Many agencies have found innovative ways to incorporate performance-based processes and targets into their planning processes and duties, supported by modeling. The MTC uses its own StreetSaver® PMS to calculate preventative maintenance funding targets for its local jurisdictions; the ratio of “actual versus targeted” determines the jurisdiction’s perfor- mance score and is a factor in calculating the amount of funding that will be allocated to that jurisdiction. Modeling is often used by agencies to evaluate progress towards targets, even when modeling itself was not used to establish the target itself. Project and policy scenarios can be tested using travel demand models, combined with other postprocessing tools to include metrics such as benefit/cost, to evaluate their relative contribution to progress towards targets. Section 3.4 of Volume I provides an in-depth discussion of the role of economic models and management sys- tems in target-setting and tradeoff analysis. Summary of Approaches Each agency or organization must select the appropriate approach or approaches for its cir- cumstances. Figure 1.2 summarizes how various factors influence the types of approaches an agency may choose; Table 1.3 summarizes the advantages and risks of each approach. II-20 Guide for Target-Setting and Data Management Type of Tool Sample Tools Advantage Disadvantage Type of Outputs Historical data Microsoft Excel spreadsheets, Enterprise Resource Planning (ERP) systems. Easy to use. Static. Does not consider future growth and change. Line plots. Projected trends Microsoft Excel spreadsheets, SAS, STATA. Easy to use. Static. Based almost entirely on historical performance. Regressions. Travel demand model – Dynamic and allows for testing of projects and scenarios. Data and resource- intensive. Can be over-relied upon. Link and system volumes, speeds. Postprocessors IDAS, TREDIS. Provide numerous metrics that are normally difficult to calculate at system level. Require a travel demand model. Travel time, delay, crashes, emissions, employment, GDP, Value-added. Economic HERS/ST, AssetManager NT. Input data already required by states Tradeoffs. Static. Relies entirely on HPMS input. B/C, delay, crashes, highway deficiencies. Economic impact tool REMI, IMPLAN. Broader, societal measures understood by all. Requires reliable inputs. Easily misinterpreted. Employment, GDP, Value-added. Research Microsoft Excel, Access, or similar Financial Model. Emerging areas. May not be based on actual practice. Benchmarks. Management systems PMS, BMS (PONTIS, Arivu, StreetSaver). Tradeoffs. Database. Siloed. Prioritization, needs. Table 1.2. Tools for modeling performance and estimating targets.

Step 4—Establish Methods for Achieving Targets Public and private organizations use several specific methods geared towards helping an agency achieve established targets. Most critical to this, in broad terms, is the integration of per- formance measurement into daily agency practice. This directs attention to key issues, promotes financial resources for PBRA, and provides the ability to develop stronger PBRA systems. Incentives The MTC allocates funding to jurisdictions based partly on progress towards roadway main- tenance targets, which are set through a modeling approach. The MTC examines each jurisdic- tion within the MPO to see how much of their budget is allocated to preventative maintenance projects and compares that to their unique target ratio of preventive to total maintenance as determined by the StreetSaver® PMS tool. Jurisdictions with good pavement conditions will have a higher preventive maintenance target than those with poor streets since the aim of preventive maintenance is to keep the good streets good, thereby reducing long-term costs. Their ratio of “actual versus targeted” determines the jurisdiction’s performance score and is a factor in calcu- lating the amount of funding that will be allocated to that jurisdiction. The allocation of regional funds conditioned on preventive maintenance is 25 percent. In the case of the Kansas State Department of Education (KSDE), Federal No-Child Left Behind legislation presented Federal targets for the state to achieve. The KSDE ties its own state- level targets, which complement the national targets, to accreditation and limited incentives. Internal employee incentives and sanctions are discussed in the following section on Person- nel Performance Appraisal. Personnel Performance Appraisal It is important to ensure that each staff person in an organization understands his/her contri- bution to the mission and that the level of contribution should be part of the staff review process. These organizations have learned to “manage by the measures.” Guide for Target-Setting II-21 Target-Setting Approach Advantage Risk Approach to Balancing Advantages with Risks Edict • Less time and money intensive. • Unequivocal and well - understood. • Lack of defensibility and inclusion. • Use hybrid approach. Expert Opinion • Insures broad understanding and acceptance within and outside agency. • May flounder in effort to be inclusive. • Appoint internal champion to lead effort to identify the “ critical few ” measures and targets. Customer Feedback • Insures more transparent process. • May be confusing to discuss technical measures with public. • Describe measures and targets in the simplest terms possible. Benchmarking • Provides a peer group comparison. • Can be misused for comparative rankings. • Continue to refine comparative analysis techniques. Modeling • Defensibility. • Better understanding of future performance. • More time and money intensive. • Models change over time. • Continue to refine modeling techniques. • Use hybrid approach. Table 1.3. Managing risk of target-setting approaches.

Salary and incentive bonuses, powerful motivators, result in targets that are driven by what people can achieve rather than calculated guesses. Also, compensation that varies depending on performance versus the target is widely viewed as an effective way of motivating perfor- mance. Agencies also can use nonmonetary recognitions, even elaborate ones, in order to achieve a similarly high motivational level. MDOT SHA (Maryland Department of Transportation State Highway Administration) is leading a pilot on behalf of MDOT to base managers’ performance appraisals on performance plans that link to office/district business plans as well as individual performance targets. SHA has completely changed its assessment forms to incorporate performance management in these per- sonnel reviews. The assessment now consists of two parts: Leadership competencies (40 percent) and an annually updated Performance Plan (60 percent). Performance is now linked to person- nel reviews for staff down to the midmanagement level. For these staff members, the focus is on output measures as opposed to outcome (longer-term strategic) measures. In the private sector, ABC Logistics uses a Darwinian performance evaluation system that favors high performers and weeds out underperformers through the judgmental application of raises (or the reverse), bonuses, and promotions. Quality Control and Support Through continuous tracking of performance and progress towards targets, agencies can iden- tify when problems occur and deal with the shortcomings in a timely manner. Some organiza- tions, especially those with a strong culture of performance management well integrated into most processes, have teams dedicated to performance management and even specific leads for individ- ual goal areas or measures; these groups and individuals can identify shortfalls in progress, work to identify why the shortfalls are occurring, and provide additional support or make other changes as necessary. II-22 Guide for Target-Setting and Data Management If Maryland Transportation Authority (MDTA) performance is below target (i.e., employee retention and invoice processing time), the Performance Management Team assigns a quality improvement team to work with the division and improve the process and increase performance. At DIY Company, failure to achieve a target results in the invocation of a performance improvement plan. Tradeoff Analysis The concept of “tradeoffs” summarizes the main challenge facing transportation agencies—there are more needs then resources available to address them. In this environment, agencies must con- tinually make difficult decisions on which areas of the transportation network to focus their limited resources. Transportation is often a zero-sum game, so additional investment in one area means that an agency must invest less in another. For example, what would be the impact on pavement per- formance if pavement funds are increased by 10 percent over the next 10 years? What would be the impact on bridge performance if this money was shifted from the bridge program? In the context of the overall performance management framework, this type of analysis can help agencies establish relative priorities, set targets, allocate resources, and better manage stakeholder expectations. Tradeoff analysis currently is limited in public sector transportation agencies. However, Sec- tion 3.4 of Volume I provides an in-depth discussion of the role of economic models and man- agement systems in target-setting and tradeoff analysis.

Competition Intracompany team competition also is an effective way to motivate performance and estab- lish the right targets based on what can be achieved through competition rather than on a calcu- lated target. Seeing a business unit compete and succeed can stimulate an aggressive response from a competitive peer. Baldrige winners use internal competition extensively to instill creativ- ity and better results. Availability of Data Wide and easy access to targets and performance data helps stimulate better knowledge of current performance and performance gaps, hence the ability to improve performance and to do it more rapidly. In contrast, complexity and waffling will dampen attempts to improve performance. Numerous private-sector Baldrige winners plus several Balanced Scorecard users rely heavily on the wide dissemination of performance data inside their companies. Corporation X posts its performance metrics to an intranet where the operating results are available to all users. Table 1.4 summarizes the advantages and risks of each target-setting approach. Guide for Target-Setting II-23 Method for Achieving Target Advantage Risk Approach to Balancing Advantages with Risks Incentives Strong motivator. Inequalities. Nonmonetary rewards. Could sacrifice performance on one parameter in trying to maximize another. Use incentives rather than sanctions. Incentives or sanctions at program or division level may not fit with public-sector priorities. Personnel Performance Appraisal Strong motivator. Inequalities. Team or group bonuses. Results in a “natural” target. Could sacrifice performance on one parameter in trying to maximize another. Nonmonetary rewards. Quality Control and Support Avoids disincentives for not meeting targets. By itself provides less direct incentive for achieving targets. Combine with other methods. Tradeoff Analysis Optimizes resources for reaching targets Stakeholder/customer understanding of resource constraints. Requires more data and modeling sophistication. Build capabilities over time. Competition Results in a “natural” target Stimulates sharing of best practices. Could sacrifice performance on one parameter in trying to maximize another. Stimulate internal competition based on a balanced scorecard of metrics. Availability of Data Facilitates more improvement ideas. Education and interaction with peripheral users of the data could derail progress. Apply varying levels of access permissions. Technology to disseminate the information could be costly. Use web-based gathering and filtering of input. Table 1.4. Methods for achieving targets.

Step 5—Track Progress Towards Targets “Measure and Report Results” already exists as an element in the Performance Management Framework; it is the final element before returning back to the beginning of the iterative process. Agencies should explicitly include tracking progress towards targets as part of this element of the Framework; four steps are suggested. II-24 Guide for Target-Setting and Data Management In the private sector, metrics are shown so that an increase (or moving up on a graph) always indicates a positive result, and percent change is a preferable reporting method. Tracking Develop Monitoring Plan The monitoring plan should address elements such as what specifically is being tracked and if data collection is needed to support target-tracking, what data is to be collected, who will collect it, how will it be collected, where it will be stored, and how it will be reported back to the end-user. The plan should build upon existing data collection efforts (including existing procedures, equipment, and schedules) as much as possible. The plan also should build upon the facts, estimates, and analyses that were conducted as part of planning, programming, and budgeting for this project. These data would have been used to evaluate and rank the project by assessing its proposed improvements to transportation system performance and comput- ing its benefits and costs. These data will now form a baseline for comparison with perfor- mance tracking; compiling these data within the plan will ensure that all parties work with the same baseline. The plan should consider the following possibilities and options: • Initial Monitoring Period. Certain transportation investments have an impact on transporta- tion system behavior immediately upon project completion (e.g., improvements to intersec- tions and signalization). For other investments, the performance improvement is apparent only after some period of time (e.g., the extension in asset life expectancy following preven- tive maintenance). The plan may need to distinguish between these two possibilities by speci- fying what measures should be tracked during the initial monitoring of each project: outcomes versus outputs. • Outputs and Outcomes. Ideally, outcomes are tracked to determine whether investments have fulfilled policy objectives and targets. Tracking of improvements in outcomes can begin within the first year for those projects that have an immediate impact on performance (e.g., the intersection and signalization improvements noted previously). For those projects where changes in outcomes are not evident for some period of time (as with preventive mainte- nance), it may be desirable to include the documentation of output measures as well during initial monitoring. Outputs document the accomplishment of work and the type of solution used (e.g., number of bridges retrofitted with seismic protection, and number of miles resur- faced with hot-mix asphalt versus number of miles chip-sealed). They verify the fulfillment of decisions on method-of-project-accomplishment made during program and budget prepara- tion and provide a quantitative basis for reporting to stakeholders and the public in a period when meaningful outcome data are not yet available.

• Information Sources. The Monitoring Plan will encompass a wide range of information sources to cover the several technical areas of the highway investment program (e.g., pavements, bridges, safety, congestion relief, environmental mitigation, etc.) and the need for output as well as out- come measures in some instances. Additional data (e.g., from planning or strategic management) will need to be monitored to analyze external effects. Track Progress After the initial monitoring period, tracking of performance outcomes becomes an ongoing process according to procedures in the Monitoring Plan and the defined performance measures. In conducting this tracking, the following two possibilities should be recognized and dealt with if needed. • Isolate External Influences. The purpose of tracking performance in PBRA is to determine the effects of transportation investments. There are times, however, when external factors can con- found this relationship. Apparent changes in performance measures that are due to external causes rather than transportation-related actions produce a misleading indication of the benefit of a transportation investment. These external factors may comprise, for example, population and demographic shifts that have not been accounted for, unanticipated changes affecting travel demand (e.g., due to price changes in gasoline), technological changes affecting the vehicle fleet, and catastrophic natural disasters. The effects of these external factors should be isolated wher- ever possible. Apparent performance trends can be compared to data and assumptions in the baseline estimates to identify anomalies that may signal external influences. Comparison of actual trends (e.g., in population and demographic characteristics) to rates established, for example, by the planning office helps discern where prior assumptions may require change. • Reconcile Competing Targets. Competing targets may create apparent tension in determining the performance benefits of investment decisions. For example, an economic development proj- ect to spur commercial activity may increase congestion at locations in the network. This type of issue can be resolved by understanding cause-and-effect, clarifying what is happening, and tak- ing appropriate steps to resolve the situation (e.g., subtracting the costs of congestion from the benefits of increased commercial activity; dealing with the congestion problem through a follow- up project if justified; clarifying and adjusting the specific economic and congestion targets for these network locations to reflect the perceived agency priorities; and proceeding forward). Develop Findings After sufficient time has passed to develop reliable performance trends (accounting for the types of checks and adjustments discussed in the previous section), progress towards the per- formance targets can be assessed. If progress appears to be on track to meet the target, the worth of the project and the merit of the decision to undertake the investment will begin to be verified. Moreover, confidence in the data and analytic models and procedures used to evaluate the proj- ect will be strengthened. If, after applying the checks and adjustments, it appears likely that the target will not be met, a review should begin to try to determine the cause of the divergence. Comparison of the actual Guide for Target-Setting II-25 Outcome measures such as condition, rideability, and serviceability can be tracked continually for pavements, but they may not show initial-period improvement from preventive maintenance investments since preventive maintenance is per- formed while the asset is still in good condition.

trend with the baseline assumptions and predictions may help to identify where and why the deviation from the intended track has occurred. Check Validity of Performance Measures and Assumptions The performance measures and analytic procedures and assumptions should be reviewed peri- odically for currency and relevance, even when targets have been met. For example, performance measures may need to be updated to reflect changes in policy or governing standards. Assump- tions (e.g., regarding population, demographic characteristics, use of different modes, and other factors driving travel demand or supply) likewise should be reviewed in light of current economic, social, technological, political, and financial trends. These are analytic checks and adjustments; the following step describes policy adjustments with respect to the performance targets. Reporting Part of tracking progress also involves communicating this performance in terms that are read- ily understood to the agency’s executive decision-makers, other stakeholders, and the public, so they also can track progress towards targets; this is particularly necessary in a collaborative process, such as “Customer Feedback” or “Expert Opinion,” in which others are directly involved in the target-setting process. Communicating targets in a manner that makes sense to the general public seems to be a strong indicator for the success of PBRA and the integration of target-setting. Coral Springs “rolls up” an extensive series of performance measures into 10 key composite meas- ures, referred to as the city’s Stock Index, summarizing, at-a-glance, city performance; Hennepin County uses a Balanced Scorecard approach in which numerous measures are evaluated and tracked in terms of multiple perspectives (customer, finance, internal process, learning and growth) and simplified into tables of information providing “warning lights” for areas in need of improvement. II-26 Guide for Target-Setting and Data Management Setting targets and monitoring achievement of targets is a powerful motivator for behavior: “success breeds success.” Mn/DOT prepares a one-page “snapshot” with performance measures and red, yellow, and green colored shapes to represent annual progress relative to targets, by state and by district. The snapshot graphically illustrates the trend direction and projects next year’s forecast. Other agen- cies use annual attainment reports (MDOT). Step 6—Adjust Targets Over Time Performance management is a dynamic process in which performance measures and targets evolve over time. The key to this evolution is periodic assessment of the impacts of the measures and targets on actual investments. The Performance Management Framework itself is an itera- tive process. Agencies should explicitly include the adjustment of targets over time within the feedback loop of the Framework. Factors driving a possible need to adjust targets from a policy perspective include the follow- ing examples: • Changes in the level of funding or in rules governing project eligibility to receive certain pro- gram funding. These changes can work in positive or negative directions, and program targets may need to be adjusted up or down to reflect these updated expectations of how and where program funding is to be applied.

• Changes in state or Federal policy, or in program priority, as affected by executive or legisla- tive action. Existing performance targets may need to be adjusted, and new targets created, to address new or revised policies and priorities. • Changes in the behavioral characteristics of the transportation system assets and vehicles. For example, greater use of hybrid vehicles may eventually cause a revision in environmental miti- gation targets. Use of innovative materials may allow a refinement of asset preservation targets. Introduction of new inspection technologies may require the creation of new performance mea- sures and associated targets. When adjusting, agencies also should consider the following items: • Adjust performance targets only after sufficient time has passed to accumulate sufficient time- series data and to make necessary checks and adjustments, such that a reliable trend has been developed. The trend should provide a fair and reasonable indication of current transporta- tion system behavior and be one that can be supported by facts, analyses, customer surveys, and other sources of information. • Resolve factors such as model updates, data collection methodologies, etc., that may be influ- encing the calculation of the target, as opposed to the influence of the actual investment itself (refer to Step 3) • Account for interagency responsibilities in monitoring and tracking performance [e.g., coor- dination with MPOs and Regional Planning Organizations (RPOs)]. These interactions should be reflected in the monitoring plan. Agencies that are only at the beginning of implementing a performance-based process gener- ally have less complete and less sophisticated target-setting processes. In general, there is a typi- cal evolutionary path that agencies follow. A corollary to this evolution is the emergence of an agency’s data sophistication. Iterations of long-range planning cycles (MTC, Mn/DOT), a solid history of performance data, and managerial comprehension and appreciation of that data allows managers to discern what they can or cannot control (WSDOT). During the development of each long-range plan, agencies can reassess what has worked and what has not, adjusting measures and targets accordingly. Targets can be reassessed on a more frequent basis depending on the level of integration of the Performance Management Framework into an agency’s planning and internal processes. MLIT’s annual Performance Measures Report/Planning Report monitors the bureau’s progress towards its annual and 5-year targets. If targets are not met for a program, then a closer review is performed to determine how the processes for that program may need to be revised, or if a new program may need to be developed to address those performance needs. Conversely, if tar- gets are consistently met earlier than anticipated, target deadlines or measures are reset to reflect more accurate expectations. Targets at the Maryland Transportation Authority (MDTA) not met are used in lessons learned: the Authority evaluates what happened and why it did not reach the target. The following questions should be asked: • Were there areas within the target that didn’t work? • Is the Authority attempting to set too high a target? • Is the Authority measuring the wrong component? Guide for Target-Setting II-27

II-28 This Guide explains how transportation agencies can use data management and governance to strengthen existing Performance Measurement and Target-Setting programs in the agency. It applies the research results presented in Volume I into practical guidance for transportation agencies. The case studies and examples examined in Volume III were used to produce the guidance. The Guide is organized under the following headings: 2.1. Establishing the Need for Data Management/Governance. Data governance is central to continuous improvement. Each private sector case study company noted that data management, data governance, and data accessibility have markedly improved their ability to meet their targets. For example, Corporation X’s data governance framework is central to ensuring data of sufficient quality to feed its operational-financial model. ABC Corporation measures its deviations intently. And both ABC, MNC, and Corporation X use data transfer protocols with supply chain partners and governmental entities to collaboratively improve performance. In support of the need for data governance, this section describes the important relationship between data management and performance measurement and provides a maturity model to assist agencies in assessing their state of data governance. This section is intended to assist data managers in demonstrating the need for data management and governance and prepares them for implementing the strategies described later in the Guidance. The remainder of the sections assumes an agency is committed to improving their data management practices. 2.2. Establishing Goals for Data Management. Once an agency has committed to making improvements in their data management practices, a plan to achieve this should be formed. This section describes the steps and processes to planning for successful data management. 2.3. Assessing Current State of Data Programs. In this section, tools and techniques are described related to the first step of the journey—assessment of current data practices, tools, and processes. 2.4. Establish Data Governance Programs. This section offers guidance for executing and maintaining institutional data management principles based on knowledge gained in Section 2.3. 2.5. Technology for Data Management. This section suggests technological tools and techniques. 2.6. Linking Data to Planning, Performance Measures, and Target Processes. This section provides detail related to success factors in this area. Guide for Data Management C H A P T E R 2

The success factors described in Volume I, Section 4.8 will be referenced in each subsection. The Guide is intended to provide more tools and details to assist agencies in implementing and applying the success factors to achieve successful data management. 2.1 Establishing the Need for Data Management/Governance The need and urgency for data management improvements are not always shared across all levels of an agency. In some cases, a senior manager within the agency identifies the need, and in other cases, individuals at lower levels recognize the value of improved data management. Nev- ertheless, a clear case must be established to secure resources and commitment to proceed with a data management improvement strategy. This section is designed to assist agencies in making that case. The first section covers the relationship between data management and performance measure- ment in a transportation agency. The second section documents definitions and advantages of data governance techniques. The third section presents a data management maturity model and the final section provides a tool for assessing how well an agency is performing in data manage- ment and governance. The two key success factors related to establishing the need for data governance are the following: • Demonstrate the Return on Investment (ROI) to the organization regarding the use of data management and data governance in order to gain buy-in from executives and decision-makers. Demonstrate with specific examples how the use of data governance can meet the goals and targets most important to executives. ROI can be determined in many ways and on many levels within an organization. For instance, in a Highway Safety Improvement Program (HSIP) ROI can be determined in the following ways: (1) from the perspective of the HSIP Statewide Coordinator, an investment in more resources (e.g., people, technology, tools) may lead to the ROI of an improved HSIP strategic plan; (2) for traffic and safety engineers, an investment in Global Positioning System (GPS) field inventory projects may lead to the ROI of improved crash locations; and (3) for the Highway Safety Planning Agency, an investment in electronic data collection may lead to the ROI of improved quality of crash records. ROI also can be realized across business functional areas within an agency or across agency boundaries. In the highway crash safety example, ROI can be realized in the following ways: (1) for law enforcement personnel, an investment in electronic crash data collection and sub- mittal may lead to the ROI of reduced time to complete the accident investigation and review; (2) for maintenance and operations personnel, an investment in digital imaging capabilities may lead to a ROI of quicker and less costly asset management inventory and reduced cost to prepare HSIP projects for the traffic and safety engineers; and (3) for the executive manage- ment, investment in an enterprise Geographic Information System (GIS) deployment may lead to the ROI for improved tradeoff analysis on project selection by visualizing the crash his- tory, traffic, and pavement condition. A data governance framework, implemented on an enterprise level, supports ROI by pro- viding a means of monitoring and tracking progress of various business programs for execu- tives as well as data stewards, stakeholders, and users of the source data. Data governance provides methods, tools, and processes for the following: – Traceability—aligning data programs with the agency’s business needs. Establishing data area communities of interest and working groups that examine needs in common areas and on a regular basis is essential. Guide for Data Management II-29

II-30 Guide for Target-Setting and Data Management – Performance Measures—should be reflective of the business needs identified in the trace- ability exercise. – Risk Assessment—requires the agency to assess (1) how much data is needed, (2) how accurate should the data be, (3) what should the refresh rate of the data be, and (4) who should have access to the data as well as many other questions which help to assess the risks associated with a particular data program. – Value of Data Programs—needs to be demonstrated to users and those who authorize investments in the data programs. This can be done effectively through the use of visuali- zation tools, use of enterprise GIS systems, collecting data once and using it for many pur- poses, and demonstrated improvements in business operations through the use of quality, accurate, timely, and easily accessible data and information. – Knowledge Management—must become part of the data governance framework in order to ensure that lessons learned and experiences pertaining to business operations within the organization are not lost. This will help to increase the ROI for time and resources com- mitted to support of data programs. • Formalize a Data Business Plan for the agency or department which identifies how each employee’s job is linked to the agency’s mission and goals, thereby clarifying the importance of their role in the overall success of the department/office. Corporation X uses a committee composed of the Finance Department, the Capital Committee, and the Performance Measure- ment Group to monitor the data collection procedures and data revisions, as well as to set data standards and operating definitions. A Data Management program strengthens support for performance management in a transportation agency through the use of a Data Business Plan. Relationship of Data Management and Stewardship to Performance Measurement and Target Setting in a Transportation Agency Each transportation agency is faced with many challenges and needs regarding the availability of data and information to support business operations. The needs described were identified by Mn/DOT in July 2008, in preparation for the development of a data business plan for that agency. They pertain to the ability of the data programs to support performance measures, target setting, and prioritization of resources in Mn/DOT. Many of these needs are relevant to transportation agencies across the nation and include the following: • More transparency and accountability, • More efficient ways to locate and take advantage of available data and information, • Better methods to look at and integrate data from multiple sources, • Processes and systems that reduce redundancy and promote consistency in data results, • More timely and real-time data and information, and • More department-wide spatial data tools. One of the ways to address these and other data-related needs is through the establishment of a structured data management program and data governance framework. Data management and data governance can help the agency to prioritize the most critical data needs and identify the resources available to address those needs in a timely manner. Institutional challenges may include: centralized policy-making and decentralized execution of those policies; limited appreciation by decision-makers of the role of data systems in support- ing business operations; and lack of formal policies and standards which guide the collection,

processing, and use of data within the organization. It is particularly critical to have standard- ized policies and procedures for management of data and information when that information is the foundation of performance measurement and target setting programs for an agency. A data management program is used to coordinate the establishment and enforcement of data policies and standards for the organization. Challenges to establishing a Data Management program may be both institutional and technical in nature. However, implementing Stewardship and Governance in the organization supports the overall role of Data Management. Definitions and Benefits of Data Management, Stewardship, and Governance Data management is defined as the development, execution, and oversight of architectures, policies, practices, and procedures to manage the information lifecycle needs of an enterprise in an effective manner as it pertains to data collection, storage, security, data inventory, analysis, quality control, reporting, and visualization. Data governance is defined as the execution and enforcement of authority over the manage- ment of data assets and the performance of data functions. The management of data assets for an organization or state DOT is usually accomplished through a data governance board or coun- cil. This role is critical in successfully managing data programs that meet business needs and in supporting a comprehensive data business plan for the organization. More information on data governance is included in Volume I, Section 4.3. Data stewardship is defined as the formalization of accountability for the management of data resources. Data stewardship is a role performed by individuals within an organization known as data stewards. A data program in this report refers to specific data systems that support a business area of the organization. The “program” usually includes the functions of data collection, analysis, and reporting. In the case of a DOT, some examples of these programs include traffic, roadway inven- tory, safety, and pavement data. The definitions and examples are covered in more detail in Volume I, Section 4.2. A strong Data Management program improves data quality and limits potential risks to the agency regarding loss of critical data and information. Data Management A Data Management program is used to do the following: • Strengthen the ability of data programs to support core business functions of the agency, • Improve data quality throughout the organization, • Protect data as an asset of the agency, and • Limit risks associated with loss of data and information. Data Governance The benefits of using data governance can be demonstrated from three different perspectives within the agency—policy, practical and technical. Guide for Data Management II-31

II-32 Guide for Target-Setting and Data Management From a policy standpoint, data governance promotes the understanding of data as a valuable asset to the organization and encourages the management of data from both a technical and business perspective. On a practical level, the use of a data governance model provides for access to data standards, policies, and procedures on an enterprise basis. It provides a central focus for identifying and establishing rules for the collection, storage, and use of data in the organization. From the technical perspective, use of data governance results in reducing the need to main- tain duplicate data systems, improves data quality, and provides new opportunities to imple- ment better tools for managing and integrating data. Incorporating some form of data management and governance within the organizational structure of the agency can benefit every transportation agency because their business operations rely on quality data programs for decision-making. In support of data quality control, Corporation X’s dedicated performance measurement group “owns” the data that is gathered by the hardware, software, and processes. In this way it controls the quality of the data so that it is neither too dirty (which would render it useless) nor too pure (which would result in an exorbitant cost). Data Management Maturity Model A maturity model is a framework describing aspects of the development of an organization with respect to a certain process. It is a helpful tool to assess where an organization stands with respect to implementing certain processes. A maturity model also can be used to benchmark for comparison or assist an agency in understanding common concepts related to an issue or process. A typical maturity model identifies levels and characteristics of those levels. The model can be used to assess an agency’s status and assist in identifying next steps to achieve success towards an ultimate goal state. A Data Management Maturity Model is used to assess how the roles of people, technology, and institutional arrangements help the agency to advance from a state that is un-governed to a governed state. A maturity model was developed here to document levels of maturity related to the develop- ment and application of data. The desired end state is the establishment and maintenance of a data governance system that supports performance measurement and target setting within a transportation agency environment. The criteria (people/processes, technology/tools, and insti- tutional/governance) are the following: • People/processes—This refers to the willingness, understanding, and commitment of people within the agency to embrace data management. It also refers to processes that may be in place to assure employees understand and appreciate the value of data management. • Technology/tools—This refers to the use of tools and techniques designed to assist agencies in collecting, integrating, analyzing, and reporting data. More details are provided in Volume I, Sections 3.4 and 3.5. • Institutional/governance—Refers to the institutional structure within an agency to ensure consistent management of data programs. More detail can be found in Volume I, Sections 3.2 and 3.3.

The levels are somewhat generic in nature and are described as: 0—Ad Hoc; 1—Aware; 2— Planning; 3—Defined; 4—Managed; 5—Integrated; and 6—Continuously Improving. Table 2.1 documents the levels of maturity within the categories. It is assumed that the model will be used to assess the overall status of data management within the entire agency; however, it also can be used to assess the status within a unit of the agency. To assist agencies in determining where they are in the process, the following characteristics are provided related to each level. People 0. Management and staff across the agency do not recognize a specific need for a data manage- ment program to support performance management. 1. Some personnel in the agency are aware of the need for a formal data management program and/or processes to support performance management but are not involved in developing such a program. 2. Some personnel in the information technology (or similar) office of the agency currently par- ticipate in the development and implementation of a data management program for the agency. 3. Work teams have been identified in several offices across the agency to participate in the development and implementation of a data management program. 4. Staff across the agency are aware of the data management program and use the program rou- tinely for the collection and use of data within the agency. 5. Staff across the agency are actively involved in recommending changes for data management policies, standards, and procedures, as business needs change and new performance manage- ment goals are identified. 6. People in the agency are fully engaged in continuous improvement related to data manage- ment and performance measures. Technology/Tools 0. The agency does not have any information technology tools in place to support data management. 1. The agency has delegated the responsibility to a specific office, such as Information Tech- nology, to determine what IT tools are needed to support data management across the agency. 2. The agency has implemented some information technology tools, including GIS, data mod- els, data repositories, data dictionaries, etc., to support data management in certain offices of the agency. 3/4. The agency uses information technology tools on a widespread basis, including such appli- cations as an enterprise data warehouse, GIS systems which integrate business data from var- ious offices, and dashboards and scorecards delivered through a web-enabled interface for access agency-wide. The agency uses Service Oriented Architecture (SOA) and Open Data- base Connectivity (ODBC) in the development of new applications to support future inte- gration of applications. 5. The agency uses a Knowledge Management system throughout the agency to support its data management program. 5. Performance management tools, such as dashboards and scorecards, are used in every office of the agency to monitor the progress of agency programs in meeting the agency mission and goals. 6. Performance measures and targets are adjusted as needed and displayed on the agency dash- board, or similar mechanism, to maintain peak program performance across the agency. 6. The use of technology and BI tools in the agency improves the overall management of pro- grams in the agency, in accordance with the strategic mission, goals, and targets. Guide for Data Management II-33

II-34 G uide for Target-Setting and Data M anagem ent Level 0 — Ad Hoc 1 — Aware 2 — Planning 3 — Defined 4 — Managed 5 — Integrated 6 — Continuous Improvement Technology/ Tools No tools in place. Planning for tools to support data management in some offices. Planning for tools to support data management across the agency or for a specific office. Implemented some tools to support data management but not widespread across the agency. Widespread implementati on of tools to support data management but not integrated. Integrated, widespread implementation of tools to support data management and performance measurement. Ongoing assessment of new technology to support and improve data management and performance me asurement. People/ Awareness Not aware of need for improved data management to support performance measurement processes. Aware of need for improved data management to support performance measurement processes. No action has been taken. Aware of need for improved data management to support performance measurement processes. Some steps have been made within the agency to improve technology or institutional setting to support data management in at least one office. Aware of need for improved data management to support performance measurement processes. Some steps have been made within the agency to improve both technology and institutional settings to support data management in more than one office. Aware of need for improved data management to support perfo rmance measurement processes. Improvements are under way to improve both technology and institutional settings to support data management across the agency. Aware of need for improved data management to support performance measurement processes. Technology and institutional processes are in place to support data management for performance measures. The agency is able to develop performance measures and predict outcomes for programs based on success with other programs. Institutional/ Governance No data governance in place. The agency is discussing needs/plans for data governance. Some level of data program assessment and formulation of roles for data managers is underway in one or more offices of the agency. Data Business Planning underway, including dev elopment of governance models for multiple offices in the agency. Data Business Plan developed with data assessment complete and data governance structure defined. Fully operational data governance structure in place. Data governance structure fully suppo rts data management activities across the agency. Table 2.1. Data management maturity model matrix.

Institutional/Governance 0. The agency is not aware of the need for an institutional arrangement or organizational struc- ture to support data governance. 0. The agency does not have strong executive level support for data governance. 0. The agency does not have a Data Business Plan in place to support management of core data programs. 0. The agency does not have defined roles, such as data stewards, stakeholders, business owners (of data), and communities of interest, to support a data governance framework. 1. Agency senior management recognizes the need for a Data Business Plan to manage critical data programs; however, a plan has not yet been developed. 2. The agency is developing a Data Business Plan to support management of strategic data programs. 3. A limited number of offices in the agency have implemented a Data Business Plan to manage the core data programs for their area. 4. The agency has strong executive and senior management support for data governance. 5. An enterprise Data Business Plan has been developed to support management of core data programs across the agency. 5. The agency Data Business Plan has been incorporated into the overall agency strategic plan. 5. Data champions have been identified in each business area of the agency. 5. Communities of interest, which are comprised of internal and external users and stakeholders for core data programs, have been defined. 5. A data governance council or data governance board exists at the agency to direct the data management activities of the agency. 5. The agency has developed and published a Data Governance manual or handbook which identifies the roles and responsibilities of staff in the agency to support data governance operations. 6. The agency has developed a data catalog with data definitions, standards, policies, and pro- cedures for the collection and use of data in the organization. The catalog is available on an enterprise basis through an electronic system such as a Knowledge Management system. Application of the Transportation Data Governance Model assumes that an agency recog- nizes the need to embrace and apply data management and governance concepts. The first suc- cess factor listed in earlier sections states that an agency should “Demonstrate the ROI to the organization regarding the use of data management and data governance in order to gain buy- in from executives and decision-makers. Demonstrate with specific examples how the use of data governance can meet the goals and targets most important to executives.” This can be done by citing examples of other agencies that have certain accomplished levels of maturity with respect to the model. Examples can be found throughout the case studies and examples cited in Section 4. Planning for Data Management There are several ways to achieve success with respect to data management and governance to support performance measures programs. One approach is to develop a Data Business Plan. Whether an agency formally refers to their process improvement as a data business plan or not, the following common steps should be taken: 1. Establish goals for data improvement process; 2. Assess data programs; 3. Establish governance programs; 4. Ensure proper use of technology/tools; and 5. Link data management to performance measures and target-setting. Guide for Data Management II-35

II-36 Guide for Target-Setting and Data Management Some agencies may choose to implement parts of this list or simply set up and maintain stew- ardship and governance policies. The following sections provide more detailed guidance related to these steps. 2.2 Establishing Goals for Data Management As with any typical planning process, defining stakeholders and setting goals are important first steps. In most cases a champion is responsible for starting this planning process. The success factors for planning for data management are the following: • Start with a smaller achievable goal when implementing data governance within an organiza- tion and build on small successes to address larger agency goals. • Use a Data Business Plan to strategically manage data programs similar to other strategically managed programs within the organization. • Manage expectations of how data governance can help an organization by explaining the ben- efits of such models for supporting business operations. • Use Business Models to help executives and managers better understand the relationship between target setting and decision-making. • Identify champions from Business and IT sides of an organization to support key systems. Partnerships between both areas are critical to successfully managing data programs. A Data Business Plan helps to do the following: • Establish goals; • Assess Agency Data programs; • Establish Data Governance; • Ensure proper use of technology/tools; and • Link Data management to Performance Measures and Target Setting. Success factors for Data Management: • Start small with achievable goals; • Use a Data Business Plan; • Manage Expectations; • Use Business-Use Case Models; and • Identify Champions. Recognizing areas for improvement is a key first step in being able to establish goals for a data program. Brainstorming sessions with affected stakeholders is a very effective way to identify both problems and solutions related to data programs. The brainstorming should lead to the establishment of a vision and set of goals for the process. It also is critical to relate the goals of the data programs to the business objectives of the agency as a whole. This is accomplished through the following steps: • Step 1—Identify the business objectives of the agency. • Step 2—Identify the business functions or services of the agency that support the business objectives.

• Step 3—Identify which business functions are supported by which data programs. • Step 4—Establish policies, standards, and procedures which mandate how data is to be collected and used within the agency. • Step 5—Establish Data Action plans on both a data program and enterprise level, to address needs and gaps in data and information across the agency. • Step 6—Establish a risk management plan for protecting data programs as valuable assets of the agency. Mn/DOT is an excellent example of an agency who conducted a thorough, detailed, and well thought out planning process. They began with the establishment of a BIC which serves as the leadership body for the development and implementation of the data business plan. Its charge was to do the following: • Craft a vision and mission for managing data and information in the department; • Develop and implement processes for identifying and prioritizing data and information gaps and needs; • Identify new data governance principles and frameworks to effectively manage information; • Develop a business plan that recommends strategies, actions, and resources required to achieve Mn/DOT’s data and information vision and mission; and • Share the data business plan with Division Directors and Commissioner’s Staff and assist with the implementation of approved actions and strategies. The vision and mission was established as follows: • Vision—All Mn/DOT business decisions are supported by reliable data. • Mission—To provide reliable, timely data and information that is easily accessed and shared for analysis, and integrated into Mn/DOT’s decision-making process. The BIC also identified a comprehensive list of issues to be addressed in their Data Business Plan listed. At the time of the writing of this Guide, the BIC was still working on the rest of these items. • Once goals for the data management process are in place, the assessment of data programs can begin. 2.3 Assessing Current State of Data Programs The previous sections provided tools to assess an agency’s state of readiness for developing and implementing data governance and laid the groundwork for beginning improvement. Once goals for the Data Management process have been established, an agency should work on clearly identifying and linking data programs to office and agency-wide goals. This section provides guidance related to assessing an agency’s data programs, so an appropriate data management improvement strategy can be established. It will assist agencies in conducting surveys, work team meetings, focus groups, or other mechanisms for gathering information regarding customer needs for data programs, agency needs for the programs, and gaps that need to be addressed with a data management program. A Risk Assessment is a key component of assessing current state of data programs. Guide for Data Management II-37

II-38 Guide for Target-Setting and Data Management This section assists in achieving the following success factors: • Performing a health assessment of data systems to determine where the most critical deficien- cies exist and to develop a strategy for addressing those deficiencies; and • Performing a risk assessment of existing data programs to highlight the importance of mission critical programs to management and, thereby, gain continued support for those programs. Identifying Data Programs The first step in assessing an agency’s data programs is to clearly identify which programs will be included in the assessment. Other components include identifying which data products are provided by the data programs and who the providers/users of the data products are. Most of the time an agency already has identified data systems, databases, data offices, or even data programs. These should be cross walked so all stakeholders can clearly understand how the data systems, processes, and programs interrelate. The identification of data programs also should include a connection to missions and business core services. An excellent exam- ple of the result of such a process is shown in Figure 2.1. Alaska DOT&PF linked the overall ADOT&PF mission to the core services, business programs, and primary and secondary data systems. This is an important step in assessing the value of the programs in terms of meeting high-level agency goals. The process of organizing data categories and relating them to other institutional frameworks within the department is important to the data business planning process for two reasons. First, it allows all stakeholders to clearly see how their data program(s) fit into the over- all existing structure of the agency. This ensures buy in for the plan and an understanding of how the data systems fit together and are essential to support the overall mission of the agency. Sec- ondly, an established list of data categories allows for the assignment of data governance roles as described later in this Guide. Alaska DOT&PF developed a more detailed framework shown in Figure 2.2. The framework links business objectives, programs, and processes to data systems, services, and products. It also starts to define how the data stewardship roles fit in. It is important to note how Alaska has carefully defined data systems, services, and products. Stakeholders, such as those responsible for collecting data, can quickly see how their institutional systems or reports fit into the overall structure. Another example, Figure 2.3, illustrates a similar framework used in Virginia. The Virginia framework clearly indicates the applications that are used to support the data products. The frameworks should be accompanied with reports defining the systems and relationships. In both of these state examples, reports were generated and distributed to a large number of stakeholders within and outside of the DOTs. Evaluating Data Programs To begin prioritizing needs for data programs, they must be carefully evaluated in terms of their ability to meet overall agency goals. For example, traffic and safety data programs must pro- duce quality data to support decision-making regarding safety and mobility projects. Criteria must be developed to assess the data programs. An example of the type of criteria that could be used were initially identified for use with the FHWA’s Traffic Data Quality Management Report

and are applicable, as well, for assessing quality of data used for performance measurement and target setting. These criteria include the following: • Accuracy—The measure of degree of agreement between a data value or sets of values and a source assumed to be correct. • Timeliness—The degree to which data values or a set of values are provided at the time required or specified. • Completeness—The degree to which the data values are present in the attributes (data fields) that require them. Figure 2.1. Alaska DOT&PF data development program. Guide for Data Management II-39

II-40 Guide for Target-Setting and Data Management • Validity—The degree to which data values satisfy acceptance requirements of the validation criteria or fall within the respective domain of acceptable values. • Coverage—The degree to which data values in a sample accurately represent the whole of that which is to be measured. • Accessibility—The relative ease with which data can be retrieved and manipulated by data consumers to meet their needs. For example, these measures helped ADOT&PF identify which data programs were most crit- ical to agency operations and also where data was lacking to meet department needs. The crite- ria were tested through interviews with key stakeholder groups. The Virginia Department of Transportation (VDOT) used a survey approach to assess their data programs. Mn/DOT also used an agency-wide survey as a valuable tool to begin the assessment process and will provide Business Programs Highway Safety Traffic Road Weather Management 511 Traveler Information GIS Services Business Objectives • Provide Federally required highway data collection and analysis to state, federal, and local agencies • Provide Geographic Information System (GIS) and Global Positioning (GPS) data collection and analysis, as well as cartographic and other technical services • Develop and administer the State Highway Safety Program • Oversee the web and phone 511 Traveler Information System and the Road Weather Information System Business Data OwnerData Custodian Communities of Interest Primary Data System HAS RWIS 511 Enterprise GeoDB TDP Data Services 1. Collection 2. Quality Assurance 3. Description/Documentation Metadata/Catalog 4. Storage, Access and Security 5. Outreach/Sharing 6. Integration and Value Added Solutions Business Processes Identify Needs and Solutions Budget & Manage Resources Manage Real-Time Data Systems Provide Data and Information Monitor and Report Performance Data Products Accident Reports Traffic Reports HPMS Travel Condition Reports Roadway Inventory GIS Basemap Road Weather Information Seasonal Weight Restriction Decision Information Assigns Use Figure 2.2. ADOT&PF data business plan framework.

the basis for further analysis of the data and information needs at Mn/DOT. Completing the assessment helped Mn/DOT do the following: • Identify data and information priorities to meet user business needs; • Determine the current ability of data and information to meet user business needs; • Determine current and anticipated gaps in data and information; • Identify methods to address current and anticipated gaps in data and information; and • Enhance user access to information on available data sources and stewards. The success of the assessment process will depend upon the commitment of the participants to identify what is working well, so those methods can be repeated with other data programs in the DOT. Likewise, this assessment will highlight areas where improvements are needed, to develop a plan of action to address gaps in the data systems. Instruments for Gathering Feedback Assessing the current state of data management and data programs in a Transportation agency can be a challenging process, depending upon the size of the organization. However, this process can be expedited through the use of structured methods and instruments for gathering feedback from staff across the organization. Guide for Data Management II-41 Data Products Roadway Inventory Asset Information - Pavement - Structure & Bridge - Safety - Drainage - ITS - Roadside - Facilities Work Tracking Traffic & Travel Characteristics Current Travel Conditions Road Event History Safety Information Financial & Resource Management M & O Needs Planned Work Land Development Data Services Business Case Design Acquisition & Updating Quality Assurance Description & Context Storage & Access Security Outreach Catalog SO Business Areas Infrastructure / Maintenance Management Equipment Management Land Development Safety Management Congestion Management Emergency Management Critical Infrastructure Management SO Business Objectives Improve Safety Improve Security Improve Highway Operational Performance Preserve the Infrastructure Achieve Ensure Provide Support Use Provide Feedback for Improvement SO Business Processes Identification of Needs & Solutions Budgeting Work Scheduling and Management Real - Time System Management Providing Traveler Information Monitoring and Reporting Performance Designate Applications RNS ADMS AMS LUPS PMSS LandTrack VaTraffic TAMS TREDS MDSS DACHS TMS EMS RWIS Manage & Consume Structures Coordinates Enable Data Architect Data Coordinator Business Owners Data Custodians Communities of Interest Data Steward Designates Designates Figure 2.3. Virginia data business plan framework.

The best approach for the development and use of these instruments depends upon the size of the agency, and the resources and funding available to develop the instruments. The type of instruments that can be considered for use in the assessment process include: surveys, focus group meetings, data program workshops, and research studies. The intent of each of these instruments is to gain perspective on the quality of data programs within the organization from the viewpoint of the audience, whether the audience is enterprise- based (using surveys) or a more limited audience which includes participants in focus groups and workshops for specific data program areas, or research studies which may assess data pro- gram performance in a specific area such as traffic or pavement. Surveys Surveys can be used to assess how well the data programs and information needs of the agency are being met, to identify gaps in needs, and potential solutions for addressing gaps. Surveys pro- vide an opportunity to reach a wide audience with a quick assessment of how well data programs are performing within the agency. Particular attention should be given to developing a survey instrument which assesses data programs across the organization, if the intent is to develop a data business plan for the entire agency. A more limited survey should be used if the data busi- ness plan being developed is for a limited division or office of the agency. Focus Groups Focus groups offer the opportunity to assess data programs at a more detailed level than sur- veys. Agencies should include data providers and data users of the particular data program(s) in the focus group discussions. The following are some suggestions regarding Focus groups: • Use in-depth discussions which focus on specific areas of data and information needs within the organization; • Develop a list of intended outcomes which are known to all participants, such as a prioritiza- tion list and ranking of needs identified for data programs and action plan recommendations for addressing those needs; • Design to allow for additional pertinent and valuable information to be provided by partici- pants that may not have been previously considered; and • Reach a consensus on the top three to five data issues that can be addressed over a short time- frame by the agency and also identify those issues that may be addressed as part of a long-term data action plan or data business plan. Data Program Workshops Data program workshops can include staff from the Information Technology office of the agency, staff from the business offices who represent the business owners of the data, and other agency staff who represent the data providers and users of the data program(s). Data program workshops are structured to address particular needs identified for a limited group of data pro- II-42 Guide for Target-Setting and Data Management Develop criteria for evaluating programs and instruments for gathering feedback. This may include the use of: • Surveys; • Focus Groups; • Workshops; and • Research Studies.

grams or a single data program. The workshops occur after data program needs have been iden- tified and strategies need to be developed to address the data and information needs using a tech- nology solution. This may or may not include the development of new systems and applications or the enhancement of existing applications. Data program workshops can include the prelimi- nary design of new data applications, or data models, and design for integration of existing data and applications into an enterprise model, which better suits the needs of the agency on a wide- spread level. The outcome of data program workshops can include preliminary architecture and system design for new applications or integration of existing applications within a new frame- work, such as a GIS. Research Studies An agency should consider the use of independent research studies to assess data program per- formance within the agency, when resources are limited to conduct the analysis internally. Some advantages of research studies include the following: • Research studies offer an unbiased assessment of the data programs at the agency; • Research studies can include benchmarking used at other agencies to assess how well similar data programs meet the needs of those agencies; • Research studies can present proposed methods for assessing data programs and addressing potential problems, based on best-practices across multiple agencies in the private and pub- lic sector; and • Research studies can be sized in scope to focus on limited or enterprise solutions to address data and information needs of the organization. Compiling and Analyzing Results Regardless of the feedback instrument used, once the information is gathered on the state of data programs at an agency, the task begins of compiling and analyzing the results. The agency should perform a preliminary and detailed analysis of the results, in order to develop the best possible solution for addressing its most critical needs regarding data programs. Guide for Data Management II-43 Determine gaps in data program needs by analyzing results of data program eval- uation instruments. Preliminary Analysis • Step 1—Compile the raw data from the instrument used. • Step 2—Evaluate the raw data by identifying the data programs which are ranked most criti- cal in supporting business operations. • Step 3—Evaluate whether those programs fully, partially, or do not meet the needs of the agency. • Step 4—Evaluate the gaps in data and information needed as identified by the audience. • Step 5—Evaluate the recommended solutions for addressing the gaps in data and information. • Step 6—Prioritize the recommended solutions. Once the preliminary analysis is completed a more detailed analysis follows. Detailed Analysis • Step 1—Evaluate the results according to: – Needs within core business areas of the organization;

– Needs based on the primary job functions of the audience, within the organization, i.e., senior, mid-level managers, business data stewards, IT data stewards, users of data, and providers of data; and – Needs of data programs to support job functions within specific business areas. • Step 2—Evaluate the recommendations made for addressing the critical needs, by business area. • Step 3—Prioritize the data program needs by business area. • Step 4—Prioritize data program needs across the organization, including the most critical needs identified by the assessment instrument. The prioritization process also includes the fol- lowing additional criteria: – Is the data program used to support performance measures and targets? – Is the program used to meet Federal or state mandates? – Is the program used to support more than one business area? – What are the known and anticipated risks to the agency associated with lack of access to data from the data program? A prioritization matrix should be developed similar to the following example, to identify the top five data programs in terms of these criteria. II-44 Guide for Target-Setting and Data Management Data Program Value Ranking (Essential, Helpful, Not Needed) Addresses Key Performance Measures Used To Meet Federal Mandate Used To Meet State Mandate Used to Support One or More Defined Business Emphasis Areas Risk Level Associated With Data Program Program A Program B Program C … Program Z • Step 5—Prepare a proposed action plan for addressing the needs of the top prioritized data programs. • Step 6—Submit the plan to senior management for consideration. Compiling and analyzing results for each data program at the agency helps to facilitate an enterprise gap analysis process, which ultimately is used to develop a data action plan to address data program needs across the agency. 2.4 Establish Data Governance Programs This section addresses techniques for accomplishing the following success factors: • Establish, update, and enforce polices and procedures to govern data management. • Implement a Data Governance Board or Council to address issues related to development, implementation and use of data programs which are critical to supporting business functions. • Clearly identify the roles/responsibilities of the staff responsible for supporting critical data systems using a Data Governance Manual or other means. • Communicate with stakeholders to sustain support for various programs. Continue to pro- vide outreach to all communities of interest to ensure that all needs are addressed.

• Manage data as an asset in the organization, through policies governing the collection, main- tenance, and use of data. • Develop a business terminology dictionary to align the use of business terms commonly used throughout an organization. This is particularly helpful to staff such as IT professionals who are often responsible for developing applications to meet business needs. • Use data standards to do the following: – Facilitate establishing targets and measures which meet agency goals; – Reduce the cost of multiple data collection efforts and maintenance of duplicate databases. Strive to collect data once and use it many times; and – Facilitate consistent reporting of information. Developing a foundation for data management traditionally relied on policies, standards, and procedures established by an IT division or office. More recently, transportation agencies have instituted a data governing council or board, comprised of senior level managers. This board is generally responsible for establishing the policies and procedures that shall be used in the collec- tion and use of data and information, across the organization, and in support of the agency mis- sion and goals. The governance board is supported by work groups or work teams whose responsibilities include the following: • Providing assistance to the governing board in recommending the development of data prod- ucts to meet business needs; • Recommending procedures to the governing board for standards and procedures regarding collection, maintenance, and use of data programs and products within the agency; and • Recommending the technology tools that may be used to support data management at the agency. The framework in which the governing board and the work teams operate is known as the Data Governance framework. Data governance provides the structure in which a data manage- ment program functions. There are a series of steps involved in developing and implementing data governance within the organization. Step 1—Develop a Data Governance Model An agency should develop a data governance model that best suits the needs of the agency. There is no single data governance framework that meets the needs of every organization. There is flexibility allowed in how the data governance model is used and over what period of time it is implemented. Some agencies have found it beneficial to start with governance on a limited scale, for a particular office or division, while other agencies decide to develop governance on an enterprise level. Guide for Data Management II-45 Develop a Data Governance Model that fits the needs and size of the agency. A standard data governance model is shown in Figure 2.4. The participants within the data governance model all have vital roles in supporting the data governance framework for the organization. More detail and examples related to data gover- nance models is found in Section 4.3 of Volume 1.

Step 2—Determine Roles and Responsibilities Each transportation agency should select the roles and responsibilities for data governance that best suits the needs of the organization. Some of the roles may be combined, depending upon the scale of data governance used at the agency. These roles include the following: • Data Governance Board or Council—Serves as the primary governing body for the manage- ment of data systems. This governing body is usually comprised of senior level managers who have authority to establish policies for the management of data and information on behalf of the agency. • Data Stewards—Individuals responsible for ensuring that the data which is collected, main- tained, and used in the agency is managed according to policies established by the data gov- ernance board or council. • Data Stewardship—Data stewardship is defined as “the formalization of accountability for the management of data resources.” Data stewardship also can be defined from three perspec- tives, similar to the three levels or perspectives of data governance for the agency. The three levels of stewardship can be summarized as follows: – Strategic enterprise level—Data Council; – Tactical level—Data domains or subject matter experts; and – Operational level—Data definers, data producers, data users. • Data Owners—Individuals from the business side of the agency that are responsible for estab- lishing the business requirements for the use of the data in their business area of the agency. They also may approve access to data applications supported by their business area. • Data Custodians—Individuals responsible for the technical support of the data applications, including maintaining data dictionaries, data models, and back-up and recovery procedures for databases. • Data Architects—Individuals who define business requirements for data storage and access services and work closely with IT staff to assist with translation of these business requirements into technology requirements (VDOT Data Business Plan, June, 2008). • Data Users or Communities of Interest—The group of persons or offices who share a com- mon interest as users of a particular data program. These can include persons both internal and external to the agency. The Communities of Interest serve a vital role in any data gover- nance framework by providing a focus for communicating business needs which are sup- ported by data programs. II-46 Guide for Target-Setting and Data Management Strategic Vision, Mission, Goals for Data Agency Data Programs Data Governance Board Data Users and Stakeholders Division(s) Mission(s) and Goals Data Steward and Custodians Figure 2.4. Standard data governance model.

Many of these roles already are being performed by individuals in both the business divisions and Information Technology offices of each agency. The data governance model offers the opportunity to formalize the institutional arrangement between these two entities to facilitate the sharing of data and information throughout the organization. Step 3—Develop a Data Governance Handbook or Manual In addition to defining data governance roles and responsibilities, the agency should develop a data governance handbook or manual to provide a single source of information for all staff on the standards, policies, and procedures regarding the use of data and data programs within the organization. The data governance handbook or manual includes the following components: • Data governance charter, • Agency formal data management policy, • Data governance model diagram used for the agency, • Roles of data governance participants, and • Glossary of terms. Step 4—Develop a Data Catalog A data catalog can be developed to supplement the information provided in the Data Gover- nance Handbook or Manual. The data catalog includes the following components: • List of data programs in the agency; • List of business owners of the data program, with their contact information; • List of data stewards responsible for the data program, with their contact information; and • Instructions for accessing data standards and definitions used with each data program. Step 5—Develop a Business Terms Glossary Agencies should consider developing a business terms glossary, in addition to data dictionar- ies, in order to standardize the use of business terms throughout the agency. It is very important for developers of new data applications to use the appropriate data term related to the correct business term when developing applications to support business operations of the agency. Regardless of the model selected for data governance, and how the agency defines the roles and responsibilities for supporting governance, technology is available to support the data gov- ernance framework, by providing mechanisms for sharing and integration of data across the organization. The next section describes some of the available tools used to enhance data shar- ing and integration. 2.5 Technology for Data Management In addition to the institutional challenges associated with establishing a data management pro- gram for an agency, there also are technology challenges. These challenges impact the ability of the agency to share and integrate data between programs within the agency and to share and inte- grate data from external sources as well. Any data management program should include standards, policies, and procedures for data integration and sharing with internal and external stakeholders. Training for staff also is essen- tial for them to become successful in the use of the tools and procedures which support the data management program. Guide for Data Management II-47

Some of the tools and procedures which can be used to support the data management pro- grams include the following: • Formal data sharing agreements can be used between internal and external offices in order to facilitate the process of sharing data and information. In order for this process to work smoothly, certain standards and communication protocols must be observed as part of the sharing process. These include the use of the following: – Data definitions; – Data file structures; – Formats used for transmission of data; – Frequency of transmission of data updates; – Names of persons/offices responsible for transmitting data updates; – Names of persons/offices responsible for receiving data updates; and – Processes to secure the transmission of confidential data and information. • Business Intelligence (BI) tools also provide the means for allowing easy access to data systems and sharing of information among employees. These tools may include Knowledge Manage- ment systems, GIS systems, dashboards, scorecards, visualization tools, and others, described in more detail in Chapter 4. • Open architecture should be used in the design of application systems in order to provide for future enhancements or integration with other systems, with minimal cost to the agency. • Annual data files should be created to be used for reporting purposes, in order to ensure that consistent answers are provided to stakeholders and decision-makers throughout the year. • Enterprise data warehouses can be used to integrate and standardize the use of data and infor- mation within the agency. Standard reports can be exported to Data Marts from the data ware- house and used for analysis of business processes, including reviewing performance measures and targets associated with data programs. • Hardware such as engine, fuel, and brake condition monitoring systems, GPS, radio frequency identification (RFID) systems, and barcodes helps to gather data from field operations. Cor- poration X and ABC Company use much of this hardware. The next sections describe the processes and tools that are recommended for implementing and maintaining a data management plan for the agency. II-48 Guide for Target-Setting and Data Management Use BI tools to address technology challenges associated with implementing Data Management programs. Data Sharing There are many methods and tools used for sharing of data and information. This section pro- vides guidance on the use of GIS systems, dashboards, and scorecards in a public sector agency. GIS GIS offer one of the best methods for integrating and sharing data. The integration process involves integrating different types of data in a geospatial data model comprised of several cata- logs and tables. Data is then linked to a linear referencing system on a map in order to locate point and linear attribute data. The advantages of a GIS system include the ability to update data in one part of the GIS model, in a particular table, without impacting other data layers in the system. The flexibility in GIS tools also offer a quick way to locate anomalies in data through visuali- zation of the data on a map or using 3-D GIS tools.

All state transportation agencies are now required to use a GIS component, known as a shape- file, for submitting the state’s transportation network data, as part of the annual Highway Per- formance Monitoring System (HPMS) 2010 report. State transportation agencies, which are lagging in the development of GIS systems to meet their business needs, should expedite this process in order to support internal data sharing needs, as well as to comply with Federal and/or state mandates. The process of improving data quality and accuracy of data delivered is greatly enhanced through the use of a GIS system and its associated tools. Dashboards and Scorecards Dashboards and scorecards offer another means for visual display of data in an easily accessi- ble and easy to use format. Some transportation agencies, such as Virginia DOT, have developed dashboards and score- cards for tracking performance measures which assess how well agency programs are perform- ing. The private sector uses dashboards as well; Corporation X has a well-defined one posted to its intranet. The following definitions explain the distinction between a dashboard and a scorecard. In management information systems, a dashboard is an executive information system user inter- face that (similar to an automobile’s dashboard) is designed to be easy to read. For example, a product might obtain information from the local operating system in a com- puter, from one or more applications that may be running, and from one or more remote sites on the Web and present it as though it all came from the same source. Key performance indicators (KPIs) and balanced scorecards are some of the content appro- priate on business dashboards. One of the prominent systems for displaying dashboards is the use of COGNOS®. The balanced scorecard is one of the components that can be displayed on a dashboard. The scorecard reports on how well specific programs are performing based on targets and goals estab- lished which are linked to strategic business objectives. The purpose of the balanced scorecard is to do the following: • Align all members of an organization around common goals and strategies; • Link initiatives to the strategy, making prioritization easier; • Provide feedback to people on key issues—notably, areas where they can have an impact; and • Be an essential decision-making tool for everyone in the organization. The Virginia Department of Transportation (VDOT) provides an excellent example of a dash- board, and their template is recommended by this Guide as a model of how to visually display and implement a dashboard for a transportation agency. Figure 2.5 illustrates the main VDOT dashboard, which can be used to navigate into more detailed areas of the dashboard, in order to view the performance reporting for projects and pro- grams in the agency. These include Engineering, Construction, Maintenance, Operations, Safety, Finance, and Environment. One of the main advantages to using this type of mechanism for sharing of data and infor- mation is that it is easy to use, and is available to anyone interested in the information, whether it is agency senior management, engineers and support staff, or the legislature and the general public. In addition to the use of dashboards, scorecards also present a ranking or score in how well pro- grams are performing, in meeting business needs of the agency. Table 2.2 illustrates a balanced Guide for Data Management II-49

II-50 Guide for Target-Setting and Data Management Figure 2.5. VDOT dashboard. Perspective Strategic Objective Measure Target Actual Comment Customer Achieve customer Outcomes. Number of high- priority issues resolved. 60 30 Need improvement, investigate process for resolving high-priority issues. Improve customer satisfaction. Percent of customers rating service very good or excellent. 80% 80% Right on Target. Finance Manage expenses. Percent increase/ decrease in annual budget. 1.5 5% Reduced expenses due to budget cuts. Maximize revenue. Percent increase/ decrease revenue derived from grants. 5% 13% Good progress. Internal Process Build effective partnerships. Number of projects involving one or more partners. 25 10 Based on the projects to date with one or more partners. Learning and Growth Retain knowledgeable staff. Employee retention rate. 95% 75% Need to monitor. Table 2.2. Hennepin county scorecard.

scorecard used in Hennepin County, Minnesota, to monitor programs in the Public Works Department of the county. Both dashboards and scorecards are an effective means for sharing of data and information as illustrated in Figure 2.5 and Table 2.2. File Exchange Protocols Electronic data interchange (EDI) has become a common technology for file exchange. DIY Corporation shares data with its trading partners via the use of automated shipping notifications (ASNs). MNC uses EDI 210 transaction invoice records to interface with its suppliers of trans- portation services. Knowledge Management Transportation agencies should consider using a Knowledge Management (KM) system to strengthen and provide support for their data management programs. A knowledge manage- ment system is used to document a wide range of activities, including work processes, which may be solely known to certain individuals. This knowledge, which can be referred to as corporate knowledge is generally considered critical for maintaining business operations. In addition to corporate knowledge, other types of knowledge may be embedded as part of the routine processes and practices of the organization. It is important that this knowledge and these processes are doc- umented for use by future employees and decision-makers. The benefits of using KM systems include the following: • KM systems can be used to archive lessons learned which are invaluable when considering future investments in data programs; • KM systems identify and document the employee networks which are involved in the trans- fer of information within and between data programs; • KM systems offers flexibility in the transfer and sharing of data in many different formats, including text, PDF, and digital images; • The training required in using a KM system is minimal, and they also provide easy to use search and retrieval functions; and • The cost of implementing a KM system is affordable, and the estimated benefits derived can be used to justify the cost. Agencies also should consider implementing a KM office to oversee the knowledge manage- ment functions of the agency. Depending upon the size of the agency or offices involved, it may be more feasible to implement a section within an office that is responsible for knowledge man- agement activities at that division or office. Training The need for training of staff cannot be underestimated as an agency begins the process of implementing its data management and data governance programs. It is normal to expect that there may be some degree of uncertainty on the part of staff who do not understand how their responsibilities may change as a result of implementing new technology, standards, and proce- dures. Communication is the key to alleviating these concerns. It is extremely important that any agency considering the options recommended in this guide prepare the staff and the audience of stakeholders and users for what is expected during and after implementation of new policies, standards, and procedures. This can be accomplished through on-site meetings, webinars, and on-line/or printed brochures which include Frequently Asked Questions (FAQs) explaining how Guide for Data Management II-51

such initiatives as a data management and data governance program will be implemented and used at the agency. Bidding, Auctions, and Cost Management Solutions A variety of bidding, auctions, and cost management solutions help to manage disaggregated data in decentralized locations. Bidding and auction software helps to keep costs low when nego- tiating an agreement, and sometimes on the spot market as well. MNC Corporation uses this software when it is time to contract with its vendors. II-52 Guide for Target-Setting and Data Management Invest in the staff through training opportunities. This will support the ROI for Data Management programs at the agency. 2.6 Linking Data to Planning, Performance Measures, and Target-Setting Processes The final step is to fully integrate an operational data management process with the agency performance measures and target-setting process. The success factors to achieving this final step are the following: • Use a hybrid approach employing modeling and benchmarking to establish agency targets and performance measures. • Do not use a one size fits all approach in establishing performance measures and targets. Use the correct metrics for making decisions. Focus on continuous improvement by revising/ adding new metrics as needed. • Link the performance measures and targets for a program to budget allocations, improving participation by staff in supporting the performance measures and targets. The performance measure and target-setting process also can be used to motivate employees by linking their performance plans to objectives identified in specific performance measures and targets. • Allow the DOT transportation planning staff routine access to other planning offices (regional, district, etc.) and technical resources available in the agency. This strongly enhances a performance-based management process. • Reward business areas which consistently meet targets and goals. Consistent achievement in meeting targets is a powerful motivator for behavior—success breeds success. • Use external data sources, such as environmental, historic, and other planning agencies for GIS data layers to improve the data used for the performance measurement process when funds are limited to collect this data using internal resources. • Utilize software that is procured or developed internally to automate as much of the perfor- mance measurement process as possible. This will allow for more time devoted to the analy- sis of the performance results. • Revise or stop using targets if performance data are not easily obtainable when a performance target is used. • Programs which do not have a direct link between that program or project and performance should not be funded. • Identify business units responsible for maintaining current metadata about each performance measure. This facilitates the analysis required for user requested data and information system changes and enhancements. • Include objectives pertaining to resource allocation in the agency Business Plan. The current Business Plan at MDTA, for example, has three separate objectives related to resource alloca-

tion. These include System Preservation, Implementing and Asset Management System, and Integrating MDTA’s financial system with other systems. • Use external data sharing agreements to obtain data for performance measures that the agency does not have. For example, MDTA collaborates with other agencies for several measures that it needs additional data for or does not have the necessary equipment to monitor itself. • Establish performance targets through a streamlined process and revisit and revise (as needed) periodically. • Incorporate customer satisfaction as a measure in setting performance targets. • Utilize incentives to facilitate meeting performance objectives, including awarding bonuses based upon job performance and using quantitative objectives embedded in professional employees’ annual objectives. • Arrange performance measures in a hierarchical order, allowing an agency to translate strate- gic goals/objectives into operational goals/objectives for each department. The U.S. DOT fol- lows this approach among its various administrations (e.g., FHWA and FTA), allowing it to provide a performance budget that can be related to actual and planned accomplishments for each department. This same scenario would apply to a state DOT, with several divisions, dis- tricts, and/or independent offices. The performance in each area then becomes a key basis of resource allocation and budgeting. A step-by-step guide is not provided for this final step—the requirement approaches will vary significantly across agencies. This Guide is designed to provide helpful advice related to all aspects of data management to support performance measures. It is ultimately up to the trans- portation agency to take full advantage of the benefits that a fully functional data management process will offer for decision-making in a transportation environment. It is presumed that agencies are directly interested in linking their data programs to goals and objectives in order that the data programs will support decision-making, including resource allo- cation and project selection within the agency. Guide for Data Management II-53

Next: Abbreviations used without definitions in TRB publications »
Target-Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target Setting and Data Management Get This Book
×
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

TRB’s National Cooperative Highway Research Program (NCHRP) Report 666: Target Setting Methods and Data Management to Support Performance-Based Resource Allocation by Transportation Agencies - Volume I: Research Report, and Volume II: Guide for Target-Setting and Data Management provides a framework and specific guidance for setting performance targets and for ensuring that appropriate data are available to support performance-based decision-making.

Volume III to this report was published separately in an electronic-only format as NCHRP Web-Only Document 154. Volume III includes case studies of organizations investigated in the research used to develop NCHRP Report 666.

NCHRP Report 706: Uses of Risk Management and Data Management to Support Target-Setting for Performance-Based Resource Allocation by Transportation Agencies was released in 2011 and supplements NCHRP Report 666. NCHRP Report 706 describes how risk management and data management may be used by transportation agencies to support management target-setting for performance based resource allocation.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!