Cover Image

Not for Sale

View/Hide Left Panel
Click for next page ( 87

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement

Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 86
II-16 Guide for Target-Setting and Data Management is a long-term business, with performance typically improved over the long term, but agen- cies operate in the context of short-term politics, which can have a very strong impact on performance-based management. Internal Support/Culture A common theme among many agencies with more developed PBRA processes is internal sup- port and an inside "champion" at a high level, such as a top-level executive in the agency (MTC, Hennepin County, Mn/DOT). The champions guide the development and implementation of Performance Management, including provisions which help assure that the new way of doing business transcends administrations and individual staff. As a result, these managers understand the importance of PBRA and are willing to ensure the process has the necessary resources to pro- ceed, such as funding, staff, data, and tools. In companies with a competitive spirit, the attitude about the numbers is more important than the numbers themselves. In companies with an analytical culture, data collection and analysis is revered and viewed as synergistic with continuous improvement. Corporation X has both budget targets and continuous improvement initiatives, and meeting the operational targets in the continuous improvement programs ensures hitting the budget targets. Leading agencies facilitate a sense of ownership and responsibility for performance measure- ment and integrate the practice into an overall agency culture of performance (Maryland DOT SHA). One way this can be achieved is by designating a performance measure "lead" who is respon- sible for maintaining and reporting data for a particular measure and ensuring data accuracy. Step 3--Select the Appropriate Approaches for Target-Setting There is a wide range of agency implementation of PBRA processes, and an equally as diverse range of implementation approaches for target-setting as an element of PBRA. The target-setting approach is determined largely by the factors influencing target-setting and evaluated in Task 2; the appropriate approaches can be selected with consideration of these factors as shown at the end of this section (Figure 1.2). In practice most agencies will use a hybrid approach (different approaches for different mea- sures but also multiple approaches for a single measure). For example, an agency could use mod- eling combined with customer feedback to arrive at a target that is both analytically grounded (to ensure a connection with predicted outcomes based on resources and existing plans) and sat- isfactory to the public and stakeholders. This often helps to mitigate risks inherent in any single approach. Edict In the private sector, this approach also is called "Ready-Fire-Aim:" just state the goal and have everybody try to hit it (MNC). The underlying principle of this approach is that success in hitting the target is entirely a function of motivation and execution and that planning is a relatively minor part of reaching the target. The advantage of the Edict approach is that the target is unequivocal and well-understood throughout the organization. The challenge is that the approach is not

OCR for page 86
Guide for Target-Setting II-17 Figure 1.2. Selecting a target-setting approach based on influencing factors. inclusive or consultative; it is more characteristic of old-fashioned hierarchical leadership. For example, the choice of metrics and targets is made by senior management and is not sub- ject to discussion. Staff members are tasked with developing a transportation investment plan to meet the target and conducting modeling and technical analysis needed to demonstrate attainment of the target under a future funding scenario. While such an approach is some- times used by elected officials or other decision-makers to prescribe a target for an agency, this approach by itself is usually unsuitable for most public sector organizations where trans- parency is expected. Expert Opinion In many cases, transportation agencies develop targets through an internal or external consensus-based planning process as part of a more comprehensive PBRA exercise. Typically, the use of specific targets derived from such a process is seen in agencies with a more sophisticated and well-developed PBRA system that has developed over several iterations. Staff resources and time permitting, such an approach can lead to very meaningful and effective targets within an agency's PBRA process. Such a process is usually informed by internal staff analysis, but ultimately approved by an agency's executive management and stakeholder committees.

OCR for page 86
II-18 Guide for Target-Setting and Data Management The MTC sets overarching goals and strategies for long-range planning at the executive level, a stakeholder subcommittee of the MTC Planning Committee derives the measures and targets with staff input, and the MTC Planning Committee votes on and approves measures and targets. This approach leverages the technical, practical, and local knowledge of members of the agency and stakeholders within the agency's jurisdiction. Not only does this help to ensure that targets will reflect local and stakeholder priorities but also that they will better reflect "on the ground" reality. Customer Feedback Under this approach, direct feedback on system performance and objectives for transporta- tion investment are gathered from the transportation system user through a variety of survey and outreach methods. This feedback is then used by the transportation agency staff to develop specific measures and targets that are closely aligned with the needs of the traveling public (the "customer"). Those agencies that have taken a clear customer-service approach within the resource allocation decision-making process understand the need to use targets that communicate to the system user (the "customer") the return on their investment. Customer satisfaction is a fundamental aspect of performance for these organizations and permeates the process for how potential investments are evaluated and selected to receive funding. Agencies can use dozens of different types of outreach tools and then analyze the customer input for trends and priorities (Coral Springs, Florida). Tools include annual surveys, public hearings, blogs, regular visioning exercises and focus group discussions, a complaint tracking system, and employee surveys. Selected processes should be extensive, formal, iterative, and continuous. Hennepin County, Minnesota, utilizes a balanced scorecard (BSC) with the "customer" as one of the four perspectives that the approach is viewed from as part of the PBRA process. In the trans- portation service area, a number of specific targets already exist and include targets related to bridge and pavement sufficiency ratings, reducing crash rates, completion of the Bicycle System Plan, and project delivery standards. The target-setting process is still informed by and supported by transportation practitioners that provide the appropriate context for establishing targets (e.g., what percentage of crashes might be expected given certain funding levels, or how much speed may decrease as a result of traffic calming measures), but ultimately processes oriented towards the customer are driven by the customer perception of what needs to be improved and by how much; the way in which measures are reported reflects this. Essentially, this process is almost always part of a hybrid approach supported by Expert Opinion or Modeling. Benchmarking Benchmarking as a target-setting approach provides a transportation agency with the means to establish targets in a relatively quick and efficient manner that can be realistically achieved. Under this approach, criteria should be set for peer group selection and analysis, such as sim- ilar investment approaches, jurisdiction, span of control, and agency size. In terms of stake- holder expectations, it is often appropriate to select peers that also excel in the specific goal

OCR for page 86
Guide for Target-Setting II-19 areas being benchmarked against. Once the peer group is set, practitioners should review each state's performance measures and targets and the degree to which those states are achieving their targets. The comparison among states will help guide the final determination of targets within selected performance measure categories. Agencies sometimes benchmark against other agencies in the region (MDOT, SHA). National datasets can often provide good compilations of data for benchmarking. The AASHTO Stand- ing Committee on Quality, recently renamed the Standing Committee on Performance Manage- ment, conducted a study in 2007 which identified best practices for highway project delivery times and cost. The committee also has identified best practices for smooth pavements. The Austroads National Performance Indicators (NPI) system includes dozens of indicators in 11 broad group- ings, covering safety, asset management, environmental impacts, system capacity, user satisfaction, and project management, among other things. The NPI data system presents consistent and com- parable data across a transportation system managed by nine separate agencies in two countries, allowing unprecedented benchmarking possibilities. In the private sector, the following are the three basic varieties of benchmarking: Best-in-Company benchmarking fosters competition between operating units on the basis of the key metrics and best-in-company performance levels (DIY Company). Best-in-Industry benchmarking analyzes the performance of companies in the same indus- try or segment and highlights the best in the group as the benchmark, even if its activities are not directly comparable to the subject company. Best-in-Class benchmarking analyzes the performance of a broad range of entities, including some with unrelated activities, and highlights the best in the group as the benchmark, even if it is in a different industry than the subject company. Additionally, strategic benchmarking can identify similar companies' strategies as a basis for setting one's own strategy. Organizational benchmarking can be either qualitative or quantitative. One common form of organizational benchmarking is measuring staff levels used to service a given level of activity. Finally, benchmarking within the public sector also can include benchmarking against fore- casted targets for other agencies, as opposed to just benchmarking against actual best practice per- formance. The MTC uses several environmental targets that are based on California state goals, but they are not required for the MPO. Modeling Both top-down and bottom-up modeling are used to set targets in many companies. Top- down modeling most commonly drives the target by high-level requirements. Top-down modeling determines the strategies or funding needed to achieve the target; Bottom-up mod- eling determines what level of performance is possible, and then uses that to calculate the expected target (MLIT). The exact use of modeling depends to some extent on the way in which an agency is utilizing PBRA. For what part of the investment process is the agency setting targets? For example, are they annual financial targets, long-term targets for a long-range transportation plan, or mid-term targets for a package of projects and programs to be included in a TIP? Table 1.2 summarizes different tools used for modeling performance and estimating targets. These range from simple interpretations of historical data, straight-line projections (Mn/DOT), and analysis of research results; to more complex (and expensive) travel demand and eco- nomic impact models. Corporation X uses a sophisticated operations and financial model that

OCR for page 86
II-20 Guide for Target-Setting and Data Management Table 1.2. Tools for modeling performance and estimating targets. Type of Tool Sample Tools Advantage Disadvantage Type of Outputs Historical data Microsoft Excel Easy to use. Static. Line plots. spreadsheets, Enterprise Does not consider Resource Planning future growth and (ERP) systems. change. Projected trends Microsoft Excel Easy to use. Static. Regressions. spreadsheets, SAS, Based almost entirely STATA. on historical performance. Travel demand model Dynamic and allows for Data and resource- Link and system testing of projects and intensive. volumes, speeds. scenarios. Can be over-relied upon. Postprocessors IDAS, TREDIS. Provide numerous Require a travel Travel time, delay, metrics that are demand model. crashes, emissions, normally difficult to employment, GDP, calculate at system Value-added. level. Economic HERS/ST, Input data already Static. B/C, delay, crashes, AssetManager NT. required by states Relies entirely on highway deficiencies. Tradeoffs. HPMS input. Economic impact tool REMI, IMPLAN. Broader, societal Requires reliable inputs. Employment, GDP, measures understood by Easily misinterpreted. Value-added. all. Research Microsoft Excel, Emerging areas. May not be based on Benchmarks. Access, or similar actual practice. Financial Model. Management systems PMS, BMS (PONTIS, Tradeoffs. Siloed. Prioritization, needs. Arivu, StreetSaver). Database. calculates the impact of seven operational variables on three key business unit and corporate financial metrics. Many agencies have found innovative ways to incorporate performance-based processes and targets into their planning processes and duties, supported by modeling. The MTC uses its own StreetSaver PMS to calculate preventative maintenance funding targets for its local jurisdictions; the ratio of "actual versus targeted" determines the jurisdiction's perfor- mance score and is a factor in calculating the amount of funding that will be allocated to that jurisdiction. Modeling is often used by agencies to evaluate progress towards targets, even when modeling itself was not used to establish the target itself. Project and policy scenarios can be tested using travel demand models, combined with other postprocessing tools to include metrics such as benefit/cost, to evaluate their relative contribution to progress towards targets. Section 3.4 of Volume I provides an in-depth discussion of the role of economic models and management sys- tems in target-setting and tradeoff analysis. Summary of Approaches Each agency or organization must select the appropriate approach or approaches for its cir- cumstances. Figure 1.2 summarizes how various factors influence the types of approaches an agency may choose; Table 1.3 summarizes the advantages and risks of each approach.