Click for next page ( 18

The National Academies of Sciences, Engineering, and Medicine
500 Fifth St. N.W. | Washington, D.C. 20001

Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement

Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 17
17 Goals/Objectives Program Development System Performance Measures to Monitor Progress Component I on Meeting Goals/Objectives Performance Targets for Planning/Programming Given Resource Availability Program Development and Project Selection Specific Set of Programs and Projects with a Defined Budget, Schedule, Scope Project Delivery Component II Performance Measures to Track Delivery Program/Project Implementation System Monitoring Component III and Reporting Result = System Performance Source: Hendren, Neumann, and Pickrell, 2004. Figure 2.3. Developing performance-based program and project delivery. The focus also changes from considerations of system per- Literature Review formance (mobility, safety, etc.) to organizational performance (project delivery, quality, etc.). The literature review for this effort focused on materials rel- Based on this review and, in following the principles above, evant to development of performance measures to support an initial draft framework was developed for the SHRP 2 C02 capacity decision making. It considered measures relevant to project. This initial framework had two basic dimensions the stages of the planning and project development process (Table 2.1): from long-range planning through environmental review. It primarily considers measures of physical impacts (i.e., amount The stage of project development (across the columns); and of congestion, level of environmental harm or benefit), and The factors considered such as mobility, safety, environment, not measures that address process and project delivery. Five economics, and community (down the rows). key subject areas were reviewed: These broad dimensions were adapted and refined as 1. General Use of Performance Measures by Transportation described in chapter 3. Agencies;

OCR for page 17
18 Project Development Phases Relevant to SHRP 2 C02 Phases Potentially Relevant Phases With Limited Relevance Long-Range Preprogram Environmental Monitoring and Factors Planning Studies Programming Review Design Permitting Construction Operations Objective Identify system needs Specify projects/ Identify mix of Select preferred Specify project Permits from Construct Operate the and projects alternatives projects to be alternative completely resource capital system in constructed agencies projects real time Mobility Measures that identify Detailed project Select projects Measures of Work zone and Evaluate mobility and prioritize needs analysis into program mobility reliability impacts to meet needs impacts issues Safety Measures that identify Detailed project Select projects Detailed measures Work zone Evaluate safety and prioritize needs analysis into program of safety issues impacts to meet needs impacts Environment Early issue overview More detailed Criteria avoid Detailed Use of environ- Permit delivery Environmental (check boxes) overview, but projects with environmental mental best monitoring still summary fatal flaws impact practices Economics Economic development/ Potential impact measures selection criteria Community Early overview of issues Criteria avoid Detailed measures Use of public (check boxes) in projects with of community involvement regional/corridor negative impacts best practices studies impacts Other Reliability Project delivery Project delivery Operational (let on time, (on time/on measures of on budget) budget) reliability Table 2.1. Initial Performance Measures Framework Issues

OCR for page 17
19 2. Performance Measures for Transportation System En- communicate with external audiences, and manage towards hancement; strategic outcomes. A state DOT's strategic mission, vision, 3. Performance Measures for Environmental Stewardship; goals, and objectives are tied to day-to-day activities via regular 4. Performance Measures for Community Enhancement; and review of performance results in core DOT business functions 5. Performance Measures for Economic Impacts and Devel- such as pavement and bridge preservation, transportation opment. safety, traffic operations, and infrastructure maintenance. Performance-based management has three essential and The first subject area describes the state of the practice of interconnected components: program development, project transportation performance measurement. It reviews the most delivery, and system monitoring and reporting: useful literature and describes the best practices and lessons learned as performance measurement has matured in the trans- 1. Program Development Program Development typically portation industry. The following four subject areas are tied begins with establishing agency goals and objectives that to sections of the performance measurement framework, and are in turn monitored through performance measures. are discussed in the relevant section of the report that addresses Taking into account resource constraints, performance the specific issue (chapters 4 through 8). targets are set and projects and programs are identified and selected based on performance criteria aimed to lead a transportation agency toward its goals/objectives. For Performance Management example, a state DOT could identify the goal "preserve No state DOT conducts highway capacity decision making the existing system" with the related performance mea- in a vacuum. The performance measurement framework sure "percentage of highway miles with acceptable pave- described in this section for supporting highway capacity ment condition." In turn, a project selection (or program decision making is envisioned as part of a broader agencywide budgeting) criterion would be its estimated impact on performance management system of the kind that some state highway pavement condition. The relationship between DOTs are adopting. Performance management is a business performance targets and project selection is an iterative process that links agency goals and objectives to resources process based on changing needs, available resources, and and results. Figure 2.4 provides two examples of performance political support. measure categories often used in making highway capacity 2. Program Delivery Program Delivery begins when a set investment decisions safety and mobility and shows how of preselected projects are "passed off" to the delivery team. performance management is used to link goals and outcomes A performance-based process uses a set of measures to via use of measures. evaluate and monitor project implementation (e.g., per- With a comprehensive performance management system in centage of construction contracts completed on time). place, DOTs use performance measures systematically across Figure 2.2 illustrates not only the separation between the a full range of core business functions to support decisions, selection and delivery of projects and programs, but also Safety Mobility Ensure High Standards of Provide for Efficient Movement Goal Safety in the System of People and Goods Reduce Rate of Motor Decrease Travel Times Objective Vehicle Crashes for Commuting Performance Crashes per VMT Hours-of-Delay Measures Crashes per Capita Travel Time Index Performance Reduce Crashes per VMT Reduce Delay by Two Percent Target by One Percent per Year per Year Travel Time Index = 1.25 Figure 2.4. Example measures/targets for highway capacity investments.

OCR for page 17
20 that there are two distinct groups of performance measures. Transportation practitioners in both the private and pub- One set of measures relates to project selection and is lic sectors have worked diligently to achieve the current linked to agency goals and objectives, while the other set degree of performance measurement competency. Numer- focuses on delivering projects. ous peer exchanges and three international invitation-only 3. System Monitoring and Reporting System Monitoring conferences have been devoted to the subject. The FHWA and Reporting measures the changes that occur due to maintains a web-based Performance Measurement Exchange, implemented projects and programs. It is the delivery of a thorough and well-maintained list of resources, including projects that produces the "result", i.e., system performance. discussion boards, citations and links to journal articles, Thus, this component of the process indicates whether the reports and studies, and a directory of practitioners (Perfor- intended goals and objectives of the project have been met mance Measurement Exchange, 2007). Furthermore, literature (e.g., reduction in travel delays). on the use of performance measures by transportation agen- cies is extensive. It documents the wide range of activities in The outcomes of performance-based management include which measures are relevant and informative. Several efforts more efficient distribution of limited resources and increased comprehensively review the history, development, state of the accountability and credibility in government. practice, best practices, and recent trends in performance- based planning (Cambridge Systematics, Inc., 2000; Hendren et al., 2005; Pickrell and Neumann, 2001; Poister, 1997; Pad- General Use of Performance Measures gett, 2006). Strategic Performance Measures for State Depart- by Transportation Agencies ments of Transportation: A Handbook for CEOs and Executives Performance-based decision making has increasingly been (TransTech Management Inc., 2003) describes the core areas used by transportation agencies over the past 10 years in a in which most DOTs use measures, and the three primary reasons they use measures: communication, management, range of contexts and applications. Defined as a systemic, and decision making. NCHRP Report 551: Performance Mea- ongoing process integrated into an agency's planning, manage- sures and Targets for Transportation Asset Management reviews ment, and decision-making activities (Pickrell and Neumann, the literature and addresses several key questions: 2001), it involves a continuous effort of monitoring and feed- back that improves decision-making capabilities over time. What criteria are used by agencies to select performance Though transportation agencies have employed performance measures? measures since the 1950s, the most dramatic evolution of the How are current performance measurement frameworks state of the practice has occurred in the last two decades. structured, and what kinds of measures do they include? Improvements have been driven by: How are performance measures being used to gauge the impacts of transportation investments, support resource Demands from the public and elected officials for increased allocation and utilization decisions, and assess agency per- accountability and performance; formance in program delivery and cost-effectiveness? Emergence of management systems for pavement, bridges, How are measures being tailored for different levels of trans- and congestion; portation organizations? Strong leadership within the agencies themselves that bor- How are measures being used to communicate program rowed from private sector-driven initiatives like "Six Sigma" status both internally and externally? and "Baldridge Awards"; and Recognition that the decision-making environment within NCHRP Report 446: A Guidebook for Performance-Based which agencies operate has become more complex (Poister, Transportation Planning provides practitioners an extensive 2005; Larson, 2005; Bremmer et al., 2005). library of measures organized by the following categories: Examining the use of performance measures by transporta- Accessibility; tion agencies is particularly informative. The Government Per- Mobility; formance Project (GPP), a nonpartisan, independent research Economic development; program of the Pew Center on the States, acknowledges many Quality of life; DOTs are leaders in their use of performance measurement and Environmental and resource conservation; can serve as models for other state agencies (Government Per- Safety; formance Project, 2005). Federal transportation agencies are Operation efficiency; often asked to pilot performance measure processes, and state System preservation; and DOTs are often asked to help other agencies with performance- Measures relevant to multiple goal categories (Cambridge based initiatives (Poister, 2005). Systematics, Inc., 2000).

OCR for page 17
21 Key Findings measures become focused on the "vital few," and technical capabilities advance. Performance-based processes should The extensive literature on transportation performance mea- inform the decision-making process, not replace it, and initial sures points to several general trends. impacts on final decisions may be subtle. Decision makers Performance measures should be driven by strategically may initially be slow to accept performance-based recom- aligned goals and objectives. Performance measures should mendations of staff, but they will adopt the additional infor- be identified in response to goals and objectives and not the mation into their decision-making process at varying rates as other way around. An agency's goals should reflect the most agencywide support for the approach builds. Staff should not important aspects of what it wishes to accomplish. Perfor- give up as providing better quality data from performance mance measures are the primary means of assessing how suc- measuring efforts can only improve the decision-making cessful an agency is in accomplishing its goals. Therefore, it process (Pickrell and Neumann, 2001). should be clear what goal(s) each measure illustrates advance- Performance measurement programs are most effective ment of. Failure to properly align measures with goals and when integrated throughout an organization. The linkage objectives can result in tracking measures that have little to do between program development and project delivery is a vital with performance of the organization or transportation sys- tem (Pickrell and Neumann, 2001; Poister et al., 2004; Kittelson component to mature, integrated, performance-based method- & Associates, Inc. et al., 2003). ologies. Few state transportation agencies have implemented Input, output, and outcome measures should all be both, and fewer have linked the two approaches together. In included in performance measurement. Many of the mea- isolation, a performance-based program development process sures used to monitor systems operations are derived from may identify the best projects to fund but may not guard those developed in the 1950s, and these measures reflect the against excessive scope creep, schedule slippage, or cost esca- values of that time, which have evolved significantly. As nega- lation. Conversely, a performance-based delivery process in tive impacts from transportation and other infrastructure isolation may result in efficient delivery of a program that investments have become apparent, mainstream concerns includes marginal projects. By executing both performance- have changed to consider a wider range of ways in which based program development and project delivery, the most transportation affects our communities (Meyer, 2001). The effective set of projects is selected and implemented efficiently type of measures considered for transportation planning has (Hendren et al., 2005; Padgett, 2006). grown to include not only those that consider input (time, Performance measurement reporting should be appro- capital, resources) and output (speed, throughput, conges- priately tailored to intended audiences. The informational tion), but also those that consider outcomes to communities needs of technical staff, decision makers, and the public are and the environment (Poister, 1997; Poister and Van Slyke, different, and presentation and depth of reporting should 2001). Consideration of both output and outcome measures reflect these needs (Shaw, 2003; Cambridge Systematics, Inc., reflect the differences in perspectives of those who manage 2006; Poister, 2005). TCRP Report 88, A Guidebook for Devel- the system and those who use it (Shaw, 2003; Kittelson & oping a Transit Performance-Measurement System highlights Associates, Inc. et al., 2003). this diversity, noting the difference in perspective of four Performance measurement efforts should concentrate different transit stakeholders: the customer, community, on the "vital few." Agencies must use simple, understand- agency, and driver. In addition to delivering the right data able measures and avoid attempting to measure everything to the right people, reporting should be as simple and con- (TransTech Management, Inc., 2006; Larson, 2005; Kassof, sistent as possible. Aside from providing clarity for decision 2001). Providing excessive or redundant measures may over- makers, simplicity and regularity facilitate process improve- whelm the end user and obscure key drivers of service quality. ment of each iteration of the performance-based process As cited in TCRP Report 88, Brown (1996) describes this as (Larson, 2005). choosing between "the vital few measures and the trivial many," Successful performance measurement programs require and suggests an upper limit of 20 measures. Some of the desired high-level buy-in. A performance-based approach must have "vital few" may not be available during the first iteration of widespread and deep-rooted support to withstand significant a performance-based process. This deficiency should be changes in leadership. Five key stakeholder groups must accept addressed in future efforts (Larson, 2005). a program for it to have long-term viability: Early attempts at performance measurement should emphasize process as well as results. Management and staff Agency management; should set realistic expectations about first iteration results as Agency staff; performance-based planning is an inherently incremental Customers; process. The implementation strategy will evolve over time as Agency's governing body; and stakeholder and leadership buy-in improve, performance Senior contractors (Kittelson & Associates, Inc. et al., 2003).