National Academies Press: OpenBook

Performance Measurement Framework for Highway Capacity Decision Making (2009)

Chapter: CHAPTER 2 - Background Research

« Previous: CHAPTER 1 - Introduction and Background
Page 15
Suggested Citation:"CHAPTER 2 - Background Research." National Academies of Sciences, Engineering, and Medicine. 2009. Performance Measurement Framework for Highway Capacity Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/14255.
×
Page 15
Page 16
Suggested Citation:"CHAPTER 2 - Background Research." National Academies of Sciences, Engineering, and Medicine. 2009. Performance Measurement Framework for Highway Capacity Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/14255.
×
Page 16
Page 17
Suggested Citation:"CHAPTER 2 - Background Research." National Academies of Sciences, Engineering, and Medicine. 2009. Performance Measurement Framework for Highway Capacity Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/14255.
×
Page 17
Page 18
Suggested Citation:"CHAPTER 2 - Background Research." National Academies of Sciences, Engineering, and Medicine. 2009. Performance Measurement Framework for Highway Capacity Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/14255.
×
Page 18
Page 19
Suggested Citation:"CHAPTER 2 - Background Research." National Academies of Sciences, Engineering, and Medicine. 2009. Performance Measurement Framework for Highway Capacity Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/14255.
×
Page 19
Page 20
Suggested Citation:"CHAPTER 2 - Background Research." National Academies of Sciences, Engineering, and Medicine. 2009. Performance Measurement Framework for Highway Capacity Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/14255.
×
Page 20
Page 21
Suggested Citation:"CHAPTER 2 - Background Research." National Academies of Sciences, Engineering, and Medicine. 2009. Performance Measurement Framework for Highway Capacity Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/14255.
×
Page 21
Page 22
Suggested Citation:"CHAPTER 2 - Background Research." National Academies of Sciences, Engineering, and Medicine. 2009. Performance Measurement Framework for Highway Capacity Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/14255.
×
Page 22
Page 23
Suggested Citation:"CHAPTER 2 - Background Research." National Academies of Sciences, Engineering, and Medicine. 2009. Performance Measurement Framework for Highway Capacity Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/14255.
×
Page 23
Page 24
Suggested Citation:"CHAPTER 2 - Background Research." National Academies of Sciences, Engineering, and Medicine. 2009. Performance Measurement Framework for Highway Capacity Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/14255.
×
Page 24
Page 25
Suggested Citation:"CHAPTER 2 - Background Research." National Academies of Sciences, Engineering, and Medicine. 2009. Performance Measurement Framework for Highway Capacity Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/14255.
×
Page 25
Page 26
Suggested Citation:"CHAPTER 2 - Background Research." National Academies of Sciences, Engineering, and Medicine. 2009. Performance Measurement Framework for Highway Capacity Decision Making. Washington, DC: The National Academies Press. doi: 10.17226/14255.
×
Page 26

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

15C H A P T E R 2 Background ResearchThe objective of the SHRP 2 C02 effort was to develop a com- prehensive performance measurement framework that would support collaborative decision making. To produce the frame- work, the research team took both a broad look at perfor- mance measurement and management and a more focused look at the application of specific performance measures in practice. The supporting research for the SHRP 2 effort was as follows: • Development of the overall performance measurement framework, based on the broad application of performance management at transportation agencies in the United States; • A review of the literature on performance measurement, with a focus on ‘nontraditional’ areas such as the environ- ment, community, and economics; • Interviews with transportation agencies to determine the extent to which they are using performance measures in var- ious areas identified in the literature; and • Targeted case studies to identify performance measures and applications at specific transportation and other agencies. Each of these efforts is described in this section. Performance Measurement Framework Many research efforts build a framework or organizational structure deductively, after conducting the research. Because the SHRP 2 C02 project covers such a wide range of efforts undertaken by state DOTs, it was useful to develop a frame- work up front to help organize the research itself. This section reports on the efforts to develop the initial framework, which was used as a straw man and modified as appropriate based on research findings.Framework Development Principles To ensure that the performance measurement framework met the objectives of the SHRP 2 effort, a set of framework devel- opment principles was adopted. The criteria used included: • The framework needs to help identify performance meas- ures for use at several levels – from initial planning and analysis to program development and into environmental assessment. • The relevant project development stages to which the frame- work applies should be consistent with those identified by the SHRP 2 C01 collaborative decision-making project (Figure 2.1). • The framework should help establish the specific issues or factors to be considered at various project development stages. These categories will vary from one transportation agency to another, but the framework should help agen- cies identify what the appropriate categories are for their consideration. Existing Performance Measure Frameworks Numerous state DOTs, as well as other transportation and nontransportation agencies, have developed performance measure frameworks that help shape their programs. As part of the early framework development, several of these existing frameworks were reviewed. Two examples of existing frameworks were particularly rel- evant for the SHRP 2 C02 effort. Figure 2.2 presents an exam- ple from the Florida Department of Transportation that clearly identifies how performance measures are intended to work through multiple layers and products of an organization. The measures are tied to specific levels of generality (from policy to project) and tied to specific products (long-range plan, short- range plan, projects, etc.). However, the framework does not specify how various issues and concerns are intended to be

16KDP Systems Planning Pre-program studies Programming good Environmental Review Design Permitting Six Core Processes Subprocesses (examples) Air Quality Conformity Financial Constraint Influencing Processes (examples) Conservation Planning KDP KDP Key decision points take place along this process Source: ICF International, Inc., 2007. Figure 2.1. SHRP 2 C01 core processes.incorporated in this process, which is a key component of the SHRP 2 C02 research. A second example comes from a white paper written for the 2nd TRB Conference on Performance Measurement on devel- oping a performance-based program development and deliv- ery process (Figure 2.3). This approach, commonly referred to as Performance Management, uses performance measures and targets to link agency goals/objectives to specific resourceallocations. These allocations usually fall within broad cate- gories such as: 1) system preservation; 2) system management and operations; and 3) system capacity expansion. This latter investment category, capacity expansion, is the focus of the current framework. One point of this framework is that the issues addressed and measures used at various stages in the process may change as an agency moves from generalized needs to specific projects.Source: Florida Department of Transportation, 2007. FTP Program and Resource Plan DOT Work Program Output Outcome Policy-Level System-Level Program-Level Project-LevelPlannedProjects 5/10-yr. FDOT Plan/ Objectives FDOT Program/ Budget Link Short-Range Component 20 yr. Plan/ Outcomes Figure 2.2. Florida performance measures framework.

17Goals/Objectives Performance Targets for Planning/Programming Given Resource Availability System Performance Measures to Monitor Progress on Meeting Goals/Objectives Program Development and Project Selection Performance Measures to Track Delivery Specific Set of Programs and Projects with a Defined Budget, Schedule, Scope Program/Project Implementation Result = System Performance Co mp on en t I – Pr og ra m De ve lop me nt Co mp on en t II – Pr oje ct De live ry Co mp on en t II I – Sy ste m Mo nit or ing an d Re po rti ng Source: Hendren, Neumann, and Pickrell, 2004. Figure 2.3. Developing performance-based program and project delivery.The focus also changes from considerations of system per- formance (mobility, safety, etc.) to organizational performance (project delivery, quality, etc.). Based on this review and, in following the principles above, an initial draft framework was developed for the SHRP 2 C02 project. This initial framework had two basic dimensions (Table 2.1): • The stage of project development (across the columns); and • The factors considered such as mobility, safety, environment, economics, and community (down the rows). These broad dimensions were adapted and refined as described in chapter 3.Literature Review The literature review for this effort focused on materials rel- evant to development of performance measures to support capacity decision making. It considered measures relevant to the stages of the planning and project development process from long-range planning through environmental review. It primarily considers measures of physical impacts (i.e., amount of congestion, level of environmental harm or benefit), and not measures that address process and project delivery. Five key subject areas were reviewed: 1. General Use of Performance Measures by Transportation Agencies;

Phases Potentially Relevant Phases With Limited Relevance Monitoring and Factors Design Permitting Construction Operations Objective Mobility Safety Environment Economics Community Other 18 ecify project completely se of environ- mental best practices se of public involvement best practices oject delivery (let on time, on budget) Permits from resource agencies Permit delivery Construct capital projects Work zone and reliability issues Work zone issues Environmental monitoring Project delivery (on time/on budget) Operate the system in real time Evaluate mobility impacts Evaluate safety impacts Operational measures of reliability Table 2.1.Project Development Phases Relevant to SHRP 2 C02 Long-Range Preprogram Environmental Planning Studies Programming Review Identify system needs and projects Measures that identify and prioritize needs Measures that identify and prioritize needs Early issue overview (check boxes) Economic development/ impact measures Early overview of issues (check boxes) in regional/corridor studies Specify projects/ alternatives Detailed project analysis Detailed project analysis More detailed overview, but still summary Reliability Identify mix of projects to be constructed Select projects into program to meet needs Select projects into program to meet needs Criteria – avoid projects with fatal flaws Potential selection criteria Criteria – avoid projects with negative impacts Select preferred alternative Measures of mobility impacts Detailed measures of safety impacts Detailed environmental impact Detailed measures of community impacts Sp U U Pr Initial Performance Measures Framework Issues

192. Performance Measures for Transportation System En- hancement; 3. Performance Measures for Environmental Stewardship; 4. Performance Measures for Community Enhancement; and 5. Performance Measures for Economic Impacts and Devel- opment. The first subject area describes the state of the practice of transportation performance measurement. It reviews the most useful literature and describes the best practices and lessons learned as performance measurement has matured in the trans- portation industry. The following four subject areas are tied to sections of the performance measurement framework, and are discussed in the relevant section of the report that addresses the specific issue (chapters 4 through 8). Performance Management No state DOT conducts highway capacity decision making in a vacuum. The performance measurement framework described in this section for supporting highway capacity decision making is envisioned as part of a broader agencywide performance management system of the kind that some state DOTs are adopting. Performance management is a business process that links agency goals and objectives to resources and results. Figure 2.4 provides two examples of performance measure categories often used in making highway capacity investment decisions – safety and mobility – and shows how performance management is used to link goals and outcomes via use of measures. With a comprehensive performance management system in place, DOTs use performance measures systematically across a full range of core business functions to support decisions,communicate with external audiences, and manage towards strategic outcomes. A state DOT’s strategic mission, vision, goals, and objectives are tied to day-to-day activities via regular review of performance results in core DOT business functions such as pavement and bridge preservation, transportation safety, traffic operations, and infrastructure maintenance. Performance-based management has three essential and interconnected components: program development, project delivery, and system monitoring and reporting: 1. Program Development – Program Development typically begins with establishing agency goals and objectives that are in turn monitored through performance measures. Taking into account resource constraints, performance targets are set and projects and programs are identified and selected based on performance criteria aimed to lead a transportation agency toward its goals/objectives. For example, a state DOT could identify the goal “preserve the existing system” with the related performance mea- sure “percentage of highway miles with acceptable pave- ment condition.” In turn, a project selection (or program budgeting) criterion would be its estimated impact on highway pavement condition. The relationship between performance targets and project selection is an iterative process based on changing needs, available resources, and political support. 2. Program Delivery – Program Delivery begins when a set of preselected projects are “passed off” to the delivery team. A performance-based process uses a set of measures to evaluate and monitor project implementation (e.g., per- centage of construction contracts completed on time). Figure 2.2 illustrates not only the separation between the selection and delivery of projects and programs, but alsoGoal Objective Performance Measures Performance Target Ensure High Standards of Safety in the System Provide for Efficient Movement of People and Goods Reduce Rate of Motor Vehicle Crashes Decrease Travel Times for Commuting Crashes per VMT Crashes per Capita Hours-of-Delay Travel Time Index Reduce Crashes per VMT by One Percent per Year Reduce Delay by Two Percent per Year Travel Time Index = 1.25 Safety Mobility Figure 2.4. Example measures/targets for highway capacity investments.

20that there are two distinct groups of performance measures. One set of measures relates to project selection and is linked to agency goals and objectives, while the other set focuses on delivering projects. 3. System Monitoring and Reporting – System Monitoring and Reporting measures the changes that occur due to implemented projects and programs. It is the delivery of projects that produces the “result”, i.e., system performance. Thus, this component of the process indicates whether the intended goals and objectives of the project have been met (e.g., reduction in travel delays). The outcomes of performance-based management include more efficient distribution of limited resources and increased accountability and credibility in government. General Use of Performance Measures by Transportation Agencies Performance-based decision making has increasingly been used by transportation agencies over the past 10 years in a range of contexts and applications. Defined as a systemic, ongoing process integrated into an agency’s planning, manage- ment, and decision-making activities (Pickrell and Neumann, 2001), it involves a continuous effort of monitoring and feed- back that improves decision-making capabilities over time. Though transportation agencies have employed performance measures since the 1950s, the most dramatic evolution of the state of the practice has occurred in the last two decades. Improvements have been driven by: • Demands from the public and elected officials for increased accountability and performance; • Emergence of management systems for pavement, bridges, and congestion; • Strong leadership within the agencies themselves that bor- rowed from private sector-driven initiatives like “Six Sigma” and “Baldridge Awards”; and • Recognition that the decision-making environment within which agencies operate has become more complex (Poister, 2005; Larson, 2005; Bremmer et al., 2005). Examining the use of performance measures by transporta- tion agencies is particularly informative. The Government Per- formance Project (GPP), a nonpartisan, independent research program of the Pew Center on the States, acknowledges many DOTs are leaders in their use of performance measurement and can serve as models for other state agencies (Government Per- formance Project, 2005). Federal transportation agencies are often asked to pilot performance measure processes, and state DOTs are often asked to help other agencies with performance- based initiatives (Poister, 2005).Transportation practitioners in both the private and pub- lic sectors have worked diligently to achieve the current degree of performance measurement competency. Numer- ous peer exchanges and three international invitation-only conferences have been devoted to the subject. The FHWA maintains a web-based Performance Measurement Exchange, a thorough and well-maintained list of resources, including discussion boards, citations and links to journal articles, reports and studies, and a directory of practitioners (Perfor- mance Measurement Exchange, 2007). Furthermore, literature on the use of performance measures by transportation agen- cies is extensive. It documents the wide range of activities in which measures are relevant and informative. Several efforts comprehensively review the history, development, state of the practice, best practices, and recent trends in performance- based planning (Cambridge Systematics, Inc., 2000; Hendren et al., 2005; Pickrell and Neumann, 2001; Poister, 1997; Pad- gett, 2006). Strategic Performance Measures for State Depart- ments of Transportation: A Handbook for CEOs and Executives (TransTech Management Inc., 2003) describes the core areas in which most DOTs use measures, and the three primary reasons they use measures: communication, management, and decision making. NCHRP Report 551: Performance Mea- sures and Targets for Transportation Asset Management reviews the literature and addresses several key questions: • What criteria are used by agencies to select performance measures? • How are current performance measurement frameworks structured, and what kinds of measures do they include? • How are performance measures being used to gauge the impacts of transportation investments, support resource allocation and utilization decisions, and assess agency per- formance in program delivery and cost-effectiveness? • How are measures being tailored for different levels of trans- portation organizations? • How are measures being used to communicate program status – both internally and externally? NCHRP Report 446: A Guidebook for Performance-Based Transportation Planning provides practitioners an extensive library of measures organized by the following categories: • Accessibility; • Mobility; • Economic development; • Quality of life; • Environmental and resource conservation; • Safety; • Operation efficiency; • System preservation; and • Measures relevant to multiple goal categories (Cambridge Systematics, Inc., 2000).

21Key Findings The extensive literature on transportation performance mea- sures points to several general trends. Performance measures should be driven by strategically aligned goals and objectives. Performance measures should be identified in response to goals and objectives and not the other way around. An agency’s goals should reflect the most important aspects of what it wishes to accomplish. Perfor- mance measures are the primary means of assessing how suc- cessful an agency is in accomplishing its goals. Therefore, it should be clear what goal(s) each measure illustrates advance- ment of. Failure to properly align measures with goals and objectives can result in tracking measures that have little to do with performance of the organization or transportation sys- tem (Pickrell and Neumann, 2001; Poister et al., 2004; Kittelson & Associates, Inc. et al., 2003). Input, output, and outcome measures should all be included in performance measurement. Many of the mea- sures used to monitor systems operations are derived from those developed in the 1950s, and these measures reflect the values of that time, which have evolved significantly. As nega- tive impacts from transportation and other infrastructure investments have become apparent, mainstream concerns have changed to consider a wider range of ways in which transportation affects our communities (Meyer, 2001). The type of measures considered for transportation planning has grown to include not only those that consider input (time, capital, resources) and output (speed, throughput, conges- tion), but also those that consider outcomes to communities and the environment (Poister, 1997; Poister and Van Slyke, 2001). Consideration of both output and outcome measures reflect the differences in perspectives of those who manage the system and those who use it (Shaw, 2003; Kittelson & Associates, Inc. et al., 2003). Performance measurement efforts should concentrate on the “vital few.” Agencies must use simple, understand- able measures and avoid attempting to measure everything (TransTech Management, Inc., 2006; Larson, 2005; Kassof, 2001). Providing excessive or redundant measures may over- whelm the end user and obscure key drivers of service quality. As cited in TCRP Report 88, Brown (1996) describes this as choosing between “the vital few measures and the trivial many,” and suggests an upper limit of 20 measures. Some of the desired “vital few” may not be available during the first iteration of a performance-based process. This deficiency should be addressed in future efforts (Larson, 2005). Early attempts at performance measurement should emphasize process as well as results. Management and staff should set realistic expectations about first iteration results as performance-based planning is an inherently incremental process. The implementation strategy will evolve over time as stakeholder and leadership buy-in improve, performancemeasures become focused on the “vital few,” and technical capabilities advance. Performance-based processes should inform the decision-making process, not replace it, and initial impacts on final decisions may be subtle. Decision makers may initially be slow to accept performance-based recom- mendations of staff, but they will adopt the additional infor- mation into their decision-making process at varying rates as agencywide support for the approach builds. Staff should not give up as providing better quality data from performance measuring efforts can only improve the decision-making process (Pickrell and Neumann, 2001). Performance measurement programs are most effective when integrated throughout an organization. The linkage between program development and project delivery is a vital component to mature, integrated, performance-based method- ologies. Few state transportation agencies have implemented both, and fewer have linked the two approaches together. In isolation, a performance-based program development process may identify the best projects to fund but may not guard against excessive scope creep, schedule slippage, or cost esca- lation. Conversely, a performance-based delivery process in isolation may result in efficient delivery of a program that includes marginal projects. By executing both performance- based program development and project delivery, the most effective set of projects is selected and implemented efficiently (Hendren et al., 2005; Padgett, 2006). Performance measurement reporting should be appro- priately tailored to intended audiences. The informational needs of technical staff, decision makers, and the public are different, and presentation and depth of reporting should reflect these needs (Shaw, 2003; Cambridge Systematics, Inc., 2006; Poister, 2005). TCRP Report 88, A Guidebook for Devel- oping a Transit Performance-Measurement System highlights this diversity, noting the difference in perspective of four different transit stakeholders: the customer, community, agency, and driver. In addition to delivering the right data to the right people, reporting should be as simple and con- sistent as possible. Aside from providing clarity for decision makers, simplicity and regularity facilitate process improve- ment of each iteration of the performance-based process (Larson, 2005). Successful performance measurement programs require high-level buy-in. A performance-based approach must have widespread and deep-rooted support to withstand significant changes in leadership. Five key stakeholder groups must accept a program for it to have long-term viability: • Agency management; • Agency staff; • Customers; • Agency’s governing body; and • Senior contractors (Kittelson & Associates, Inc. et al., 2003).

22Stakeholders, especially those held accountable for results, should be involved in deciding what to measure, how to measure, and how to convey the results. Most importantly, measures should be used to represent the current state of the system and to focus on opportunities for improvement rather than blame (Kassof, 2001). Practitioners should strive for consistency of performance measurement terms and definitions. A persistent challenge amongperformancemeasurement practitioners is one of termi- nology.Areportin1999notes that even discussing performance- based planning first requires a discussion on what exactly the involved parties are talking about (Cambridge Systematics, Inc., 1999). Six years later, two separate reports acknowledge the problem remains persistent (Bremmer et al., 2005; Poister, 2005). Measuring Performance Among State DOTs (TransTech Man- agement, Inc., 2006) describes how state departments of transportation may develop and use comparative perfor- mance measures to improve communication, promote aware- ness about best practices and innovations, improve business processes, and increase responsiveness to customers’ needs. In-Depth Interviews As part of the initial research conducted for the SHRP 2 C02 project, the research team conducted 17 interviews of state DOTs and MPOs to gather information about current use of performance measurement to support capacity decision making. Feedback from these interviews was used as an input into an initial list of performance measures, analysis tech- niques, and policy considerations to be considered during development of the performance measures framework. The interview focus was on performance measurement for capac- ity enhancements, but performance measurement processes associated with other project types were reviewed if there was a logical extension to the capacity enhancement approach. Transportation agencies were selected for profile based on the following criteria: • Capacity Expansion – Is the agency pursuing new capac- ity or identifying methods to evaluate and prioritize capac- ity projects? • Performance Measures – What is the agency’s history with the use of performance measures of any kind? Have they actively used them to evaluate and prioritize projects, or monitor outcomes? • Nontraditional Measures – Does the agency use non- traditional performance measures or is it developing programs or initiatives to address environmental issues, economic impacts, or community impacts in a unique way? • Data – Does the agency have data management, collection, or sharing programs that are worthy of note?Agencies interviewed as part of the initial process are iden- tified in Table 2.2. Interviews were completed over the course of three months, May through July 2007, via telephone. Inter- views were designed to provide a broad understanding of the use of performance measures in an agency, with additional detail on the specific quantitative methods used, as well as issues and concerns associated with implementing a perfor- mance measurement system. Specific interview questions centered around four main topic areas: 1. Agency Basics – Overview of the agency role and author- ity in managing the transportation system. 2. Project Identification and Prioritization – Detail related to the process used to identify, evaluate, and prioritize proj- ects with a focus on the specific criteria and performance measures used to evaluate and select highway capacity proj- ects for funding. 3. Data and Analysis – Description of the data and analytic techniques needed to support performance-based decision making. Agency Type Agencies Contacted State DOT • Arizona Department of Transportation • Florida Department of Transportation • Minnesota Department of Transportation • Montana Department of Transportation • Ohio Department of Transportation • Oregon Department of Transportation • Pennsylvania Department of Transportation • Texas Department of Transportation • Virginia Department of Transportation • Washington Department of Transportation • Wisconsin Department of Transportation MPO • Atlanta Regional Commission (Atlanta, Georgia) • Delaware Valley Regional Planning Commission (Philadelphia, Pennsylvania) • East-West Gateway Coordinating Council (St. Louis, Missouri) • Metroplan (Little Rock, Arkansas) • New York Metropolitan Transportation Coun- cil (New York, New York) • Sacramento Area Council of Governments (Sacramento, California) Table 2.2. Transportation Agency Interviews

234. Political Context – Additional detail related to the politi- cal context impacting funding decisions and “unofficial” constraints on the prioritization of projects. Although no one interview provided complete information to support development of a performance measures frame- work, each area provided unique examples of current practice, areas of innovation, and constraints on the decision-making process. Current practice in the area of performance-based decision-making varies greatly by area, depending largely on the level of sophistication in the analysis tools and processes maintained by each state DOT or MPO, as well as data avail- ability and staff/financial constraints. Despite the variability in performance measurement systems, it is clear that there is increasing momentum and support for implementation of performance-based decision-making processes within most state DOTs and MPOs. Summary of National Trends in Performance-Based Decision Making The in-depth case studies completed for this phase of the back- ground effort supported the findings of the literature review, discussed in chapter 2. Specifically, performance measurement is becoming much more prevalent, and is being applied in a wide range of contexts throughout the work of both DOTs and MPOs. A number of significant advancements in performance- based processes have occurred over the last decade, largely driven by increasing demands from the public and elected officials for more accountability in the transportation decision- making process, as well as the recognition that the decision- making environment has become much more complex and, therefore, requires more structure and organization. Transportation agencies have developed both formal and informal performance-based approaches to support trans- portation investment decisions. The practice at most state DOTs is some variation on a two-stage process. The first stage includes decisions about the amount of funding that will be directed to general program or project types, e.g., maintenance, capacity additions, and operations are typical high-level allo- cation categories. Decision makers at this level use perfor- mance measures to determine the trend in certain key aspects of the system and decide whether more or less funding is needed. In the second stage, decisions are made within project type subcategories, to help prioritize projects with similar need/ purpose to aid in programming decisions. The attributes within project type are more nearly similar and quantitative data plays a more important role in developing performance mea- sures to assist in project evaluation and prioritization. For MPOs, project evaluation processes tend to be more focused, e.g., on particular funding categories and within particular geographic areas. This is largely a result of the more stringentfiscal constraint requirements and air quality requirements in air quality nonattainment areas that must be met at the MPO (rather than state) level as part of long-range transportation plan development. Performance criteria by which projects are evaluated are typically well established and vetted, with more refined prioritization processes in place. Evaluation tends to be more specific for projects programmed in the short-term Transportation Improvement Program (TIP), where there is substantially more project information available to support a detailed project review. For all the state DOTs and MPOs interviewed as part of the interview effort, limited federal and state funding was cited as a key constraint on prioritization efforts for capacity-adding projects. In particular, it was emphasized that the proportion of funding allocated to system preservation and maintenance is increasing significantly, and that the share of funding for capacity improvements is decreasing as a result. Most trans- portation agencies have a policy in place to preserve and maintain existing systems before constructing new projects. Examples include Ohio DOT’s “Fix It First” policy, Minnesota DOT’s “Safeguard What Exists” policy, and a preservation first policy in Florida that has been adopted into state law. As emphasized through each interview, more funding is required to preserve aging infrastructure, leaving less money available for programming new (capacity-adding) projects. In many areas, (e.g., Atlanta, Denver), project prioritization is occur- ring to “deselect” projects that were previously programmed, in order to ensure a fiscally constrained transportation plan and program in the context of limited transportation funding. In light of severe funding constraints, many areas interviewed have defined a priority transportation network to focus trans- portation investment, in particular for capacity improvements. Florida, for example, developed the Strategic Intermodal Sys- tem; the Atlanta Regional Commission and Metroplan in Little Rock, Arkansas, also have identified regional strategic systems within their MPO planning areas to help focus limited funding. Capacity-adding roadway projects are typically the focus of increased scrutiny due to the greater cost and physical impact of these types of projects. Several agencies interviewed have developed a more refined definition of capacity-addition to demonstrate the capacity benefits of other types of smaller- scale projects. For example, Minnesota DOT and Montana DOT have redefined capacity away from solely mega-projects that typically yield a great benefit for a smaller segment of the system, towards lower-cost projects that provide more incre- mental capacity across a larger part of the system without major construction and with a smaller physical footprint (e.g., geometric redesign, shoulder improvements). Virginia, by con- trast, includes most of smaller capacity improvements in its maintenance budget. For both state DOTs and MPOs, the critical driver of imple- menting a performance-based system is using data-driven

24decision processes to support improved decision making and accountability within the organizations and, ultimately, to improve transportation system performance through better project selection. Only a few state DOTs and MPOs have begun the transition to a full-fledged performance measurement approach toward decision making. However, it seems likely that as budgetary pressures on transportation agencies con- tinue to grow, the use of performance-based decision-making systems will increase. Common Themes The interviews uncovered a number of common themes related to implementation of performance measurement systems. Most importantly, and common to all states and MPOs inter- viewed, the processes are viewed and used as decision-support tools. As such, they do not completely replace qualitative con- siderations or political realities, and they do not override common sense. Application of Performance Measures Is Limited by Tools and Data Available Performance measurement systems and project evaluation/ prioritization processes should be quantitatively informed and integrate available data and technical methodologies where available; however, both state DOTs and MPOs report a limited scope for application of data-driven performance measurement systems. In nearly every state, detailed project evaluation tends to be made for capital investments–typically on highways–with congestion reduction and safety impact as the two key performance criteria used in project-level evalu- ations. This is largely driven by the availability of data and tools needed for detailed analysis, and not a reflection of a lack of comprehensive objectives for transportation investment (e.g., economic development, connectivity, accessibility, etc.) or the lack of support for more expansive project-level assess- ment (e.g., environmental or community impact). Travel demand models in use by most state DOTs and MPOs do an excellent job of demonstrating the congestion reduction potential of highway capacity-adding projects and generating VMT estimates commonly used in safety analyses, but are limited in their ability to demonstrate additional “off-model” benefits or impacts. Most states and MPOs reported difficulty integrating environmental and community concerns in an objective manner into the project evaluation process. Increasing Use of Top-Down Approach for Performance Measure Systems One of the ways that state transportation officials have of target- ing budgetary resources is to establish performance measuresand/or targets that are adopted by state DOT management, or in some cases, incorporated by the state legislature into law. These measures and/or targets can then be clearly communi- cated to transportation agencies throughout the state for implementation. In Minnesota, for example, the Mn/DOT’s “Safeguard What Exists” policy has three major elements: 1. Preserve essential elements of existing transportation systems; 2. Support land use decisions that preserve mobility and enhance the safety of transportation systems (most measures and targets under this policy are not yet operational); and 3. Effectively manage the operation of existing transportation systems to provide maximum service to customers. Each policy has a set of measures and targets that allow Mn/DOT officials to monitor progress over time, and that can be used by the district offices in establishing their policy directions and refining project selection. In many states, including Montana, Florida, and Minnesota, a cross-program analysis is used to determine funding alloca- tions by project type, typically in an attempt to meet pre- defined targets. Specific project selection responsibility is then devolved to districts, MPOs, or specific program man- agers within the project types. This typically occurs through a two-step process in which state funding allocations are determined by program/project type within statewide finan- cial constraints through systems-level analysis in the first phase. The funding allocations then serve as the foundation for DOT districts or MPOs to define projects to meet the per- formance thresholds as part of the second phase. District offices are given a high degree of flexibility in selecting proj- ects, but they are still selected in the context of priorities and funding allocations established through the first phase. Another interesting example of a top-down approach to establishing performance measurements and targets at an MPO level is found in Atlanta. In March 2005, a Congestion Mitigation Task Force (CMTF) was established by Georgia Governor Sonny Perdue to develop strategies, benchmarks, and goals to cost-effectively reduce congestion in the metro- politan Atlanta area. The CMTF developed three recommen- dations for incorporation into the regional transportation planning process as shown below: 1. Refine the current transportation project selection process to increase the weighting of the congestion factor to 70 percent; 2. Develop and implement a technically consistent and trans- parent methodology for benefit/cost analysis; and 3. Use the Travel Time Index to measure improvement in congestion. The Task Force recommended a regional Travel Time Index goal of 1.35 by 2030 for the Atlanta area.

25The Atlanta Regional Commission has subsequently devel- oped a refined project evaluation process to accommodate these prescriptive recommendations which was used to develop the latest Regional Transportation Plan. Increasing Priority to Projects That Are “Deliverable” – Early Environmental Screening Several states and MPOs interviewed are conducting more robust environmental screening of projects early in the plan- ning process. This supports two larger goals – one, the need to streamline environmental review and reduce the time it takes to implement a project and, two, to help prioritize investment on projects that are deliverable in a reasonable amount of time. Florida has developed the Efficient Transportation Decision- Making Process (ETDM), led by its Central Environmental Management Office, to streamline environmental review and involve resource agencies early on in the planning process. Many other states (Montana, Arizona, Ohio, and Texas) and regions (Atlanta, Denver) also are conducting high-level envi- ronmental screening processes as part of plan and program development to “red flag” projects that may take significantly more time to implement due to environmental issues. Increasing Implementation of Nontraditional Performance Measures A number of areas interviewed are actively looking to refine project evaluation processes to include nontraditional per- formance metrics. For example, Florida and Atlanta are both looking to improve travel-time reliability measures, the Ore- gon DOT has established a Sustainability Program and CS3 framework (context-sensitive and sustainable solutions) to incorporate sustainability goals into the highway project deliv- ery process, and New York now requires a greenhouse gas inventory to be completed by MPOs as part of transportation plan/program development. Though the trend is to expand the type of metrics included as part of project-level evaluation, transportation agencies are largely limited by data and tools needed to perform meaningful analyses. Targeted Case Studies Following the in-depth interviews of state and regional trans- portation agencies described in the previous section, the research team conducted targeted case studies of programs employing or contributing to a performance-based decision- making process across a wide range of agency types and levels of government. The purpose of this second phase outreach was to observe performance-based decision making in action, with the goal of filling out and refining the performance measures frame-work based on real world experience. For each of the factors established by this effort, research team staff: • Gathered all relevant research or descriptions of performance measure programs; • Interviewed agencies with experience developing perfor- mance measures or data-driven decision making; and • Synthesized the information gathered from these efforts around four key themes: – A description of the relevance of individual factors to highway capacity decision making across the stages of the project development process; – Detailed lists and descriptions of performance mea- sures and how they might be used in the process; – Detailed descriptions of data sources and tools that are used to support the evaluation of a particular factor; and – Summary of resources for the factor broadly (e.g., web sites and reports to find out more information about the factor), as well as specific cases of application of per- formance measures (e.g., case studies). A total of 54 case studies were developed as part of this effort. A complete list of case studies, including several detailed write- ups is available in Appendix A. Common Themes, Trends, Successes, and Challenges This stage of the research process focused on specific pro- grams and not broad initiatives. The objective was to identify measures and processes that could be built into the perfor- mance measures framework described in chapters 3 through 8. As such, most of the research generated does not lend itself to broad themes, but is instead more narrowly focused. How- ever, several key themes were identified from this phase of the research. One of the keys to successful performance-based decision making common to many of the factor areas was the establish- ment of early warning/clearance mechanisms. Such systems seek to identify “red flags” that might derail or complicate a project before significant resources are invested into planning the project, and before it is programmed. The presence of factors such as significant wetlands, endangered habitats, or archaeological sites can be determined even before formal feasibility studies are conducted, and the discovery of such factors may impact a decision on whether or not to proceed with project planning. Conversely, if a candidate project is found to be unaffected by such factors, its odds of reaching completion without significant unforeseen complications are improved. One of the more innovative examples of this is Minnesota’s Mn/Model, a predictive model that uses a variety of available data sources to predict the presence of buried

26artifacts. The Washington DOT Transportation Project Miti- gation Cost Screening Matrix is another program specifically designed to provide early clearance or red flagging of the potential mitigation requirements of a proposed project, in the form of a “Mitigation Risk Index” (MRI). Florida’s recently implemented Efficient Transportation Decision Making (ETDM) environmental review process also makes significant strides toward identifying and addressing con- flicts earlier in the process. Another encouraging trend found primarily in the environ- mental area is a shift in emphasis from quantity of resources to quality of resources. This shift has been particularly notice- able in wetlands research, where some of the most advanced practices now focus on the performance of mitigated wet- lands, rather than simply the volume. In New Jersey, for example, the Department of Environmental Projection has developed the Wetland Mitigation Rapid Assessment Tool, which seeks to assess whether a mitigated wetland – natural or created – will perform “wetland functions” in the future. Similarly, programs in Pennsylvania and Florida use point systems to evaluate the current status of environmental fea- tures throughout the state (Pennsylvania’s program focuses on level of wetland degradation, while Florida’s measures habi- tat conservation priority), in order to provide ready informa- tion before a corridor-specific study is undertaken. A related theme that has arisen is the potential for tech- nology to improve the tools and data that support decisionmaking. For example, much of the research on environmental quality is in its infancy, but as it develops it has the poten- tial to produce significant evaluation criteria in the trans- portation sector. This will require a process to translate scientific research into readily used and understood perfor- mance metrics. A common goal of many programs has been to accumulate more advanced, statewide datasets, rather than relying more heavily on project-specific inquiries, and this process has often been combined with more user-friendly reporting methods, often in the form of scoring systems. Such efforts were incor- porated into programs studied in Minnesota, Pennsylvania, Washington, and Florida, among others. Finally, it is clear from the research that the complexity of the issues identified in the SHRP 2 C02 research requires significant translation work to make this information use- ful to decision makers, especially at early stages of the proj- ect development process. During planning and early project development, it is necessary to develop a broad under- standing of all of the types of impacts to be considered. Per- formance measures should help address questions such as: Does a project meet the goals of the transportation agency? Does the project have significant potential environmental or community impacts? These broad questions can be informed by different types of data, but for the information to be useful to decision makers, the basic answers need to be kept simple.

Next: CHAPTER 3 - Performance Measurement Framework »
Performance Measurement Framework for Highway Capacity Decision Making Get This Book
×
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

TRB’s second Strategic Highway Research Program (SHRP 2) Report S2-C02-RR: Performance Measurement Framework for Highway Capacity Decision Making explores a performance measurement framework that is designed to support the collaborative decision-making framework (CDMF) for additions to highway capacity being developed under the SHRP 2 Capacity research program. The report examines five broad areas of performance including transportation, environment, economics, community, and cost. Under these headings, the report identifies 17 performance factors, each of which are linked to key decision points in the CDMF.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!