Click for next page ( 23

The National Academies of Sciences, Engineering, and Medicine
500 Fifth St. N.W. | Washington, D.C. 20001

Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement

Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 22
22 Stakeholders, especially those held accountable for results, Agencies interviewed as part of the initial process are iden- should be involved in deciding what to measure, how to tified in Table 2.2. Interviews were completed over the course measure, and how to convey the results. Most importantly, of three months, May through July 2007, via telephone. Inter- measures should be used to represent the current state of the views were designed to provide a broad understanding of the system and to focus on opportunities for improvement rather use of performance measures in an agency, with additional than blame (Kassof, 2001). detail on the specific quantitative methods used, as well as Practitioners should strive for consistency of performance issues and concerns associated with implementing a perfor- measurement terms and definitions. A persistent challenge mance measurement system. amongperformancemeasurement practitioners is one of termi- Specific interview questions centered around four main nology.Areportin1999notes that even discussing performance- topic areas: based planning first requires a discussion on what exactly the involved parties are talking about (Cambridge Systematics, Inc., 1. Agency Basics Overview of the agency role and author- 1999). Six years later, two separate reports acknowledge the ity in managing the transportation system. problem remains persistent (Bremmer et al., 2005; Poister, 2005). 2. Project Identification and Prioritization Detail related Measuring Performance Among State DOTs (TransTech Man- to the process used to identify, evaluate, and prioritize proj- agement, Inc., 2006) describes how state departments of ects with a focus on the specific criteria and performance transportation may develop and use comparative perfor- measures used to evaluate and select highway capacity proj- mance measures to improve communication, promote aware- ects for funding. ness about best practices and innovations, improve business 3. Data and Analysis Description of the data and analytic processes, and increase responsiveness to customers' needs. techniques needed to support performance-based decision making. In-Depth Interviews As part of the initial research conducted for the SHRP 2 C02 project, the research team conducted 17 interviews of state Agency Type Agencies Contacted DOTs and MPOs to gather information about current use State DOT Arizona Department of Transportation of performance measurement to support capacity decision making. Feedback from these interviews was used as an input Florida Department of Transportation into an initial list of performance measures, analysis tech- Minnesota Department of Transportation niques, and policy considerations to be considered during Montana Department of Transportation development of the performance measures framework. The Ohio Department of Transportation interview focus was on performance measurement for capac- ity enhancements, but performance measurement processes Oregon Department of Transportation associated with other project types were reviewed if there was Pennsylvania Department of Transportation a logical extension to the capacity enhancement approach. Texas Department of Transportation Transportation agencies were selected for profile based on Virginia Department of Transportation the following criteria: Washington Department of Transportation Capacity Expansion Is the agency pursuing new capac- Wisconsin Department of Transportation ity or identifying methods to evaluate and prioritize capac- MPO Atlanta Regional Commission ity projects? (Atlanta, Georgia) Performance Measures What is the agency's history with Delaware Valley Regional Planning the use of performance measures of any kind? Have they Commission (Philadelphia, Pennsylvania) actively used them to evaluate and prioritize projects, or East-West Gateway Coordinating Council monitor outcomes? (St. Louis, Missouri) Nontraditional Measures Does the agency use non- Metroplan (Little Rock, Arkansas) traditional performance measures or is it developing New York Metropolitan Transportation Coun- programs or initiatives to address environmental issues, cil (New York, New York) economic impacts, or community impacts in a unique Sacramento Area Council of Governments way? (Sacramento, California) Data Does the agency have data management, collection, or sharing programs that are worthy of note? Table 2.2. Transportation Agency Interviews

OCR for page 22
23 4. Political Context Additional detail related to the politi- fiscal constraint requirements and air quality requirements in cal context impacting funding decisions and "unofficial" air quality nonattainment areas that must be met at the MPO constraints on the prioritization of projects. (rather than state) level as part of long-range transportation plan development. Performance criteria by which projects are Although no one interview provided complete information evaluated are typically well established and vetted, with more to support development of a performance measures frame- refined prioritization processes in place. Evaluation tends to work, each area provided unique examples of current practice, be more specific for projects programmed in the short-term areas of innovation, and constraints on the decision-making Transportation Improvement Program (TIP), where there is process. Current practice in the area of performance-based substantially more project information available to support a decision-making varies greatly by area, depending largely on detailed project review. the level of sophistication in the analysis tools and processes For all the state DOTs and MPOs interviewed as part of the maintained by each state DOT or MPO, as well as data avail- interview effort, limited federal and state funding was cited as ability and staff/financial constraints. Despite the variability a key constraint on prioritization efforts for capacity-adding in performance measurement systems, it is clear that there is projects. In particular, it was emphasized that the proportion increasing momentum and support for implementation of of funding allocated to system preservation and maintenance performance-based decision-making processes within most is increasing significantly, and that the share of funding for state DOTs and MPOs. capacity improvements is decreasing as a result. Most trans- portation agencies have a policy in place to preserve and maintain existing systems before constructing new projects. Summary of National Trends in Examples include Ohio DOT's "Fix It First" policy, Minnesota Performance-Based Decision Making DOT's "Safeguard What Exists" policy, and a preservation The in-depth case studies completed for this phase of the back- first policy in Florida that has been adopted into state law. As ground effort supported the findings of the literature review, emphasized through each interview, more funding is required discussed in chapter 2. Specifically, performance measurement to preserve aging infrastructure, leaving less money available is becoming much more prevalent, and is being applied in a for programming new (capacity-adding) projects. In many wide range of contexts throughout the work of both DOTs and areas, (e.g., Atlanta, Denver), project prioritization is occur- MPOs. A number of significant advancements in performance- ring to "deselect" projects that were previously programmed, based processes have occurred over the last decade, largely in order to ensure a fiscally constrained transportation plan driven by increasing demands from the public and elected and program in the context of limited transportation funding. officials for more accountability in the transportation decision- In light of severe funding constraints, many areas interviewed making process, as well as the recognition that the decision- have defined a priority transportation network to focus trans- making environment has become much more complex and, portation investment, in particular for capacity improvements. therefore, requires more structure and organization. Florida, for example, developed the Strategic Intermodal Sys- Transportation agencies have developed both formal and tem; the Atlanta Regional Commission and Metroplan in Little informal performance-based approaches to support trans- Rock, Arkansas, also have identified regional strategic systems portation investment decisions. The practice at most state within their MPO planning areas to help focus limited funding. DOTs is some variation on a two-stage process. The first stage Capacity-adding roadway projects are typically the focus of includes decisions about the amount of funding that will be increased scrutiny due to the greater cost and physical impact directed to general program or project types, e.g., maintenance, of these types of projects. Several agencies interviewed have capacity additions, and operations are typical high-level allo- developed a more refined definition of capacity-addition to cation categories. Decision makers at this level use perfor- demonstrate the capacity benefits of other types of smaller- mance measures to determine the trend in certain key aspects scale projects. For example, Minnesota DOT and Montana of the system and decide whether more or less funding is DOT have redefined capacity away from solely mega-projects needed. In the second stage, decisions are made within project that typically yield a great benefit for a smaller segment of the type subcategories, to help prioritize projects with similar need/ system, towards lower-cost projects that provide more incre- purpose to aid in programming decisions. The attributes within mental capacity across a larger part of the system without project type are more nearly similar and quantitative data major construction and with a smaller physical footprint (e.g., plays a more important role in developing performance mea- geometric redesign, shoulder improvements). Virginia, by con- sures to assist in project evaluation and prioritization. For trast, includes most of smaller capacity improvements in its MPOs, project evaluation processes tend to be more focused, maintenance budget. e.g., on particular funding categories and within particular For both state DOTs and MPOs, the critical driver of imple- geographic areas. This is largely a result of the more stringent menting a performance-based system is using data-driven

OCR for page 22
24 decision processes to support improved decision making and and/or targets that are adopted by state DOT management, or accountability within the organizations and, ultimately, to in some cases, incorporated by the state legislature into law. improve transportation system performance through better These measures and/or targets can then be clearly communi- project selection. Only a few state DOTs and MPOs have begun cated to transportation agencies throughout the state for the transition to a full-fledged performance measurement implementation. In Minnesota, for example, the Mn/DOT's approach toward decision making. However, it seems likely "Safeguard What Exists" policy has three major elements: that as budgetary pressures on transportation agencies con- tinue to grow, the use of performance-based decision-making 1. Preserve essential elements of existing transportation systems will increase. systems; 2. Support land use decisions that preserve mobility and enhance the safety of transportation systems (most measures Common Themes and targets under this policy are not yet operational); and The interviews uncovered a number of common themes related 3. Effectively manage the operation of existing transportation to implementation of performance measurement systems. systems to provide maximum service to customers. Most importantly, and common to all states and MPOs inter- viewed, the processes are viewed and used as decision-support Each policy has a set of measures and targets that allow tools. As such, they do not completely replace qualitative con- Mn/DOT officials to monitor progress over time, and that siderations or political realities, and they do not override can be used by the district offices in establishing their policy common sense. directions and refining project selection. In many states, including Montana, Florida, and Minnesota, a cross-program analysis is used to determine funding alloca- Application of Performance Measures Is Limited tions by project type, typically in an attempt to meet pre- by Tools and Data Available defined targets. Specific project selection responsibility is Performance measurement systems and project evaluation/ then devolved to districts, MPOs, or specific program man- prioritization processes should be quantitatively informed agers within the project types. This typically occurs through and integrate available data and technical methodologies a two-step process in which state funding allocations are where available; however, both state DOTs and MPOs report determined by program/project type within statewide finan- a limited scope for application of data-driven performance cial constraints through systems-level analysis in the first measurement systems. In nearly every state, detailed project phase. The funding allocations then serve as the foundation evaluation tends to be made for capital investmentstypically for DOT districts or MPOs to define projects to meet the per- on highwayswith congestion reduction and safety impact as formance thresholds as part of the second phase. District the two key performance criteria used in project-level evalu- offices are given a high degree of flexibility in selecting proj- ations. This is largely driven by the availability of data and ects, but they are still selected in the context of priorities and tools needed for detailed analysis, and not a reflection of a lack funding allocations established through the first phase. of comprehensive objectives for transportation investment Another interesting example of a top-down approach to (e.g., economic development, connectivity, accessibility, etc.) establishing performance measurements and targets at an or the lack of support for more expansive project-level assess- MPO level is found in Atlanta. In March 2005, a Congestion ment (e.g., environmental or community impact). Travel Mitigation Task Force (CMTF) was established by Georgia demand models in use by most state DOTs and MPOs do Governor Sonny Perdue to develop strategies, benchmarks, an excellent job of demonstrating the congestion reduction and goals to cost-effectively reduce congestion in the metro- potential of highway capacity-adding projects and generating politan Atlanta area. The CMTF developed three recommen- VMT estimates commonly used in safety analyses, but are dations for incorporation into the regional transportation limited in their ability to demonstrate additional "off-model" planning process as shown below: benefits or impacts. Most states and MPOs reported difficulty integrating environmental and community concerns in an 1. Refine the current transportation project selection process objective manner into the project evaluation process. to increase the weighting of the congestion factor to 70 percent; 2. Develop and implement a technically consistent and trans- Increasing Use of Top-Down Approach parent methodology for benefit/cost analysis; and for Performance Measure Systems 3. Use the Travel Time Index to measure improvement in One of the ways that state transportation officials have of target- congestion. The Task Force recommended a regional Travel ing budgetary resources is to establish performance measures Time Index goal of 1.35 by 2030 for the Atlanta area.