Performance Measures to Improve Transportation Systems: Summary of the Second National Conference (2005) / Chapter Skim
Currently Skimming:
Pages 7-84
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 7... ...
SESSIONS AND TOPICS 99395mvp9_38 12/13/05 12:24 PM Page 1
|
From page 8... ...
99395mvp9_38 12/13/05 12:24 PM Page 2
|
From page 9... ...
This conference focuses on the ultimate purpose of performance measures: to improve transportation services for our customers. The opening session this afternoon highlights the objectives of this conference and summarizes the state of the practice in the use of performance measures by transportation agencies at different levels of government throughout the country.
|
From page 10... ...
I would like to recognize Sandy Straehl from the Montana Department of Transportation, who served with me as cochair of the Conference Planning Committee. Many of you participated in the first conference, which focused primarily on defining the concept of performance measurement, identifying potential performance measures, and promoting the use of performance-based management within transportation agencies.
|
From page 11... ...
The use of performance measures helps ensure that transportation agencies follow up on commitments. FHWA also helped sponsor a roundtable discussion of performance measures and statewide transportation planning in Washington, D.C., in October 2003.
|
From page 12... ...
Imme6 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS Program Construction Maintenance Safety Operations Public Transportation Outputs Projects Completed Lane Miles Bridges Built Miles Resurfaced Repairs Made Treatments Applied Projects Completed Turn Lanes Added Stripes Painted Messages Displayed Incidents Cleared Signals Timed Vehicle Hours Vehicle Miles Seat Miles Immediate Outcomes > Capacity > Connectivity > Condition Smoother Pavements < Hazards More Efficient Operation > Coverage < Headways Intermediate Outcomes < Congestion < Travel Times > Convenience > Ride Quality < Operating Expense < Crashes < Injuries < Fatalities < Congestion < Delays < Crashes < Waiting > Ridership > Convenience Longer-Term Outcomes Mobility Quality of Life Economic Development Environmental Enhancement Community Development System PerformanceProgram & Service Delivery Impacts FIGURE 1 Example of transportation program logic model.
|
From page 13... ...
For example, the Maryland State Highway Administration's CHART Program measures incident duration, initial response time, and overall recovery time. Performance measures at many state departments of transportation address program delivery.
|
From page 14... ...
The Florida Department of Transportation integrates the Florida longrange transportation plan, the short-range component, the annual strategic objectives, and the executive board initiatives through performance measures. The Minnesota Department of Transportation's measurement pyramid helps link the use of performance measures in the different plans.
|
From page 15... ...
A number of continuing challenges face transportation agencies in the application of performance measures. Some of these challenges include agreeing on common terminology for terms such as dashboards, benchmarking, and performance management (see the resource paper "Performance Measurement in Transportation: State of the Practice" on pages 81–98 of these proceedings for examples)
|
From page 16... ...
I think policy makers and the public in many states are interested in project delivery performance measures. Delivering projects on time and on budget has become the mantra at transportation agencies.
|
From page 17... ...
As the Monday evening session on the international scan will point out, performance measures have been used in Canada, Australia, New Zealand, and Japan to target, deliver, and measure safety projects. As you all know, there are many ways to describe congestion.
|
From page 18... ...
1 2 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS 99395mvp9_38 12/13/05 12:24 PM Page 12
|
From page 19... ...
Freight companies, railroads, airlines, and public transit agencies have been using performance measures for many years. External groups, especially state legislatures and governors, have become involved when there is a perceived lack of accountability at state transportation agencies.
|
From page 20... ...
Measures for performance-based contracting will also be developed, along with benchmarking. 1 4 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS Programming and Project Delivery Develop Financial Policies Monitor Performance Establish Policy and Plans FIGURE 1 Performance measurement cycle of the Florida Department of Transportation.
|
From page 21... ...
in the San Francisco Bay Area with the use of performance measures involving multiple jurisdictions: a local pavement management system and a ferry boat fare box recovery measure. The pavement management system in the San Francisco Bay Area is a successful example of integrating performance measures across multiple jurisdictions.
|
From page 22... ...
There is also discussion of establishing a fare box recovery ratio and other operational performance measures for new transit projects funded through a recent increase in tolls. It is too soon to tell for certain how the ferry boat fare box recovery measure will work out.
|
From page 23... ...
Finally, I will highlight the importance of communicating the results of performance monitoring to key external audiences, especially the public, the media, and policy makers. FHWA uses congestion measures focusing on the average duration of congested travel, the travel time index, and the buffer index.
|
From page 24... ...
It is important to provide a message that resonates with the audience you are trying to reach. CEO PERSPECTIVE: WHAT DO YOU DO WITH THE MONEY David Ekern We have heard a number of common themes from speakers this afternoon related to the growing use of performance measures by state transportation agencies.
|
From page 25... ...
Our challenge was to present these trends and the department's performance measures to the legislature in a meaningful way. Rather than just present the department's budget in the normal way, we identified the six major products or services the department provides and highlighted the funding associated with each.
|
From page 26... ...
Robert Romig, Florida Department of Transportation Jeff Price, Virginia Department of Transportation Brian Smith, California Department of Transportation Gregory Selstead, Washington State Department of Transportation PERFORMANCE CHALLENGE: LINKING PERFORMANCE-BASED PROGRAM DEVELOPMENT AND PROJECT DELIVERY Steven M Pickrell Good morning.
|
From page 27... ...
The Washington State Department of Transportation's Gray Notebook provides one example of sharing 2 1PERFORMANCE-BASED PROGRAM DEVELOPMENT AND DELIVERY Co m po ne nt 1 Pr og ra m D ev el op m en t Co m po ne nt 2 Pr oje ct D el iv er y Co m po ne nt 3 Sy st em M on ito rin g a n d Re po rti ng System Performance Measures to Monitor Progress on Meeting Goals/Objectives Performance Targets for Planning/ Programming Given Resource Availability Specific Set of Programs and Projects with a Defined Budget, Schedule, Scope Performance Measures to Track Delivery Result = System Performance Program Development and Project Selection Program/Project Implementation Goals/Objectives FIGURE 1 Performance-based management structure.
|
From page 28... ...
The objectives are 2 2 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS LONG RANGE Florida Transportation Plan Goals and Objectives INTERMEDIATE RANGE Short-Range Component of the FTP Strategic Goals Short-Range Objectives Program Resource Plan Operating Policies SHORT RANGE Work Program List of Projects and Activities FIGURE 2 Funding directed by policy and program objectives identified in legislation and Florida Department of Transportation plans.
|
From page 29... ...
The following websites provide additional information on the various programs and policies in Florida: • Agency overview: www.dot.state.fl.us/financial planning/AGENCY_OVERVIEW.pdf • 2020 Florida Transportation Plan: www.dot.state. fl.us/planning/ftp2020/default.htm • Short-Range Component and Annual Performance Report: www.dot.state.fl.us/planning/policy/pdfs/src.pdf • Work Program Instructions: www.dot.state.fl.us/ programdevelopmentoffice/Development/WP_instruc tions.shtm • Florida Transportation Commission Performance Measures: www.ftc.state.fl.us/performance%20and% 20production%20review%200102.pdf 2 3PERFORMANCE-BASED PROGRAM DEVELOPMENT AND DELIVERY ASSET MANAGEMENT Data Collection Performance Measures Management Systems Decision Support Tools FIGURE 3 Components of Florida's asset management program.
|
From page 30... ...
In 1986 we tracked 276 performance measures, none of which were paramount. While we still track performance in many areas, the focus under the current commissioner is on the core measures of ontime and on-budget program delivery.
|
From page 31... ...
2 5PERFORMANCE-BASED PROGRAM DEVELOPMENT AND DELIVERY FIGURE 4 Virginia Department of Transportation dashboard watch list.
|
From page 32... ...
My office, which conducts process improvement studies, has seen a dramatic increase in requests for new studies. It is not enough simply to develop a performance measure, set a target, and tell people that 2 6 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS FIGURE 5 Virginia Department of Transportation report card.
|
From page 33... ...
LINKING SYSTEM PERFORMANCE MEASUREMENT TO DECISIONS -- PROGRESS SINCE THE FIRST NATIONAL CONFERENCE Brian Smith At the conference 3 years ago, Caltrans's then Chief Deputy Tony Harris described California's activities related to performance measures. I will not repeat his comments related to California's use of performance measures to drive programming decisions.
|
From page 34... ...
The California Transportation Plan 1 Performance Measures Module developed in 1997 and 1998 included nine outcomes, indicators, and decision-making linkages. It established the need to integrate performance measures and long-range planning supporting decisions.
|
From page 35... ...
Keeping measures outcomebased and ensuring that the right projects and services are delivered efficiently and effectively are also critical. THREE DIMENSIONS OF PROJECT DELIVERY MEASUREMENT AND REPORTING Gregory Selstead I appreciate the opportunity to provide additional information on the use of performance measures at the Washington State Department of Transportation.
|
From page 36... ...
3 0 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS 99395mvp9_38 12/13/05 12:24 PM Page 30
|
From page 37... ...
I thank the two coauthors of the resource paper, Frances Harrison and Anita Vandervalk of Cambridge Systematics, Inc., for their assistance. The purpose of this presentation is to highlight some of the technical issues associated with using existing data and tools for performance measurement at transportation agencies.
|
From page 38... ...
Examples of mobility measures include the travel time index, which focuses on the average congestion level; the planning time index, which focuses on arriving on or ahead of schedule 19 times out of 20; and the average travel rate, which was recommended by NCHRP Report 398. A reliability challenge is processing extensive data flows to determine the locations, magnitudes, and durations of disruptions to expected travel time patterns.
|
From page 39... ...
Data elements must have a data owner, a data element definition, a schedule for updating, and a fixed amount of precision. Precise data definitions are particularly important where data from secondary sources are being used to derive performance measures.
|
From page 40... ...
My presentation will cover some of the lessons learned, as well as the historical perspective and the current experience with the use of different techniques for presenting information on performance measures. I will describe the approaches being used at four state transportation agencies.
|
From page 41... ...
3 5DATA AND TOOLS REQUIRED TO SUPPORT DECISION MAKING FIGURE 1 Delaware INFORM home page. FIGURE 2 Delaware INFORM layers.
|
From page 42... ...
3 6 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS FIGURE 3 Example of INFORM drill down capabilities. FIGURE 4 Example of Vermont Agency of Transportation's route selection tool.
|
From page 43... ...
Data on accidents, pavement condition, bridge condition, and a number of other elements are included. GRIP provides drill down capabilities for users.
|
From page 44... ...
Weekend congestion associated with special 3 8 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS 99395mvp39_69 12/13/05 12:29 PM Page 38
|
From page 45... ...
USING PERFORMANCE MEASURES TO IMPROVE HIGHWAY PROGRAM EFFECTIVENESS Mark Wolfgram I appreciate the opportunity to participate in this session and to share some of the Wisconsin Department of Transportation's experiences with the use of performance measures for highway program effectiveness. My comments will touch on some of the topics discussed during this morning's session and will describe what we have accomplished since the first performance measures conference in 2001.
|
From page 46... ...
Finally, since our last conference in 2001, several program performance measures were developed for program development staff in the central and district offices. The measures are focused on ensuring that funding is actually used at the right time, at the right place, to do the right thing.
|
From page 47... ...
It has been an excellent tool for developing, analyzing, and monitoring performance measures. The system was a great benefit in the recent development of Access Ohio, the long-range transportation plan.
|
From page 48... ...
The international performance measures scan occurred in spring 2004. The primary purpose of the scan was to learn about the experiences of other countries in using performance measures and performance management in planning and programming decision making for all modes of transportation.
|
From page 49... ...
Five priority topics per year are selected for scanning tours. The program began with a technical orientation, but the scanning tours conducted to date have covered a wide range of topics, including the use of performance measures.
|
From page 50... ...
4 4 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS 99395mvp39_69 12/13/05 12:29 PM Page 44
|
From page 51... ...
We have found that three-dimensional maps like these are an excellent tool to communicate with policy makers and the public. 4 5INTERNATIONAL USE OF PERFORMANCE MEASURES Time lost due to congestion (million person -hours)
|
From page 52... ...
I look forward to learning more about the approach in use here in the United States during this conference. 4 6 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS ( National Route 16 0.0 200.0 400.0 600.0 800.0 1,000.0 1,200.0 1,400.0 Ti m e lo ss d ue to c on ge st io n/ km (1, 00 0 p ers on ho ur/ ye arl y k m)
|
From page 53... ...
Transportation agencies in the countries we visited seem to have used performance measures more effectively in their highway safety programs than we have. New Zealand and Australia have seen dramatic improvements in highway safety performance as a result of this focus.
|
From page 54... ...
There were a number of things that I found of interest in the site visits and in our meetings with agency personnel. First, performance measures are being used by transportation agencies and are being monitored in the countries we visited.
|
From page 55... ...
The second observation relates to the power of performance measures in educating both elected officials and the public. I would take that one step further.
|
From page 56... ...
Performance measures addressing environmental quality tend to be similar to those used in the United States. While there is an interest in develop5 0 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS 99395mvp39_69 12/13/05 12:29 PM Page 50
|
From page 57... ...
The state governments or parliaments in Australia also provide direction to state transportation agencies. These general policies are used to develop more specific agency performance measures.
|
From page 58... ...
It is also important to remember that congestion performance measures are of interest to a wide range of groups, including the public, travelers and shippers, the business community, elected and appointed officials, agency leaders, and agency staff. We need to be able to communicate with all these groups.
|
From page 59... ...
The buffer index, which is the difference between the average travel time and the planned travel time, can be used in a number of ways. We can also match the results of customer satisfaction surveys with reliability performance measures.
|
From page 60... ...
Thank you. TOOLS FOR UNDERSTANDING CUSTOMERS IN MAKING INVESTMENT DECISIONS Randy Halvorson I will start my presentation with a story about linking customer satisfaction, performance measures, and incident management, and communicating with elected officials.
|
From page 61... ...
An example of a goal is achievement of an average travel speed of 60 mph on 88 percent of miles of high-priority interregional corridors by 2005. In developing performance measures by using customer information, the importance of transportation services to our customers should be understood first; only then should we examine how satisfied they are with these services.
|
From page 62... ...
This measure covers several 5 6 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS Lo w H ig h Sa tis fa ct io n Ra tin gs Possible Overcommitment by the Department: Plants, grasses, and flowers by roadside look good Customer Irritants: Weeds on roadside eliminated Areas for Improvement: Road surfaces smooth and comfortable to drive on Road stripes, markings clearly visible Strengths of the Organization: Roads clear of ice and snow High Importance Rating Low • • • • • FIGURE 5 Customer satisfaction with roadway maintenance activities. 18 H ou rs 16 14 12 10 8 6 4 2 0 98 –9 9 99 –0 0 00 –0 1 01 –0 2 Winter Season 02 –0 3 03 –0 4 04 –0 5 05 –0 6 10.3 Hrs.9.0 9.0 10.5 7.3 5.0 7.8 4.1 Hrs.Ta rg et R an ge 03 –0 4 Significantly Under Expectation Under Expectation On Target Over Expectation FIGURE 6 Snow and ice removal: statewide annual average hours to achieve bare lanes after a snow or ice event.
|
From page 63... ...
As illustrated in Figure 8, we have been below our target of 60 percent of customers regarding us as a reliable source of information for the past 4 years. The department uses market research information on customer satisfaction indirectly to validate some of its performance measures.
|
From page 64... ...
This figure accounted for 21.3 percent 5 8 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS 1994 1996 Year 2000 2001 2002 2003 8.4 8.2 8.0 7.8 7.6 7.4 7.2 7.0 7.0 6.8 6.6 6.4 6.2 6.0 M ea n R at in g P os iti ve N eu tra l 6.7 6.6 6.5 6.2 6.3 6.4 FIGURE 10 Customer satisfaction with road smoothness: rating on a scale of 1 to 10. (Source: Office of Investment Management, Market Research, Minnesota Department of Transportation, February 2004.)
|
From page 65... ...
To develop state freight performance measures, we began by analyzing current freight system performance in terms relating to customer requirements. Here are two examples of how the Washington State freight system serves regional economies.
|
From page 66... ...
As I noted in my introductory comments on Sunday afternoon, the first observation is that the use of performance measures has become widespread in many transportation agencies. We have learned a great deal about the application of performance measures since the first conference in 2000.
|
From page 67... ...
In closing, I would like to thank all of the sponsors -- the Transportation Research Board (TRB) , the Federal Highway Administration (FHWA)
|
From page 68... ...
Craig Secrest, Wilbur Smith Associates Gregory Selstead, Keith Cotton, and Barbara Ivanov, Washington State Department of Transportation Robert Romig and Brian Watts, Florida Department of Transportation Leonard Evans, Ohio Department of Transportation Mark C Larson and Randy Halvorson, Minnesota Department of Transportation Anthony Pietropola, GeoDecisions Kent Barnes and William Cloud, Montana Department of Transportation Robert Winick, Motion Maps, LLC Jeff May, Denver Regional Council of Governments Louis H
|
From page 69... ...
• Examine how changes in the governor's office or the state legislature have influenced the use of performance measures for planning and project delivery by the state transportation agency and the specific measures used by different agencies. Provide approaches state transportation agencies may consider in communicating performance measures and performance-based management to new elected officials.
|
From page 70... ...
Environmental groups were perceived to have a powerful influence over the project development process, and failure to develop project solutions that respond appropriately to their concerns may create barriers to attaining mobility, accessibility, and other performance targets. Some participants suggested that measurement results for mobility and other performance measures can be skewed because of events and activities that transportation agencies cannot control.
|
From page 71... ...
Research Needs • Develop strategies and best practices for engaging external audiences and managing their expectations, specifically in creating and implementing performance measures and performance targets. • Develop strategies and best practices for educating external audiences about the connection between policy challenges and performance measurement needs.
|
From page 72... ...
6 6 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS 99395mvp70_128 12/13/05 12:36 PM Page 66
|
From page 73... ...
HOW TO SET PROGRAM TARGETS AND MAKE TRADE-OFF DECISIONS Robert Romig, Moderator Brian Watts, Recorder Topics Discussed Participants discussed the issues typically associated with setting performance measure targets and making trade-off decisions. A major focus of discussion was performance measures addressing preservation versus adding capacity.
|
From page 74... ...
Participants discussed possible trucking and freight performance measures; sources of data on freight movements; and coordination among public agencies, trucking companies, and businesses. Several participants noted that it is often difficult to obtain data from the trucking industry.
|
From page 75... ...
WORKING WITH LEGACY DATABASES AND MANAGEMENT SYSTEMS Leonard Evans, Moderator Mark C Larson, Recorder Topics Discussed Participants in this breakout group discussed the issues and opportunities associated with legacy databases and their use with transportation performance measures.
|
From page 76... ...
STANDARDS FOR DATA AND PERFORMANCE MEASURES William Cloud, Moderator Robert Winick, Recorder Topics Discussed Participants in this breakout session discussed issues and opportunities associated with standards for data and performance measures. They discussed the experiences and challenges with different approaches.
|
From page 77... ...
PERFORMANCE FORECASTING: FINANCIAL ISSUES, SCALE, AND IMPLEMENTATION TRACKING Jeff May, Moderator George Gerstle, Recorder Topics Discussed Participants discussed a variety of financial, scale, and implementation tracking issues associated with performance measures. Issues associated with the scale of performance measures and concerns related to financial forecasting were discussed.
|
From page 78... ...
• What can agency managers do to address any linear referencing and dynamic segmentation barriers that constrain expanded use of performance measures? • What is the relationship of linear referencing to performance-based decision support?
|
From page 79... ...
Topics included current performance measures, possible barriers, expansion of the use of congestion and reliability measures, and other possible measures. Participants also identified areas for further research.
|
From page 80... ...
• Participants discussed measures related to congestion and reliability that need further examination. • Ensure that whenever possible, performance measures are comparable across regions, states, and time frames.
|
From page 81... ...
ENVIRONMENTAL Michael Meyer, Moderator Topics Discussed Participants in this breakout session discussed a variety of environmental issues and possible performance measures for addressing these concerns. Topics included human health, ecosystem health, environmental quality, and land use.
|
From page 82... ...
A framework may be missing for transportation agencies to evaluate transportation's role in ecosystem health issues. Participants discussed possible performance measures and how to measure and monitor ecosystem health.
|
From page 83... ...
Larson, Recorder Topics Discussed Participants in this breakout session discussed techniques for obtaining information on customer satisfaction with the products and services provided by transportation agencies. The use of customer satisfaction information with performance measures was also discussed.
|
From page 84... ...
Potential freight performance measures include freight mobility indicators, bottlenecks affecting freight movements, and freight travel and delay times. Participants discussed potential relationships between freight and economic growth.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.