National Academies Press: OpenBook
« Previous: Preface
Page 7
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 7
Page 8
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 8
Page 9
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 9
Page 10
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 10
Page 11
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 11
Page 12
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 12
Page 13
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 13
Page 14
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 14
Page 15
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 15
Page 16
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 16
Page 17
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 17
Page 18
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 18
Page 19
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 19
Page 20
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 20
Page 21
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 21
Page 22
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 22
Page 23
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 23
Page 24
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 24
Page 25
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 25
Page 26
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 26
Page 27
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 27
Page 28
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 28
Page 29
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 29
Page 30
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 30
Page 31
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 31
Page 32
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 32
Page 33
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 33
Page 34
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 34
Page 35
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 35
Page 36
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 36
Page 37
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 37
Page 38
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 38
Page 39
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 39
Page 40
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 40
Page 41
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 41
Page 42
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 42
Page 43
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 43
Page 44
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 44
Page 45
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 45
Page 46
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 46
Page 47
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 47
Page 48
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 48
Page 49
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 49
Page 50
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 50
Page 51
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 51
Page 52
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 52
Page 53
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 53
Page 54
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 54
Page 55
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 55
Page 56
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 56
Page 57
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 57
Page 58
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 58
Page 59
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 59
Page 60
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 60
Page 61
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 61
Page 62
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 62
Page 63
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 63
Page 64
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 64
Page 65
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 65
Page 66
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 66
Page 67
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 67
Page 68
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 68
Page 69
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 69
Page 70
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 70
Page 71
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 71
Page 72
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 72
Page 73
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 73
Page 74
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 74
Page 75
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 75
Page 76
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 76
Page 77
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 77
Page 78
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 78
Page 79
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 79
Page 80
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 80
Page 81
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 81
Page 82
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 82
Page 83
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 83
Page 84
Suggested Citation:"Sessions and Topics." National Academies of Sciences, Engineering, and Medicine. 2005. Performance Measures to Improve Transportation Systems: Summary of the Second National Conference. Washington, DC: The National Academies Press. doi: 10.17226/13658.
×
Page 84

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

SESSIONS AND TOPICS 99395mvp9_38 12/13/05 12:24 PM Page 1

99395mvp9_38 12/13/05 12:24 PM Page 2

3WELCOME Tony Kane On behalf of the Transportation Research Board (TRB), the American Association of State Highway and Trans- portation Officials (AASHTO), the Federal Highway Administration (FHWA), the Federal Transit Adminis- tration (FTA), and other sponsors, it is a pleasure to welcome you to the Second National Conference on Per- formance Measures. The first conference was held in 2000. This conference focuses on the ultimate purpose of performance measures: to improve transportation ser- vices for our customers. The opening session this after- noon highlights the objectives of this conference and summarizes the state of the practice in the use of perfor- mance measures by transportation agencies at different levels of government throughout the country. Representatives from some 20 state departments of transportation are participating in this conference. An equal number of personnel from local and regional agen- cies are attending. Performance measurement and per- formance-based management aids the decision-making process at all levels of government. Our partners in the private transportation sector also use performance-based management—for example, to ensure the timely delivery of freight and the efficient movement of travelers. We have seen numerous changes and advances in the application of performance measures within transportation agencies since the first conference in 2000. Legislation and policy directives at the fed- eral, state, and local levels have influenced the use of performance-based management techniques. Perfor- mance measures are also being used by transporta- tion organizations throughout the world. The Monday night session will highlight examples of international applications of performance-based management. Representatives from a number of state departments of transportation will discuss the evolving role perfor- mance measures are playing in the decision-making process during sessions throughout the conference. Dur- ing my tenure at FHWA and the past 4 years at AASHTO, I have seen advances in the state of the prac- tice with performance-based management. State depart- ments of transportation have a strong interest in sharing their experiences and in learning from the experiences of others. At AASHTO, we are developing a new strategic plan for 2005 through 2010. Performance measurement plays a key role in the new strategic plan, which was developed by a 15-member committee made up of sec- retaries of state departments of transportation. The plan will be voted on by the board of directors at the annual meeting in Philadelphia this September. The Conference Planning Committee has done an excellent job of organizing interesting sessions. I hope you will participate actively in the conference and share your thoughts and ideas on performance measures. Opening Session Tony Kane, Moderator, American Association of State Highway and Transportation Officials Lance A. Neumann, Cambridge Systematics, Inc. Gloria Shepherd, Federal Highway Administration Theodore H. Poister, Georgia State University Douglas MacDonald, Washington State Department of Transportation 99395mvp9_38 12/13/05 12:24 PM Page 3

CONFERENCE OBJECTIVES Lance A. Neumann It is a pleasure to welcome you to the Second National Conference on Performance Measures on behalf of the Conference Planning Committee and the TRB Commit- tee on Performance Measurement. I would like to rec- ognize Sandy Straehl from the Montana Department of Transportation, who served with me as cochair of the Conference Planning Committee. Many of you participated in the first conference, which focused primarily on defining the concept of performance measurement, identifying potential performance mea- sures, and promoting the use of performance-based man- agement within transportation agencies. In the 4 years since the first conference, we have seen widespread use of performance measures by state departments of trans- portation and a continuing interest in this topic at all levels of government. We have learned a great deal over the past 4 years through projects and studies sponsored by FHWA, TRB, the National Cooperative Highway Research Program, state departments of transportation, and other agencies. As a result, the planning committee believed that this conference should focus on the implementation and use of performance-based management and how we moni- tor progress to ensure that these programs do make a difference in the delivery of transportation products and services. The planning committee wanted to highlight the wealth of experience in the use of performance measures at this conference and especially to share lessons learned. The workshops this morning provided a great starting point for sharing experiences and learning from each other. This conference focuses on the implementation of per- formance measures as a practical management tool and on the steps needed to accomplish this goal. As with the first conference, the topics to be covered over the next 2 days are both broad and comprehensive. The sessions focus on a series of themes the planning committee identified as important in promoting the implementation and ongoing use of performance measures. The workshops this morn- ing addressed the basics of performance measurement and communications. This opening session highlights the use of perfor- mance measures at state transportation agencies and presents the views of a senior executive on the use of performance measures in a political decision-making environment. The second session this afternoon will explore the influence of introducing performance mea- sures on internal organizational relationships, as well as external institutional relationships and partnerships. The first session on Monday will examine linking per- formance management in plan development, program development, and budgeting with performance manage- ment in program and project delivery. Experience indicates that this link is critical to the successful use of performance- based management. Potential issues and opportunities associated with the data needed for performance measures will be addressed at the second session on Monday. Speakers at the Monday night session will share information on the international experience with per- formance-based management. The conference will close on Tuesday with a session on performance areas that are difficult to measure. You will also have the opportunity to share your experiences and views during the breakout sessions on Monday and Tuesday. These sessions focus on a variety of topics and issues and will add to the breadth of the conference. You are encouraged to participate in the breakout sessions that best meet your interests. FHWA and FTA helped fund this conference. The members of the planning committee put forth creative ideas, hard work, and dedication in organizing this con- ference. Kim Fisher, Freda Morgan, and other TRB staff provided outstanding support. Finally, Katherine Turn- bull of the Texas Transportation Institute (TTI) will be compiling the conference proceedings. In summary, the conference objectives include defin- ing the state of the practice and acknowledging recent work in the use of performance measures, sharing expe- riences and resources, and identifying key areas that need further research or additional peer exchange. I encour- age you to participate in all parts of the conference actively. I look forward to productive discussions. Thank you for participating in this important confer- ence. I look forward to hearing your thoughts and ideas in helping to advance the state of the practice in the use of performance-based management. WELCOME AND FEDERAL PERSPECTIVE Gloria Shepherd It is a pleasure to participate in this conference on behalf of FHWA. We at FHWA have been pleased to work with many other partners in organizing and supporting this important conference. I should recognize my colleagues at FHWA who helped with the conference planning activities and who are participating in the session. Jeff Lindley, Director of Operations, willingly supported funding the conference. Jeff will be speaking at the second session this after- noon. Dave Ginger is participating, and Mike Halladay from Safety will be speaking Monday night. I also rec- ognize Bob Arnold, who was on the international per- formance measures scan, and the other FHWA staff participating in the conference. 4 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS 99395mvp9_38 12/13/05 12:24 PM Page 4

As you can tell both by FHWA’s willingness to be a funding partner and by the number of staff participating in this conference, performance measures are an impor- tant subject at FHWA. We assisted in sponsoring a perfor- mance measures peer exchange in South Carolina this past May. Representatives from a number of states shared their experiences in the use of performance measures at the peer exchange. One of the areas of interest was implementing performance measures in specific program areas, includ- ing safety, asset management, and operations. Discussions focused on both potential performance measures and implementation strategies for incorporating performance- based management into state transportation agencies. The summary of this peer exchange has been posted by TRB as an e-circular at trb.org/publications/circulars/ ec073.pdf. Numerous stakeholder groups are also inter- ested in the application of performance measures by transportation agencies. Among the stakeholders are public officials, interest groups, and environmental organizations. They are interested in how transporta- tion agencies allocate public funds and make decisions on project priorities. Performance-based management can help state departments of transportation communicate needs and priorities to the public and to decision makers. Commu- nicating effectively with these groups is especially important in times of limited resources. Requests for additional funding, bonding authority, and other financ- ing options have been considered in many states recently. Clearly communicating how these funds will be used and tracking progress on promised projects appear to be important factors in successful initiatives. We have an obligation to ourselves as transportation professionals and to the public we serve to spend funds effectively and efficiently. The use of performance mea- sures helps ensure that transportation agencies follow up on commitments. FHWA also helped sponsor a roundtable discussion of performance measures and statewide transportation planning in Washington, D.C., in October 2003. Issues discussed at the roundtable included strategic planning; measurement of the cost and performance of alternative projects; and techniques to compare, prioritize, and select alternative investments. Modal investment strate- gies, monitoring of the performance of transportation organizations and multimodal transportation systems, and techniques for communicating performance mea- sures to stakeholders were also discussed. The issue of how we communicate with the public and with policy makers is important. These groups want to know how public funds are being spent. They want to know what improvements or better services they will receive from increased investments in the transportation system. At FHWA, we are interested in working with you to address these issues and other topics of concern. We need to look at how performance measures can improve the safety and the efficiency of our transportation sys- tem. We are also interested in the use of performance measures to enhance the planning process. We are inter- ested in how well the transportation planning process works, how well it informs the public, and how we can improve long-range plans. We at FHWA are pleased to participate in this confer- ence. We look forward to continuing to partner with all of you to enhance the use of performance measures with all elements of the transportation system. Thank you. STATE OF THE PRACTICE OF PERFORMANCE MEASUREMENT Theodore H. Poister As one who has had a window on this field for many years, it is a privilege to be able to participate in this conference. It is clear that interest in the use of performance measures has grown tremendously over the past 20 years. At this point performance measurement has become an integral part of the way many state departments of transportation do business. It may be helpful at the outset to remember the over- all governmental context within which performance measures are applied. The federal Government Perfor- mance and Results Act of 1993 requires all federal agen- cies to develop and use performance measures. Most states have some type of executive or legislative man- date to use performance-based management, and many state transportation agencies have been using perfor- mance measures for a number of years. There are also initiatives at the local government level, although they are not as far reaching. Phoenix, Arizona; Dallas, Texas; Charlotte, North Carolina; and Dayton, Ohio, are just a few cities that have been using performance measures for some time. Little research has assessed whether management matters and whether performance measures matter. We believe that performance measures make a difference, but research to support this conclusion is lacking. Two recent studies focus on the impacts of strategic planning and management. The first examined police depart- ments throughout the country, and the second focused on fire departments in New York State. Both studies (1, 2) concluded that management does make a difference and that agencies with strong management practices, including performance measures, do perform better than agencies without strong management practices. One of the more ambitious research projects is the Government Performance Project (GPP), which was con- ducted by a group of university researchers and Govern- ing magazine. Complete information with regard to the 5OPENING SESSION 99395mvp9_38 12/13/05 12:24 PM Page 5

purpose, approach, and results is available at gppon- line.org. Through an extensive survey and follow-up site visits and interviews, the project graded all 50 state gov- ernments, a sample of federal agencies, and 25 local gov- ernments over a 3-year period. The agencies were graded on financial management, human resources, information technology, capital management, and managing for results. Planning, goal setting, and management and eval- uation were included in the assessment. (See additional information at gpponline.org.) As you might expect, grades for the various states var- ied widely. A second round of the GPP focusing on state departments of transportation and environmental protec- tion programs in particular, in addition to general state government, is under way. All the state transportation departments will be involved in this effort. I think transportation agencies are on the leading edge in the application of performance measures. At the federal level, transportation agencies participated in testing many planning and measurement efforts. At the state level, the transportation agencies are frequently the leading agencies in applying performance-based management. At the local level, transit agencies have used performance measures for many years. For exam- ple, the city of Charlotte, North Carolina, pioneered the use of the balanced scorecard in the public sector, and the Charlotte Department of Transportation was the city agency selected to pilot test the balanced scorecard application. There has been substantial growth in the develop- ment and use of performance measures by transporta- tion agencies during the past 20 years, especially in the past 5 to 10 years. Currently, transportation agencies vary widely in the approaches used and the level of expertise within the agencies. Recent trends and the current status of performance measures can be examined from a number of perspec- tives. We can focus on what is being measured, how per- formance is being measured and reported, and how performance measures are being used in the decision- making process. We can first look at what state transportation agen- cies are measuring. We are seeing a move toward more comprehensive approaches in the application of perfor- mance measures. We are also seeing performance mea- sures being used as part of transportation agencies’ overall strategic management processes. A number of transportation agencies use the balanced scorecard application, and many of them have adapted or modi- fied the original balanced scorecard model to meet their own particular needs. (See the resource paper “Perfor- mance Measurement in Transportation: State of the Practice” in these proceedings for examples of balanced scorecard models.) Figure 1 is an example of a logic model developed out of the tradition of evaluation research. It provides an example of how evaluation research maps programs and services, system performance, and impacts. Imme- 6 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS Program Construction Maintenance Safety Operations Public Transportation Outputs Projects Completed Lane Miles Bridges Built Miles Resurfaced Repairs Made Treatments Applied Projects Completed Turn Lanes Added Stripes Painted Messages Displayed Incidents Cleared Signals Timed Vehicle Hours Vehicle Miles Seat Miles Immediate Outcomes > Capacity > Connectivity > Condition Smoother Pavements < Hazards More Efficient Operation > Coverage < Headways Intermediate Outcomes < Congestion < Travel Times > Convenience > Ride Quality < Operating Expense < Crashes < Injuries < Fatalities < Congestion < Delays < Crashes < Waiting > Ridership > Convenience Longer-Term Outcomes Mobility Quality of Life Economic Development Environmental Enhancement Community Development System PerformanceProgram & Service Delivery Impacts FIGURE 1 Example of transportation program logic model. 99395mvp9_38 12/13/05 12:24 PM Page 6

diate, intermediate, and longer-term outcomes are all identified and monitored. While the outputs focus on agency activities and service delivery processes, the immediate outcomes tend to focus on conditions, such as pavement smoothness. The intermediate outcomes focus more on system performance, such as travel times and safety. The longer-term outcomes address broader impacts. This model is an example of the holistic approach being used at many transportation agencies. Many performance measures focus on transportation agency operations. Incident management represents one function that has come to the forefront recently. State departments of transportation realize that clearing inci- dents quickly is important in maintaining traffic flow and minimizing secondary accidents. For example, the Maryland State Highway Administration’s CHART Program measures incident duration, initial response time, and overall recovery time. Performance measures at many state departments of transportation address program delivery. The Virginia Department of Transportation’s new strategic plan focuses on the effective delivery of annual state trans- portation improvement plans. Many states have experi- enced increases in funding, which result in more programming activities. That generates additional pres- sure to deliver the program. For instance, Georgia’s bud- get will roughly double as a result of the governor’s new Fast Forward program. Thus, on-time and on-budget performance measures continue to be a major focus at many state transportation agencies. Cycle times for overall projects, as well as particular elements, are also being considered. The Pennsylvania Department of Transportation has 10 teams examining all aspects of program delivery to streamline the process, and they are attempting to develop performance measures in each of those areas. Performance measures at many state departments of transportation continue to address system condition. Pavement condition, pavement roughness, and bridge condition are common performance measures at most state departments of transportation. Safety performance measures also continue to be important. Typical safety performance measures focus on crashes, injuries, and fatalities. Other elements such as pedestrian and bicycle accidents and at-grade railroad crossing accidents are measured in some states. Improving safety continues to be a high priority of FHWA as well as state and local transportation agencies. Applying performance measures to traffic flow and congestion is a growing area of interest. Volume–capac- ity ratios have traditionally been used to measure con- gestion. The annual urban mobility report published by TTI examines performance measure data for 75 urban areas in the country. The reports present the travel time index, percentage of congested vehicle miles traveled (VMT), delay per person, percentage of congested lane miles, cost of congestion, and percentage of congested time for each area. FHWA’s urban congestion report examines monthly data for 10 metropolitan areas that are instrumented to provide real-time data. Performance measures used in that report include the travel time index, the buffer index, the average duration of congested travel per day, and the percentage of congested travel. Some agencies apply performance measures to envi- ronmental and economic factors. Environmental indica- tors may address acres of wetlands replaced, acres of reforestation, storm water enhancements completed, and air quality noncompliance days in urban areas. Economic development indicators may focus on jobs created or retained through initiatives where trans- portation is a contributing factor. Many state departments of transportation conduct regular customer satisfaction surveys. Regular surveys of motorists or the public at large are conducted in Min- nesota, New Mexico, Illinois, Kentucky, Pennsylvania, Ohio, Georgia, and many other states. Florida uses a mix of survey methods for different customer segments, such as resident travelers, visiting travelers, disabled travelers, property owners, and elected government offi- cials. The Minnesota Department of Transportation is one of the few state departments of transportation with an in-house market research group. The department conducts regular surveys of residents and other user groups. The Pennsylvania Department of Transporta- tion conducts an annual highway administration cus- tomer survey. This program includes a large sample mail-out survey to obtain reliable data at the county level. The findings are presented at the county and dis- trict levels on an annual basis. The information is used in the development of an annual work program for each district. Performance is measured and reported by state depart- ments of transportation in a variety of ways. Performance measures focus on resources and workload, outputs, ser- vice quality, efficiency, and productivity. Other measures may examine outcomes, cost-effectiveness, benefit–cost ratios, return on investment, and life-cycle costs. Performance indicators are specified in a number of ways. How performance measures are specified is impor- tant. For example, performance measures addressing highway safety often include traffic fatalities per 1,000,000 VMT and traffic fatalities per 100,000 resi- dents. The results of these two measures may suggest dif- ferent problems and solutions. Pavement condition measures typically include the percentage of lane miles in good condition and the percentage of VMT on lane miles in good condition. Focusing on just one of these measures would promote different types of project programming and investment decisions. 7OPENING SESSION 99395mvp9_38 12/13/05 12:24 PM Page 7

Many indices are used with performance measures. Examples include pavement rating scales, productivity indices, and customer satisfaction indices. The Florida Department of Transportation focuses on 11 key mea- sures tied to strategic objectives and executive board ini- tiatives. These measures are reviewed on a monthly basis. Some of these measures, including work program delivery, employee satisfaction, system condition, sys- tem performance, and customer satisfaction are weighted indices combining numerous indicators. The Ohio Department of Transportation’s organiza- tional performance index (OPI) focuses on the perfor- mance of 12 districts and 88 county-level maintenance units. The OPI consists of measures in eight functional areas, most of which are indices themselves. These mea- sures are also combined into a single index of overall department performance. The aggregate OPI provides a reading of overall performance at the executive level. Where there is slippage, managers can drill down to look at specific measures in individual districts to identify problems and formulate corrective actions. Benefit–cost ratios are also used as a performance measure by some state departments of transportation. Benefit–cost ratios are typically estimated for projects in the planning stage. However, VicRoads in the state of Victoria, Australia, tracks aggregate benefit–cost ratios after the fact for all projects completed during a given year. The agency monitors an achievement index for all completed projects. It examines the actual benefit–cost ratios after 2 years in relation to the benefit–cost ratios originally estimated. State transportation agencies use a number of meth- ods and techniques to report performance measure data. These formats include scorecards, dashboards, and roll- up and drill-down features. Performance measures are reported both internally and externally. State trans- portation agencies communicate performance measures to the public and other external stakeholders through reports, updates, and Internet sites. For instance, the Maryland Department of Transportation publishes an Annual Attainment Report of Transportation System Performance. The Washington State Department of Transportation issues a Measures, Markers, and Mile- posts report. The Virginia Department of Transporta- tion uses a quarterly report card or project dashboard on the department’s Internet site. Many states use performance measures for integrat- ing strategic planning and management. Pennsylvania, Minnesota, Kentucky, Virginia, Maryland, California, Illinois, South Carolina, New Mexico, and Georgia all emphasize performance measures linked to strategic planning processes. Performance measures are also used in the trans- portation systems planning process in some states. Performance-based planning uses performance mea- sures to tie planning to goals, standards, and targets for system performance. Ohio, Minnesota, and Pennsylvania emphasize this approach. Performance measures are used in some states for programming and project selection. These efforts use performance measures to evaluate alternatives and assess trade-offs in costs and performance among com- peting projects. They typically focus on pavement, bridge, safety improvement, and congestion mitigation projects. Montana has a performance programming process, and New York uses an asset management pro- gram with reduction in excess user costs as a common performance criterion. Other states use performance measures in program and project delivery. Management information systems track the achievement of project milestones and the bud- get status of individual projects to monitor on-time and on-budget measures, usually on a district and statewide basis. Many states, including Virginia, Washington, and California, report project status data to the public on their Internet sites. Operations management represents another area for applying performance measures. One approach applies performance-based management for districts, divisions, and organizational units through goal setting and mea- surement with regard to business plans, program plans, work programs, and operating plans. Performance measures may also be used internally. One example of an internal program is employee perfor- mance planning and evaluation. Examples of this approach include South Carolina’s accountability sys- tem, the California Department of Transportation’s per- formance agreements, and Ohio’s career professional service. Performance-based budgeting is also used in some states. Examples of performance-based budgeting include Colorado’s investment strategy, Minnesota’s activities-based budgeting focused on project and service lines, and New Mexico’s program budget. The use of performance measures in contract man- agement appears to be increasing at many state trans- portation agencies. Performance measures may address contract design, construction, maintenance, and ser- vices. Comparative measurement and benchmarking may be used in these efforts. Many transportation agencies are integrating various performance measurement systems. The Florida Depart- ment of Transportation integrates the Florida long- range transportation plan, the short-range component, the annual strategic objectives, and the executive board initiatives through performance measures. The Min- nesota Department of Transportation’s measurement pyramid helps link the use of performance measures in the different plans. In conclusion, a number of common themes have emerged since the first national conference on this topic 8 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS 99395mvp9_38 12/13/05 12:24 PM Page 8

in 2000. First, the state of the practice continues to advance. Second, there continues to be wide variation in programs among state departments of transportation, with practices evolving within individual agencies. Third, some agencies have mature measurement systems. These include a range of interrelated measurement programs, alignment of measures with goals and objectives, perfor- mance reporting tailored to audiences, systematic proce- dures for reviewing performance data, and data used to strengthen planning and decision making and communi- cation with external stakeholders. Finally, agencies are learning from each other and sharing experiences with different approaches. A number of recent trends can also be identified. First, measures are more strategic and outcome- and customer-oriented. Second, while there is an emphasis on program delivery, a strong focus on system perfor- mance continues. Third, we are seeing an increased use of customer satisfaction measures. Fourth, more holistic approaches are being applied in terms of coverage and integrating systems. Fifth, performance is being moni- tored at various levels with data analysis systems pro- viding roll-up and drill-down capabilities. Sixth, the use of standards and numerical targets is increasing. Other recent trends include more sophisticated soft- ware applications, system support, and data displays. While there is greater proliferation of performance mea- sures, some departments of transportation are recogniz- ing the need to focus more selectively on a few vital measures. There are also more disciplined efforts to align measures with goals and objectives and to focus on real-time, actionable measures. It is also fair to say that many departments of transportation are making more intentional use of measurement systems to sup- port other management, planning, and decision-making processes. Finally, there is increased reporting of perfor- mance data directly to the public and other stakeholders to promote transparency in government. A number of continuing challenges face transportation agencies in the application of performance measures. Some of these challenges include agreeing on common terminology for terms such as dashboards, benchmark- ing, and performance management (see the resource paper “Performance Measurement in Transportation: State of the Practice” on pages 81–98 of these proceed- ings for examples). As will be discussed in other sessions, we need improved measures in difficult-to-measure areas, such as congestion, delay, travel time, and reliability; freight transportation; environmental impacts; and safety and security. Developing measures that facilitate cross- modal comparisons with regard to service levels, quality, travel times, and costs is also important. Obtaining systematic feedback from other external stakeholders beyond motorists and the public at large, such as other user groups, local governments, legisla- tors, and the media, continues to be a challenge. Inter- preting the implications of customer feedback in rela- tion to engineering and professional planning criteria is also a challenge. Setting appropriate targets that are aggressive yet realistic continues to be a challenge. And many states need to use measures in a more disciplined way to articulate the relationship between strategic plans and transportation system plans more clearly. We also need to place greater emphasis on measuring results after projects have been completed. Implement- ing workable comparative measurement systems to sup- port benchmarking and process improvement represents another challenge for many state departments of trans- portation. Strengthening linkages between measurement systems and employee performance management processes is also being considered. Finally, institutional- izing strategic planning and performance measurement more effectively in agencies, through developing both internal and external buy-in, continues to be a challenge for many transportation agencies. References 1. Nicholson-Crotty, S., and L. J. O’Toole, Jr. Public Management and Organizational Performance: The Case of Law Enforcement Agencies. Journal of Public Administration Research and Theory, Vol. 14, No. 1, 2004, pp. 1–18. 2. Donahue, A. The Influence of Management on the Cost of Fire Protection. Journal of Policy Analysis and Management, Vol. 23, No. 1, 2004, pp. 71–92. USING PERFORMANCE MEASURES IN A POLITICAL DECISION-MAKING PROCESS Douglas MacDonald I appreciate the invitation to participate in this confer- ence. In listening to the other speakers this afternoon, I am struck by what is being accomplished by different transportation agencies throughout the country. Perfor- mance measures are being applied in a wide range of sit- uations to improve the transportation project selection process and the delivery of transportation services. I was asked to discuss performance measures and the political decision-making process. I think you can sim- plify the discussion of politics and performance mea- sures by focusing on a single mission. That mission is to increase the investment in transportation assets and ser- vices. In addressing this mission we first have to con- vince the public of the value received for tax dollars. We must also convince the public that what we are going to do with the next tax dollar makes sense. 9OPENING SESSION 99395mvp9_38 12/13/05 12:24 PM Page 9

It is important to start with the fundamental issue that we are in a crisis situation in transportation. More fund- ing is needed for transportation, and it is critical that we address transportation investment needs. The reasons for the current crisis are well known. First, our trans- portation infrastructure is aging. Second, our current system is not keeping up with the demands of a growing population. Third, funding for transportation has been diminishing. Finally, transportation is a critical compo- nent of our economic and social well-being. I think most people would agree that these four conditions define a crisis situation for our transportation system. The solution to this crisis is more funding for trans- portation. To obtain additional funding we need to focus on two key elements. First, we must convince taxpayers that they get a dollar’s worth of value for a dollar’s worth of tax. Transportation agencies must be accountable to the public. Performance-based management can help establish and maintain accountability. Performance measures provide internal guidance to ensure that agencies are in fact providing a dollar’s worth of service for a dollar’s worth of tax. Performance measures help identify performance weaknesses as well as performance strengths. Agencies must address their weaknesses. One of the criteria an agency can use to test the ability of its performance measures system is what has been learned about the agency’s strengths and weak- nesses. The integrity of a performance measures system is also critical. The integrity of performance measures is linked to an agency’s credibility. In the political process, credibility is critical. It takes a long time and a lot of effort to estab- lish credibility with policy makers. Credibility can be lost quickly if inaccurate information is presented or if attempts are made to cover up errors or problems. All of the performance measures described in the pre- vious presentations are relevant to accountability and reporting on values. The measures addressing project and service delivery are especially important. I think policy makers and the public in many states are inter- ested in project delivery performance measures. Deliver- ing projects on time and on budget has become the mantra at transportation agencies. Delivering projects on time and on budget is not easy, as we all know. I was fortunate to have a lot of experience in project delivery before coming to the Washington State Depart- ment of Transportation. I spent 9 years as head of the agency responsible for cleaning up the Boston Harbor. At any given time, we had 25 to 30 prime contractors working to modernize the wastewater treatment system and plant. The benchmarks for compliance on the cleanup were set by a federal district court judge, who entered the schedule for improvements as an order of the court. We had a large and complicated program to address the requirements. We were also fortunate, how- ever, to have an excellent federal district court judge. He brought a lot of proactive judgment to the process. By meeting schedules and being accountable, we were able to build strong working relationships with the various groups involved in the cleanup process. Some state transportation agencies are experiencing an infusion of new funding. These funds are frequently 1 0 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS Puget Sound Spokane Puget Sound Tri -Cities Current Daily Vehicle Hours of Delay per Lane -Mile Measuring Congestion 24 hour vehicle delay, in WSDOT ’s view, is the most basic and accessible measure for describing congestion. It indicates which roadways are congested, and gives an indication of the severity of congestion and how long it lasts. Congestion is primarily concentrated in the urban areas, especially Puget Sound, Vancouver and Spokane. The highest spike depicted on the map is located at the interchange for I-5 and I -90 in Seattle, where the average tally is about 825 vehicle hours of delay per lane mile per day. FIGURE 2 Current daily vehicle hours of delay in Puget Sound region. With demand growing and supply stagnant, congestion as measured by traveler delay has increased. WSDOT = Washington State Department of Transportation. 99395mvp9_38 12/13/05 12:24 PM Page 10

linked to specific projects or programs, however. In Washington State, the recently passed Nickel Tax pro- gram is dedicated to some 150 to 160 projects over the next 10 years. We are committed to delivering the proj- ects on time and on budget, but we all know the diffi- culties that can arise in constructing and reconstructing transportation facilities. It is important to build credibility in meeting on-time and on-budget expectations. One way to build credibil- ity is to use narrative reporting in addition to charts and other graphics. Do not underestimate the effectiveness of telling your story, including possible problems, in narrative reporting. The second part of accomplishing our mission is to convince taxpayers that what we are going to do with the next tax dollar makes sense. I think one of the traps transportation professionals sometimes fall into is to talk about strategies. I suggest that we do not invest in strategies. We invest in opportunities. We have opportu- nities to do various things with available funding. The challenge is to use performance measures to help define and select among the opportunities generated for addi- tional investments. As the Monday evening session on the international scan will point out, performance mea- sures have been used in Canada, Australia, New Zealand, and Japan to target, deliver, and measure safety projects. As you all know, there are many ways to describe congestion. Among them are trip times, travel speeds, trip-time reliability, delay times, and the buffer index. One of the techniques we saw used in Japan on the scan tour illustrates traveler delay. We believed this was a powerful way to present infor- mation on congestion to the policy makers and the pub- lic graphically. Figure 2 is one example showing current daily vehicle hours of delay per lane mile in the Puget Sound region. We also use Figure 3 to illustrate the rela- tionship between delay and efficiency. Figure 4 shows the percentage of productivity lost because of delay on freeways in the Puget Sound region. There are two ways of dealing with the congestion problem. We can increase capacity and we can increase efficiency. We know that incidents and accidents degrade the efficiency of our freeways and that one way 1 1OPENING SESSION I-405 NB @ 24th NE, Weekdays in May, 2001 0 10 20 30 40 50 60 70 80 0 20 0 40 0 60 0 80 0 10 00 12 00 14 00 16 00 18 00 20 00 22 00 24 00 Hourly Volume/Lane Sp ee d Slightly lower speed, higher throughput Much lower speed, lower throughput Max throughput is reached at roughly 50 mph. Volume and Speed Relationship Northbound I-405 at NE 24th Street 0% 20% 40% 60% 80% 100% 12 A M 3 AM 6 AM 9 AM N oo n 3 PM 6 PM 9 PM Lost Capacity Percent of Productivity Lost Due to Delay (b)(a) FIGURE 3 Relationship between delay and efficiency. (a) Maximum freeway throughput is typically at speeds of 45 to 50 mph. This accommodates about 2,000 vehicles per hour per lane. System throughput drops dramatically when traffic volume forces speeds to drop below 50 mph. (b) During the peak period on I-405, congestion reduces the throughput of the two general-purpose lanes in Renton to the capacity of one free-flowing lane. Percent of Productivity Lost Due to Delay 50% 100% FIGURE 4 Percentage of productivity lost to delays in the Puget Sound region. Lost productivity on Puget Sound freeways is staggering. In the peak travel period on an average weekday, delay causes signifi- cant loss in productivity—as much as 60 percent. 99395mvp9_38 12/13/05 12:24 PM Page 11

to increase efficiency is through the use of incident response teams or highway helper programs. We have documented the average delay savings with incident response teams through the use of performance mea- sures. Presenting this type of information to policy mak- ers and the public is critical to build support for these types of programs. We have also been able to use performance measures and monitoring to show the benefits of restriping a seg- ment of SR-167 from two lanes to three lanes, which was suggested by a commuter, and the opening of a high-occupancy vehicle lane extension on I-5 South. Both of these situations provide powerful information to present to policy makers and the public concerning the benefits of transportation improvements and what the department can do with additional funds. The key is to use this type of information to build a balanced program that includes both operational and capital improvements, along with enhancement proj- ects, that will provide the greatest return on invest- ments. The ability to tell the story and to communicate with policy makers is a key part of the process. We must clearly articulate where transportation funds come from and how they are allocated. 1 2 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS 99395mvp9_38 12/13/05 12:24 PM Page 12

1 3 Impact of Performance Measures on Internal and External Relationships Robert Johns, Moderator, University of Minnesota Mark C. Larson, Minnesota Department of Transportation Lisa Klein, Metropolitan Transportation Commission Sarath Joshua, Maricopa County Jeffrey Lindley, Federal Highway Administration David Ekern, Idaho Department of Transportation ORGANIZING FOR PERFORMANCE-BASED MANAGEMENT Mark C. Larson I appreciate the opportunity to summarize the confer- ence resource paper that addresses organizing for per- formance-based management. Performance measures have been used at many transportation agencies for the past 15 to 20 years. The use of performance measures has been driven by a variety of external and internal fac- tors. In talks with representatives from a number of state departments of transportation and metropolitan planning organizations (MPOs) in preparing the resource paper, some common elements emerged that appear to influence successful programs. External groups, including state legislatures, com- missions, and governors, helped promote the movement toward performance-based management. Internal fac- tors include quality management programs, transporta- tion planning requirements, and changes in leadership at many agencies. Performance-based management was well established in the private sector before it was intro- duced into public agencies. Freight companies, rail- roads, airlines, and public transit agencies have been using performance measures for many years. External groups, especially state legislatures and gov- ernors, have become involved when there is a perceived lack of accountability at state transportation agencies. Our challenge is to move forward and set our own agen- das. If we do not, the likelihood of outside forces doing it increases greatly. As other speakers have noted, a number of states have tied increases in transportation funding to specific accountability measures. Examples include the 1980s gasoline tax increase in Florida, Vision 21 in Mississippi, the Nickel Tax in Washington, and the Cooper River Bridge project in South Carolina. Other examples are the 5-cent sales tax increase in Arizona, the dedication of increased bridge tolls for transit in the San Francisco Bay Area, and the Jobs and Progress Plan in Ohio. The Jobs and Progress Plan in Ohio included a 6-cent increase in the state gasoline tax. The increase will pro- vide an additional $5 billion over the next 10 years for transportation projects in the state. The challenge to the Ohio Department of Transportation is to deliver the promised projects on time and on budget. The resource paper outlines three stages of develop- ment in the application of performance measures at trans- portation agencies. The first stage focuses on establishing performance measures and monitoring progress. An annual report to the legislature represents a common reporting method during this stage. There may be system or process measures in this stage, but they are not usually connected. Agencies may also focus too much on trying to develop perfect performance measures during this initial stage. The second stage focuses on a more real-time or future orientation. Performance measures in this stage are used in the planning process and in managing project delivery. This stage also includes setting targets, which can bring focus to a department and promote change. More analy- sis is conducted by using the performance measures. The third stage has a strong future orientation. Rather than 99395mvp9_38 12/13/05 12:24 PM Page 13

single measures, this stage uses a package of measures to optimize benefits. The resource paper examines the current agendas of many transportation agencies. At least five common items seem to be prevalent at many transportation agen- cies. First, most states have project delivery reporting systems. Second, many states are reducing the number of performance measures in use and aligning key mea- sures with the priorities of the state plan and the state administration. Third, states are improving decision support tools and models. Fourth, transportation agen- cies are focusing on more complex performance mea- sures addressing quality of life, economic development, mobility, and safety. Finally, some states are expanding their measurement framework to encompass modal and intermodal measures. The resource paper highlights examples of elements of successful practices. First, leadership and the attitudes of top management are keys to the successful use of perfor- mance measures. An ongoing commitment to improving the application of performance measures is part of this leadership. Second, regular monitoring and reporting are critical. Successful programs are based on a culture of accountability and regular monitoring and reporting. Third, policy-driven performance measures appear to be important factors at many agencies. The links between policy, programming, and monitoring establish the basis for performance measures in many states. Figure 1 illus- trates the continuous cycle linking these elements at the Florida Department of Transportation. A fourth element of successful programs is building ownership among staff and programs within the depart- ment. Fifth, the use of practical measures tied to com- pelling priorities can help build support. Sixth, ensuring that there is support from the legislature and other pol- icy makers is important. Seventh, providing information to customers on the benefits of performance measures is important. Finally, setting targets is critical. If we do not set our own targets, we risk other groups setting them for us. Successful practice requires integration and institu- tionalization within transportation agencies. This inte- gration encompasses legislative governance, executive initiatives, the budget process, and plans and programs. Incorporation of strategic plans, transportation plans, capital programs, and project selection and design into the process is important. Montana’s performance programming process pro- vides one example of an integrated approach. It pro- vides a method to develop an optimal funding allocation and investment plan based on strategic highway system performance goals and the continual measurement of progress toward meeting these goals. The Minnesota Department of Transportation has established 10 policies in the statewide plan. Each pol- icy has related performance measures and targets. The department is now working on establishing priorities among these policies. Twenty-year targets have been set for the various performance measures. The targets have been established on the basis of customer expectations, engineering, and other factors. The Minnesota Department of Transportation’s dis- trict planning process focuses on a 2008 to 2030 hori- zon. Two scenarios are included. The first is the performance-based plan, which includes the invest- ments needed to meet targets by 2023. The second is the fiscally constrained plan, which includes priorities based on forecast revenues. System preservation is a top prior- ity at the district level. Another priority is to allocate resources in constrained plans to meet pavement targets by 2014 and to make progress toward bridge targets by 2023. The integration of performance measures into proj- ect delivery, maintenance, operations, and information technology projects and administrative support is also occurring at many transportation agencies. Examples of these approaches include Wisconsin’s maintenance accountability program and Minnesota’s operations and maintenance performance snapshot. The resource paper identifies a number of emerging activities under way at transportation agencies. These activities focus on more proactive applications of per- formance measures. Examples of these approaches include forecasting, modeling, scenario planning, life- cycle costing and optimization targets, and trade-off analysis. The future toolbox for performance measures will include a number of elements. First, we will see more use of geographic information systems to map performance gaps and other information. Second, dashboards and other measurement software will be enhanced. Third, auditing performance data will become more common- place. Measuring cost and competitiveness will be a pri- ority. Measures for performance-based contracting will also be developed, along with benchmarking. 1 4 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS Programming and Project Delivery Develop Financial Policies Monitor Performance Establish Policy and Plans FIGURE 1 Performance measurement cycle of the Florida Department of Transportation. 99395mvp9_38 12/13/05 12:24 PM Page 14

A number of lessons can be identified from the expe- rience to date with the use of performance measures at transportation agencies. First, it is important to keep per- formance measures simple and understandable. Second, take small steps, establish your system, learn as you go, and keep improving. Do not wait for perfect measures and data. Finally, policies must drive measures. In closing, I would like to highlight a few of the chal- lenging issues facing transportation agencies. Working to focus on a few critical strategic measures rather than accountability for everything we do continues to be a challenge. Linking agency performance measures to individual performance accountability is being explored at some agencies. Many agencies continue to struggle with vision and innovation versus day-to-day manage- ment. Finally, cooperation among state, regional, and local partners continues to be a challenge in some areas. Thank you. INTEGRATING PERFORMANCE MEASURES ACROSS MULTIPLE JURISDICTIONS Lisa Klein I appreciate the opportunity to participate in this con- ference. I have been asked to talk about integrating per- formance measures across multiple jurisdictions. I will use two examples to highlight some of the experiences we have had at the Metropolitan Transportation Com- mission (MTC) in the San Francisco Bay Area with the use of performance measures involving multiple juris- dictions: a local pavement management system and a ferry boat fare box recovery measure. The pavement management system in the San Fran- cisco Bay Area is a successful example of integrating performance measures across multiple jurisdictions. MTC developed a pavement management software sys- tem in the 1980s to document aggregate funding needs for local street repair in the area’s long-range planning process. The pavement management system provides an average score for pavement condition in each jurisdic- tion. It provides a numerical value from 0 to 100, with 100 equating to brand new pavement. There are 101 cities in the nine-county Bay Area. Most cities use MTC’s pavement management system software. Starting in 2001, the average condition score was pub- lished in the Bay Area State of the Transportation System report. The scores are reported by jurisdiction and ranked in order from highest to lowest. The local media report the results of the annual pavement management system rank- ings, especially the communities with the best and the worst average pavement condition scores. We have found that the annual results are discussed at the community level. The city of Petaluma, which has had the lowest score for the past 2 years, provides one example of how the pavement condition index measure contributes to local discussions and actions. The city’s low rating has been a topic of discussion among the city council, local interest groups, and other organizations. A year ago, the city council placed a measure on the ballot to increase the city utility fee to help fund pavement improvements. While the measure was not approved by voters, the city council has allocated additional funding for pavement improvements. City officials have been quoted in the local paper citing their new, improved pavement condi- tion score. The city of Santa Clara is at the other end of the rating scale, ranking first in 2002. The Santa Clara city council formally recognized the public works department for achieving the top pavement rating. A number of elements appear to contribute to the successful use of the pavement management system. First, the ability to contribute to the discussion of pave- ment needs is based on a substantial investment by MTC. A significant cost was involved in the initial development of the software. There is also a significant cost associated with its ongoing use and with providing technical assistance to communities. This support has established confidence among MTC, the MPO, and the local jurisdictions in the quality of the data used in the system. Second, a state law aids in the use of the pavement management system. California law requires that MPOs certify that local jurisdictions have a pavement manage- ment system in order to receive federal and state fund- ing. This ensures that we have access to the data on a regular basis. Third, the pavement management system measures a concept the customer understands. Although the pavement condition index is abstract, it is under- stood by community staff and policy makers, interest groups, and the public. There is some concern, however, that the focus on individual rankings may detract from the overall need to invest in pavements in the region. Fourth, the pavement management software and process provide a consistent measure that is accepted by technical staff. Timeliness is a problem, however, because there is a lag time between the reporting of data by communities, the time the data are available to MTC, and completion of the annual updates. The second example of integrating performance mea- sures across jurisdictions is the ferry boat fare box recovery measure. Funding for ferry capital and operat- ing costs has traditionally come from bridge toll rev- enues, which are dedicated to transit. There has been concern recently about the cost-effectiveness of some of the ferry routes. In 2002 the commission established a 40 percent fare box recovery standard for ferry opera- tors. A 3-year ramp-up period was provided. This fiscal year will be the third year of the ramp-up period, and 1 5IMPACT ON INTERNAL AND EXTERNAL RELATIONSHIPS 99395mvp9_38 12/13/05 12:24 PM Page 15

the next fiscal year will be the first when the fare box recovery ratio will be considered in allocating funding. In 2000 the fare box recovery ratio of the three ferry operators ranged from 40.5 to 73.2 percent, and the 3- year average from 1998 to 2000 ranged from 34.5 to 70.9 percent. There is also discussion of establishing a fare box recovery ratio and other operational perfor- mance measures for new transit projects funded through a recent increase in tolls. It is too soon to tell for certain how the ferry boat fare box recovery measure will work out. It will proba- bly not be as successful as the pavement management score in terms of integration across multiple jurisdic- tions. It appears that no service changes have been made in response to the measure. A number of factors may contribute to some of the push back from the three oper- ators. First, there are unique elements or special circum- stances associated with the different ferry routes. Second, compared with the pavement management example, the ferry box recovery measure lacks an estab- lished foundation. The operators would say that MTC is on dangerous ground when it comes to measuring transit operations. The stakes are also higher in terms of influence on future funding levels. Third, the measure may not be aligned with constituents’ interests. Reduc- ing ferry service that does not meet the fare box recov- ery ratio is certainly not in the best interests of residents who use these routes. Finally, with only three operators, compared with the 101 cities with pavement manage- ment systems, it is easy to make the case that each has somewhat special circumstances to be considered. INVOLVEMENT OF CUSTOMERS IN PERFORMANCE-BASED MANAGEMENT Sarath Joshua I appreciate the opportunity to talk about the experi- ence with involving customers in performance-based management at the Maricopa Association of Govern- ments (MAG). The Phoenix area is experiencing rapid population growth and development. While improve- ments are being made to the freeway system, they are not keeping up with development. I will first describe the customers of an MPO. I will highlight what MPOs manage and discuss ways MPOs can measure perfor- mance. Finally, I will provide some examples of how MPOs can involve customers in developing and using performance measures. It is first important to define the customers of an MPO. An MPO is primarily a regional collaboration of local governments. As a result, an MPO’s primary customers are local jurisdictions and agencies. MPOs are in turn customers of the state and federal governments. MPOs work with customers to develop regional solutions to crit- ical problems. Examples of MAG plans addressing important issues include the regional long-range trans- portation plan, the 911 system plan, the air quality plan, the intelligent transportation system strategic plan, and the concept for a transportation operations plan. While MPOs are typically not operating agencies, they are responsible for transportation planning and programming activities. MPOs are responsible for the development of a region’s transportation plan identify- ing how state and federal transportation funds will be spent. MPOs are responsible for developing plans to meet air quality conformity requirements. MPOs also develop future growth scenarios for the region through extensive modeling to assist in decision making. MPOs can use performance measures both internally and externally. Examples of internal performance mea- sures include regional funds obligated on time and proj- ects completed on time and on budget. Outcome measures can be established and program activity can be monitored. Measures from an external customer viewpoint might include maintaining pace with chang- ing conditions, maintaining or improving the quality of life, improving safety, reducing traffic congestion, and maintaining or reducing travel times. MPOs can involve their customers in developing and monitoring performance measures in a number of ways. First, decisions at MPOs are made in an open environ- ment with direct involvement of customers. MPOs have formal committee structures involving all stakeholder groups. All MPO plans and decisions are subject to extensive public review. The development of the regional concept plan for trans- portation operations at MAG provides a recent example of involving customers in developing performance mea- sures. The plan focuses on improving transportation oper- ations in the region. We asked representatives from the cities, counties, transit agencies, and other organizations to identify the current operation of the regional, local/regional, and local transportation systems. Partici- pants were then asked their expectations of operations in 3 years and in 5 years if different improvements were made. The results of this assessment were used to help identify the goals of the operations plan. The regional transportation plan includes perfor- mance measures for a number of areas. Maintenance and safety performance measures address cost and the number of crashes. Access and mobility performance measures focus on the level of service and delay. There are also performance measures related to sustaining the environment, accountability, and planning. The regional concept plan for transportation operations includes per- formance measures of freeway mobility, arterial mobil- ity, incident duration, and integrated operations. For example, the 3-year goal is to reduce the duration of 1 6 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS 99395mvp9_38 12/13/05 12:24 PM Page 16

incidents by 10 percent. The 5-year goal is to reduce the duration of incidents by 70 percent. LINKING NATIONAL PERFORMANCE MEASURES TO EXTERNAL CUSTOMERS Jeffrey Lindley It is a pleasure to have the opportunity to share with you the Federal Highway Administration’s (FHWA’s) 3 years of experience in developing and tracking conges- tion performance measures at the national level. FHWA started an effort in late 2000 to develop metrics for its internal planning process. We were also interested in better understanding the causes of traffic congestion, being able to track congestion trends, and presenting this information to external audiences. We now have a fairly extensive ongoing program of congestion monitoring, management, and research. The FHWA Office of Operations provides consider- able information on congestion monitoring on its webpage, ops.fhwa.dot.gov. Congestion and reliabil- ity performance measures will be discussed in more detail at the Tuesday morning session. There is also a breakout session on congestion and reliability. My comments focus on four general topics. First, I will discuss the number and the type of measures that were selected. Second, I will describe how the best mea- sures were identified. Third, I will address the process of setting targets for the selection of performance mea- sures. Finally, I will highlight the importance of com- municating the results of performance monitoring to key external audiences, especially the public, the media, and policy makers. FHWA uses congestion measures focusing on the aver- age duration of congested travel, the travel time index, and the buffer index. There are two measures addressing the average duration of congested travel. The first is that for any 5-minute interval a trip is congested if its duration exceeds 130 percent of free-flow or uncongested duration. The second measure is that if more than 20 percent of all trips in the network are congested in any 5-minute inter- val, the entire network is congested for that time interval. The travel time index is defined as the ratio of congested and uncongested travel times averaged over all congested trips. The buffer index is defined as the ratio of total travel budget required for 95 percent on-time reliability over the uncongested travel time averaged over all congested trips. The buffer index provides a measure of not only how con- gested the system is but also how reliable the system is. It provides a performance measure from the customer’s per- spective. The buffer index represents the amount of time commuters need to build into their trip to arrive at their destination on time 95 percent of the time. Figure 2 illustrates the concept of the buffer index. The dashed line is the travel time index. It reflects the typical morning and afternoon peak periods, when con- gestion levels are highest. The buffer index is the solid line. While it tracks the travel time index, the buffer index line is higher, reflecting the extra time that must be built into a trip to arrive on time. As you can see, the buffer index for the afternoon peak period is about 100 percent higher than the travel time index. Survey research indicates that travelers value reliabil- ity and that system unreliability is a major concern. Travelers, especially peak-hour commuters, are willing to put up with some congestion. What they do not like is when a trip takes 20 minutes one day, 40 minutes another, and 90 minutes on a day with an accident. FHWA highlights these performance measures in a dashboard that is updated monthly. Figure 3 is an exam- ple of a page from the dashboard, which provides a summary of the current status of the three performance measures. A monthly urban congestion report is pre- pared and presented to FHWA management. This is cur- rently an internal document. I will highlight a few of the things we have learned in developing and applying national congestion perfor- mance measures. The first lesson relates to the number and the type of measures. We spent a good deal of time examining possible performance measures. We finally settled on the three I described previously. While one performance measure may not be enough for most mea- surement areas, focusing on a few key measures is important. Also, do not just select a measure because it is easy to get the needed data. It is important to identify measures that will be of use, not measures that have data available. We would not have developed the buffer index if we had focused only on easy-to-obtain data. The second lesson addressed selection of the measures that best captured the most important aspects of a prob- lem. The measures FHWA selected focus on the extent of 1 7IMPACT ON INTERNAL AND EXTERNAL RELATIONSHIPS 1.00 1.20 1.40 1.60 1.80 2.00 2.20 2.40 0:00 2:00 4:00 6:00 8:00 10:00 12:00 14:00 16:00 18:00 20:00 22:00 0:00 Time of Day (Average Weekdays Only) Index Value or Congested Travel (1.0 = 100%) Travel Time Index Buffer Index Travel Time Reliability — Reduce the Gap Unreliable travel On time 95% conditions FIGURE 2 Performance measurement from the customer’s perspective: reliability—travel time index and buffer index by time of day. 99395mvp9_38 12/13/05 12:24 PM Page 17

congestion, the duration of congestion, the intensity of con- gestion, and the reliability of the system. The third lesson focuses on selecting performance measure targets. Identify- ing appropriate targets may be more of an art than a sci- ence. One issue to consider is how aggressive the targets should be. FHWA’s current congested travel target is sim- ply to reduce the growth in congested traffic. This target has been kept simple until we have real-time traffic data. It is also important to remember that how a target is defined may influence behavior. We are currently exam- ining the targets being used in different areas for inci- dent management. Some states, such as Washington and Florida, have set a target of clearing all incidents within 90 minutes. Other areas, including Dallas, have set a target of clearing incidents in an average of 15 minutes. While both targets use clearance times, the targets may lead to different behavior. Areas with a 90-minute target may focus more on clearing major incidents. Areas using an average clearance time might focus on clearing minor incidents more quickly. The final lesson focuses on communicating with external audiences, which include policy makers, the media, and the public. Our experience indicates that while these groups may not understand everything about the buffer index, they do understand reliability. Measures can be as complicated as they need to be, but how we explain what they mean needs to be straight- forward and easy to understand. It is important to pro- vide a message that resonates with the audience you are trying to reach. CEO PERSPECTIVE: WHAT DO YOU DO WITH THE MONEY David Ekern We have heard a number of common themes from speakers this afternoon related to the growing use of performance measures by state transportation agencies. First, the intellectual capacity and the cultural changes in agencies are a critical aspect of initiating performance measures. Second, performance measures need to be tai- lored to the audience. Third, performance measures are being undertaken as part of a family of initiatives. Finally, in many ways, performance measures are all about translation, sales, and speed. I have completed my first year as Director of the Idaho Department of Transportation. When I started there were a number of key elements influencing the department. First, funding focused on the remnants of a 4-cent increase in the gasoline tax that was approved in 1996 on the basis of a commitment by the depart- ment to improve the performance of the transporta- tion system. Second, there was apprehension of a change in administration and a worry about shifting focus. Third, the department had disconnected initia- tives. Finally, transportation funding in the state had been stagnant for the previous 4 years, with no new taxes. The future transportation system in Idaho is interna- tional in scope, intermodal in form, intelligent in char- acter, and inclusive in service. On the basis of recent dialogues with citizens throughout the state, I suggest that customers expect at least four basic things from the transportation system. First, customers want a multi- modal system that provides choices. Second, the trans- portation system should provide a quality of life that respects history and protects the environment. Third, transportation agencies should engage people in making decisions about the system. Finally, customers want a system that achieves goals within the bounds of reason- able funding. You will note the use of the term reason- able funding rather than adequate funding. Reasonable funding was the term we heard from our customers and policy makers. We spent a good deal of time within the department talking about the transformation to performance-based 1 8 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS Dashboard Status: Yellow NUMBER OF CITIES DOWN > 5% NO CHANGE UP > 5% DOWN > 5% NO CHANGE UP > 5% DOWN > 5% NO CHANGE UP > 5% Vs. Same Qtr 2002 3 2 5 4 3 3 5 2 3 UCR Composite NATIONAL CONGESTION INDICATORS Cong. Travel Time/Day Travel Time Index Buffer Index 6.3 Hours 3.6% 1.50 2.0% 2.04 1.0% Extent Intensity Reliability FIGURE 3 Example of FHWA dashboard congestion measures. UCR = urban congestion report. 99395mvp9_38 12/13/05 12:24 PM Page 18

management and what is expected of department employees. We also made presentations to numerous groups and talked with customers throughout the state. Figure 4 highlights some of the emerging changes in cul- ture within the department related to focusing more on performance-based management. The department’s budget and funding request is pre- sented to the legislature every year. We presented the state’s trend indicators, which include vehicle miles trav- eled, automobile registrations, driver’s licenses issued, population, and total fuel consumed. All of these indi- cators are increasing. We use these trend indicators to help show policy makers and the public that Idaho is a growth state. The state does not experience the severe congestion levels and other problems facing many parts of the country. We will become more like those areas, however, if we do not deal with these issues. The flat budget over the past 4 years has not helped address these problems. Our challenge was to present these trends and the depart- ment’s performance measures to the legislature in a meaningful way. Rather than just present the department’s budget in the normal way, we identified the six major products or services the department provides and highlighted the funding associated with each. A consistent format was used that focused on three messages. These messages were the funding amount, the output, and the context. This approach provides a consistent message for the annual appropriations process. Figure 5 provides an example of the information presented for contract con- struction services. The department’s legislative approach built on posi- tive factors. For example, we highlighted the fact that the department returns 94.5 percent of every dollar it receives in some type of product or service. It appears that this approach is working, because the legislature approved the department’s budget as presented for the first time in 22 years. 1 9IMPACT ON INTERNAL AND EXTERNAL RELATIONSHIPS 20th Century • Public works (output) • Project - focused • Our jurisdiction • 8-5 • Reactive • Business as usual • Do it our way 21st Century • Mobility (outcomes) • Customer-oriented • System -focused • 24X7 • Proactive • Performance-driven • Partnerships Contract construction 56.7% $241,599,800 15 Bridge Projects $ 24.9 Million (rehabilitation and improvement) 79 Lane Miles of Major Capacity Projects $ 81.3 Million 1,141 Lane Miles of Preservation Projects $ 57.4 Million 47 Safety and Operations Projects $ 19.5 Million Intersections, turn bays, guardrail, ports of entry, operations, bridge decks, planning and research, intelligent transportation systems, and training. 86 Design and Right-of-Way Projects $ 25.1 Million 128 Local Highway Projects $ 33.4 Million ECONOMIC INVESTMENT: $241.6 Million • Represents one year of a six - year program ($1.5 billion) • 100% of contract construction budget is paid to the private sector • 14,000 good-paying jobs sustained Factoids: FIGURE 4 Emerging changes in cultures at the Idaho Department of Transportation. FIGURE 5 Example of Idaho Department of Transportation’s legislative message: FY 2005 projected outputs for contract construction services. 99395mvp9_38r1 12/16/05 9:56 AM Page 19

2 0 Tying Together Performance-Based Program Development and Delivery George Scheuernstuhl, Moderator, Denver Regional Council of Governments Steven M. Pickrell, Cambridge Systematics, Inc. Robert Romig, Florida Department of Transportation Jeff Price, Virginia Department of Transportation Brian Smith, California Department of Transportation Gregory Selstead, Washington State Department of Transportation PERFORMANCE CHALLENGE: LINKING PERFORMANCE-BASED PROGRAM DEVELOPMENT AND PROJECT DELIVERY Steven M. Pickrell Good morning. It is a pleasure to have the opportunity to present the resource paper on linking performance- based program development and project delivery. I acknowledge my two coauthors, Patricia Hendren and Lance Neumann, both from Cambridge Systematics, Inc. I will start by describing performance-based program management and performance-based program develop- ment. I will discuss project delivery and performance measures and provide examples from state departments of transportation and other agencies. I will highlight the importance of linking performance-based program devel- opment and project delivery and some of the challenges that may arise in making this connection. Performance-based program management is made up of three components: performance-based program development, performance-based project delivery, and system monitoring and reporting. The focus of our resource paper and my presentation today is on linking program development and delivery. Application of performance-based program develop- ment varies across agencies. Examples include the Pennsyl- vania Department of Transportation’s performance-based long-range transportation plan, which links objectives and actions to broad statewide goals; the performance-driven project prioritization approach to long-range planning adopted by the Arizona Department of Transportation; and performance-based programming at the Montana Department of Transportation. Possible benefits of performance measures in pro- gram development include linking statewide goals and projects and prioritization of the most effective pro- grams and projects. Performance measures also enhance accountability and assist in allocating funds. Perfor- mance measures can be used in trade-off analysis and in benchmarking. The results of these efforts can be com- municated to diverse audiences inside and outside an agency. Performance-based program delivery applications also vary across agencies. Examples include a project delivery management group at the Arizona Department of Transportation and the use of performance-based interactive web-based tools at the Virginia Department of Transportation. The California Department of Trans- portation (Caltrans) uses a performance-based program delivery process in response to legislative mandates, and the New Jersey Department of Transportation uses a performance-based accounting system. The real question addressed in performance-based pro- gram delivery is whether the expectations to deliver proj- ects on time, within scope, and within budget are being met. There will invariably be changes in a state depart- ment of transportation’s program as individual projects are delivered; what is needed is a better way to track and deal with changes in project scope and schedule. There is a need to understand, manage, and explain changes that occur to projects between the time the program is devel- oped and the time projects are delivered. Many times there is not an explicit feedback loop in the process, so that the 99395mvp9_38 12/13/05 12:24 PM Page 20

benefits of the program, as actually delivered, are never compared with the expected or “promised” benefits that were used to sell the initial program. Benefits of performance measures in program delivery include promoting efficient program management and program delivery. They can help minimize unnecessary or avoidable changes in scope, cost, and schedule and main- tain the integrity and intended impact of the approved program. Performance measures can be used to track the number, extent, and cause of scope, budget, and schedule changes. This information can be used to identify process improvements. The results of these efforts can be com- municated both internally and externally. They also demonstrate accountability to policy makers. There are many reasons to consider linking perfor- mance measures for project programming and project delivery. With performance-based program develop- ment only, there is a chance that the most effective pro- gram of projects will be identified but then delivered inefficiently. With performance-based project delivery only, there is a chance that project delivery will be effi- cient but that the overall program will include marginal or suboptimal projects. When project programming and project delivery are linked, the most effective projects are selected and are delivered efficiently. It is important to understand key objectives for using performance measures with planning and program devel- opment and with program delivery. A key objective in applying performance measures to planning and program development is allocating resources to programs and proj- ects to achieve system performance goals. A key objective with program delivery is delivery of selected programs and projects as efficiently as possible with minimal impact on cost, scope, and schedule. The types of performance measures used with project programming and project delivery are different. Perfor- mance measures with planning and program develop- ment address system condition and performance, such as pavement condition, congestion, and safety. Performance measures with program development typically address cost, scope, schedule, and work safety and quality. Data are collected and reported over longer periods of time with planning and development performance measures, since the influence of selected programs and projects typically is not known for several years. Project and program delivery information is usually tracked on a regular basis, such as annually, quarterly, or monthly. External factors that may influence planning and pro- gram development performance measures include driver behavior, demographic changes, and related factors. Unexpected changes in external factors may influence pro- gram delivery performance measures. Challenges related to planning and program development performance mea- sures include selecting measures, data availability, analytic tools to predict performance, external factors, defining expenditure impacts on system performance, and moni- toring over time. Challenges related to program delivery performance measures include selecting appropriate mea- sures, data availability, tracking project changes, external factors, and assessing the impact of program and project changes on system performance. Figure 1 illustrates the performance-based manage- ment structure. The three major components include program development, project delivery, and monitoring and reporting. The feedback loops are important to determine whether the set of projects ultimately deliv- ered had the desired impact on system performance and broad goals and objectives. Greater integration of the program development and delivery functions will help to improve outcomes, that is, efficient delivery of an effective program of projects. Bringing program developers and project managers together early in the process can help establish ongoing coordination and operation. It is important to emphasize the value and importance of both processes, so that inter- nal teams are equally committed to one another’s goals and to the ultimate objective, system performance. Applying the same criteria used in the original project selection to evaluate the impact of proposed scope changes at the proj- ect delivery stage will help determine whether the changes should be accepted and what the impact of the changes on overall program benefits will be. Posting construction man- agement results internally and externally is also beneficial in keeping agency staff and stakeholders involved in the outcome. Sharing knowledge among all groups is impor- tant. The Washington State Department of Transporta- tion’s Gray Notebook provides one example of sharing 2 1PERFORMANCE-BASED PROGRAM DEVELOPMENT AND DELIVERY Co m po ne nt 1 Pr og ra m D ev el op m en t Co m po ne nt 2 Pr oje ct D el iv er y Co m po ne nt 3 Sy st em M on ito rin g a n d Re po rti ng System Performance Measures to Monitor Progress on Meeting Goals/Objectives Performance Targets for Planning/ Programming Given Resource Availability Specific Set of Programs and Projects with a Defined Budget, Schedule, Scope Performance Measures to Track Delivery Result = System Performance Program Development and Project Selection Program/Project Implementation Goals/Objectives FIGURE 1 Performance-based management structure. 99395mvp9_38 12/13/05 12:24 PM Page 21

information among numerous groups. Project delivery becomes an agency goal, not just a department goal, with performance-based project development and delivery. Agencies may face numerous challenges in the imple- mentation and application of performance-based proj- ect development and delivery. Examples of possible challenges include time and resource constraints, inter- nal organization restrictions, internal and external fac- tors, and communication. The resource paper includes additional suggestions for improving the link between performance-based program development and delivery. SETTING TARGETS AND MAKING TRADE-OFFS: PERFORMANCE-BASED RESOURCE ALLOCATION Robert Romig It is a pleasure to have the opportunity to talk about performance-based resource allocation at the Florida Department of Transportation. I will start by providing an overview of the department and then talk about Florida’s asset management process and performance- based resource allocation. The Florida Secretary of Transportation is the chief administrative officer of the department. The Florida Transportation Commission is the advisory board to the department. The Florida Department of Transportation is a decentralized agency, with a $6 billion annual budget. The agency is trust funded. It currently has about 8,000 employees, which is down from a high of 10,300 employ- ees in 1999. The department has moved toward more extensive use of the private sector over the past 5 years. Consultants are used for some 75 percent of planning activ- ities, 82 percent of design, 100 percent of construction, and 80 percent of maintenance. The department is responsible for major elements of the transportation system in the state. Approximately 12,000 of the 114,500 centerline miles of public roads are owned by the department. These state roads carry about two-thirds of all public road traffic. The depart- ment maintains 6,200 of 11,000 bridges in state. It also provides funding and technical support to 14 seaports, 22 commercial airports, 3,000 miles of rail, and 18 transit systems. Florida’s asset management process is policy driven. It is based on a strong statutory policy framework. Trade- offs between preservation and capacity programs are made at the policy level. There are both management systems and performance-based programming and bud- geting systems. The asset management process provides a systematic approach to decision making. It represents a continuous cycle that includes evaluation and feedback. Policy guidance for investment decisions comes from a number of sources at the federal, state, regional, and local levels. Federal and state laws and regulations provide gen- eral guidance. The Florida Transportation Plan and the Strategic Intermodal System (SIS) provide more specific direction for investments. Regional policy plans, local com- prehensive plans, and metropolitan planning organization (MPO) plans also provide detailed guidance. The legislative direction provides the statutory policy framework for the process. Principles guiding invest- ments include the preservation and maintenance objec- tives and the capacity objectives. Among the capacity objectives are that at least 50 percent of new discre- tionary funds will be allocated to the Florida Intrastate Highway System (FIHS) and that a minimum of 15 per- cent of state funds will be dedicated to public trans- portation. The planning and programming requirements provide program stability and help link statewide and local priorities. Figure 2 illustrates the linkage between goals and program funding. It shows the links among the long- range transportation plan, the short-range component, and the short-range work program and list of projects. Figure 3 illustrates the elements under the general umbrella of asset management. Data collection, perfor- mance measures, management systems, and decision support tools are all part of the process. The pavement management system is a major com- ponent of the preservation portion of the program. The department conducts an annual pavement condition survey that examines ride quality, crack severity, and rutting. The pavement management system objective is to have at least 80 percent of pavement on the state highway system meet department standards. The bridge management system is another compo- nent of the preservation program. Bridges in Florida are inspected every 2 years. These inspections help deter- mine the need for preventive maintenance, major or minor repair work, and replacement. The objectives are 2 2 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS LONG RANGE Florida Transportation Plan Goals and Objectives INTERMEDIATE RANGE Short-Range Component of the FTP Strategic Goals Short-Range Objectives Program Resource Plan Operating Policies SHORT RANGE Work Program List of Projects and Activities FIGURE 2 Funding directed by policy and program objectives identified in legislation and Florida Department of Transportation plans. 99395mvp9_38 12/13/05 12:24 PM Page 22

that at least 90 percent of department-maintained bridges meet department standards and that all bridges open to the public be safe. The maintenance program is the third element of the preservation and maintenance program. A maintenance rating program is used to evaluate field conditions. The following are elements of this program: • Roadway—potholes and pavement joints; • Traffic services—signs, lighting, and striping; • Roadside—unpaved shoulders, fences, and sidewalks; • Drainage—storm drains and ditches; and • Vegetation/aesthetics—mowing, litter, and trimming. The objective is to have 100 percent of roads on the state highway system achieve the maintenance standard. The highway capacity program focuses on adding new facilities and expanding existing roadways as iden- tified in the various plans. Improvements to the FIHS are needs driven on the basis of the 10- and 20-year plans and the 20-year cost-feasible plan. The decision support system includes safety, conges- tion, intermodal connections, economic development, and pavement condition. Capacity projects on other arterials fund priorities of MPOs and counties. There are capacity program performance measures for highways, public transportation, and the SIS. The high- way performance measures include maintaining the rate of change in person-hours of delay on the FIHS in urban- ized areas through 2007. The nonurbanized measure is presently under development. The transit performance measure is to increase transit ridership at twice the average rate of population growth through 2011. The SIS mea- sures are being developed but will have a system focus and will consider efficiency, reliability, and economic impacts. One issue of concern to all transportation agencies is monitoring project implementation. In Florida, the strong policy framework and program stability help ensure that plans are implemented. Also, financial plan- ning helps support program objectives. The department uses monthly production management meetings, annual agency performance reports, and annual evaluations of work programs by the commission as part of an ongoing performance production-monitoring program. A number of approaches are used to monitor and to report on progress. These approaches include the ongo- ing Transportation Commission productivity and perfor- mance review, the Florida Department of Transportation Executive Board annual program planning review process, the annual report on legislative-directed mea- sures, and monthly reviews and monitoring by the Executive Board of dashboard measures and other infor- mation. As do those of other states, the department faces a number of challenges in using and reporting on perfor- mance-based resource allocation techniques. The first challenge relates to communication and how the perfor- mance reports should be adapted to various users. A sec- ond is integrating large databases and management systems. A third is having consistent data over time, espe- cially data from various sources. A fourth relates to developing performance measures for mobility and inter- modal systems. A final challenge relates to accountabil- ity for performance measures outside the responsibility of the department. In closing, I would like to highlight a few of the crit- ical success factors in the department’s approach. First, there is accountability due to a clear link between pol- icy, programming, and performance monitoring. Sec- ond, there is strong decision-maker support and involvement. Third, the long-term and short-term cycli- cal process supports the program focus. Fourth, the pro- gram is based on credible data from solid data collection, quality control, and analysis techniques. Fifth, these data are converted into useful information. The following websites provide additional informa- tion on the various programs and policies in Florida: • Agency overview: www.dot.state.fl.us/financial planning/AGENCY_OVERVIEW.pdf • 2020 Florida Transportation Plan: www.dot.state. fl.us/planning/ftp2020/default.htm • Short-Range Component and Annual Performance Report: www.dot.state.fl.us/planning/policy/pdfs/src.pdf • Work Program Instructions: www.dot.state.fl.us/ programdevelopmentoffice/Development/WP_instruc tions.shtm • Florida Transportation Commission Performance Measures: www.ftc.state.fl.us/performance%20and% 20production%20review%200102.pdf 2 3PERFORMANCE-BASED PROGRAM DEVELOPMENT AND DELIVERY ASSET MANAGEMENT Data Collection Performance Measures Management Systems Decision Support Tools FIGURE 3 Components of Florida’s asset man- agement program. 99395mvp9_38 12/13/05 12:24 PM Page 23

Thank you for the opportunity to describe the Florida Department of Transportation’s performance-based asset management program. I look forward to discussing your thoughts and ideas in the breakout session. PERFORMANCE-BASED PROGRAM DEVELOPMENT AND DELIVERY Jeff Price It is a pleasure to participate in this conference and to have the opportunity to share our experience in Virginia with performance-based program development and delivery. During the 1990s and early 2000s, the public perceived the Virginia Department of Transportation as a poorly managed organization that did not fulfill its promises. In 2002 Governor Warner took office and appointed a commissioner with a “no-nonsense” busi- ness reputation to address problems the department was having with program development and delivery and to improve the department’s credibility. Immediately upon his appointment, the commissioner began seeking ways to improve the situation. The commissioner found that to assess the construc- tion program, he had to meet with a room full of peo- ple. Even then he could not get a clear answer. Lines of responsibility were confused, and it was difficult to identify who was accountable. The commissioner eliminated several executive posi- tions, reassigned executive staff, and reorganized the department. He also set goals, clarified responsibilities, communicated that staff would be held accountable for fulfilling their responsibilities, and started examining ways to measure and manage performance. The commissioner made a departmentwide announce- ment that the focus going forward would be on the flaw- less delivery of the 6-year improvement program. Virginia’s General Assembly requires the Commonwealth Transportation Board to approve the program, which provides the development and funding schedules for all construction projects over a 6-year planning horizon. To gauge how well the department is achieving this goal, measures that apply to both project development and delivery were developed. We track on-time perfor- mance in preliminary engineering and construction of each project. A project is considered developed on time if it goes to advertisement by the original advertised date, which is set following the initial scoping meeting. A project is delivered on time if the completed project is inspected and accepted by the original due date. A project is delivered on budget if it is completed for 110 percent or less of the original award amount. We use 10 percent over the award amount to account for risks such as unexpected field conditions. This percentage is being scrutinized, and we may change the definition to 100 percent of award amount. We believe strict interpretation of on time, on budget sends a message that we will be accountable, we will be good stewards of the taxpayer’s money, project scope should not creep, and we will deliver what we promised when we promised it. It forces us to do a better job of project scoping, design and plan review, cost estimation, contract development, and project management. The Virginia Department of Transportation has changed in a number of ways. Performance measures were not new to the department. In 1986 we tracked 276 per- formance measures, none of which were paramount. While we still track performance in many areas, the focus under the current commissioner is on the core measures of on- time and on-budget program delivery. We have developed better tools and management information systems, includ- ing the dashboard, the watch list, and the quarterly report. In addition to improving our ability to monitor develop- ment and delivery of projects, the commissioner began con- ducting monthly meetings with all project managers to discuss performance of active projects. The dashboard is a web-based tool used by the department, contractors, and the public to monitor the development and delivery of construction projects. It provided a series of web pages highlighting information on projects and their on-time and on-budget status as they progress from development to delivery. Users can search and select projects by location, type, or date and can then drill down to obtain more specific information. Users can also communicate with the project manager through e-mail. The watch list is an internal tool used by the com- missioner and project managers to monitor the status of active construction projects. The commissioner uses the watch list during the monthly videoconference to dis- cuss the status of the program, performance in each dis- trict, and projects that are experiencing problems. The open format of the meetings provides project managers with the opportunity to speak frankly and directly to the commissioner and often generates suggestions for dealing with emerging problems. Figure 4 illustrates the dashboard watch list. The top section shows the status of construction contracts. It compares the inspector’s estimated completion date with the original completion date to determine the num- ber of days the project is behind. The total also includes contracts not started on time. The report card, illustrated in Figure 5, is published quarterly and can be downloaded from the department’s website. It is the commissioner’s report to the Common- wealth Transportation Board and the citizens of Vir- ginia. It includes his statement of the accomplishments and shortcomings of the previous quarter with graphs depicting our on-time and on-budget performance. 2 4 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS 99395mvp9_38 12/13/05 12:24 PM Page 24

Figure 6 shows the improvement in the department’s on-time performance over the past 4 years for construction and maintenance projects. Performance has improved from a low of 20 percent of construction projects and 38 percent of maintenance projects being delivered on time in fiscal year 2001 to 36 percent of construction and 51 percent of maintenance projects in fiscal year 2004. Figure 7 shows the improvement in the department’s on-budget performance over the past 4 years for construc- tion and maintenance projects. Performance has improved from a low of 51 percent of construction projects and 59 percent of maintenance projects being delivered on budget in fiscal year 2001 to 73 percent of construction and 81 percent of maintenance projects in fiscal year 2004. The bubble chart in Figure 8 illustrates both on-time and on-budget performance in fiscal year 2002. The size of the bubbles reflects the size of the project in dollars. The x-axis represents on-time performance, and the y-axis represents on-budget performance. All of the bubbles in the lower left quadrant were on time and on budget. All of those in the upper right quadrant were late and over budget. Now compare the performance in fiscal year 2002 with performance in fiscal year 2003 and fiscal year 2004, as shown in Figures 9 and 10, respectively. These figures clearly show an improvement, as more projects have been completed on time and on budget. Figure 11 illustrates overall performance for the past 3 completed fiscal years. Each bubble represents the cen- ter of gravity from all the projects shown in each of the previous three figures. Part of our work in measuring performance involves identifying and analyzing factors that influence perfor- mance. We examined hundreds of work orders, which everyone believes is the main cause of projects being late or over budget. We found that generally work orders were written for a few reasons. These reasons included contract omissions, plan or design errors, utility and local government delays, force majeure, and unexpected site conditions. However, regression analysis indicated that while these are indeed significant factors, collec- tively they explain little of the variation in project delays and budget overruns. So while the department addresses these factors, we continue to seek other factors influencing program development and delivery. It has become apparent that success depends on many factors working together. 2 5PERFORMANCE-BASED PROGRAM DEVELOPMENT AND DELIVERY FIGURE 4 Virginia Department of Transportation dashboard watch list. 99395mvp9_38 12/13/05 12:24 PM Page 25

Creating tools that, in a very public way, show how you are doing and identify who is responsible has been a great motivator. Staff are more focused on achieving milestones and improving the processes they use to reach the targets set by the agency. We are examining contracts more carefully, with a focus on how we iden- tify and allocate risk. We are taking a more systematic approach to risk assessment; we draw on data and expe- riences to formulate probabilities and contingencies for risk events. We realize that utilities and local govern- ments operate on their own schedules and that we must do a better job of coordinating our plans with theirs. Again, our observation is that adopting an environ- ment of accountability stimulates improvement. If you put the information out there and identify the people responsible, they will be more focused on achieving their targets. Tracking performance with the tools we have developed has focused people’s attention on getting the job done. My office, which conducts process improve- ment studies, has seen a dramatic increase in requests for new studies. It is not enough simply to develop a performance measure, set a target, and tell people that 2 6 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS FIGURE 5 Virginia Department of Transportation report card. 0% 10% 20% 30% 40% 50% 60% FY01 FY02 FY03 FY04 Const on Time Maint on Time FIGURE 6 Virginia Department of Transportation construc- tion and maintenance projects on time. 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% FY01 FY02 FY03 FY04 Const on Budget Maint on Budget FIGURE 7 Virginia Department of Transportation construc- tion and maintenance projects on budget. 0% 50% 100% 150% 200% 0% 100% 200% 300% Percent of Scheduled Time Used to Complete the Contract Pe rc en t o f C on tra ct A wa rd Pa id FIGURE 8 Virginia Department of Transportation contracts with final vouchers paid in fiscal year 2002. 0% 100% 200% 300% Percent of Scheduled Time Used to Complete the Contract 0% 50% 100% 150% 200% Pe rc en t o f C on tra ct A wa rd Pa id FIGURE 9 Virginia Department of Transportation contracts with final vouchers paid in fiscal year 2003. 99395mvp9_38 12/13/05 12:24 PM Page 26

by next month or next year you have to hit this target. You have to provide the means to get there. We now incorporate performance measurement in all process improvement studies. Since data from several systems are used to measure performance, there has been a renewed interest in data accuracy. People who know they are responsible for spe- cific data elements that influence what is seen on the dashboard are now much more careful about the accu- racy, the completeness, and the timeliness of the data they enter. This has also led to discovering problems with projects sooner. In addition, because getting a project to advertise- ment requires so many parallel functions and individual milestones, we are now examining the measurement of on-time performance in program development by the use of more than just the advertisement date. We are now considering measures of intermediate milestones, such as preliminary engineering approvals, environmen- tal approvals, plan and design reviews, and right-of-way acquisition. In conclusion, our experience has been that setting targets and demanding accountability have focused peo- ple’s attention on the role they play in developing and delivering projects on time and on budget. Requests for assistance to improve business processes and perfor- mance have increased considerably. We are now incor- porating performance measurement in all process improvement studies. We have found that systems used to store informa- tion are of little value if that information is never used. The dashboard is simply a reporting tool. It contains no information. It retrieves information from other sys- tems. When they were first implemented, we found that these other systems contained many errors and gaps. Staff advised us that we could not rely on these systems to provide accurate information about the status of projects. However, we found that once it was under- stood that project performance would be based on information in the underlying systems, the quality of data in those systems improved tremendously. Imple- menting the dashboard, the watch list, and other per- formance-reporting systems enables us to discover and address project problems sooner. Thank you. LINKING SYSTEM PERFORMANCE MEASUREMENT TO DECISIONS— PROGRESS SINCE THE FIRST NATIONAL CONFERENCE Brian Smith At the conference 3 years ago, Caltrans’s then Chief Deputy Tony Harris described California’s activities related to performance measures. I will not repeat his comments related to California’s use of performance measures to drive programming decisions. Instead, I will describe some of the progress the department has made since the first conference in 2001. First, to set the stage, you have to remember the size of California. The state has 36 million people and is still growing. California has the world’s fifth-largest econ- omy. The annual gross state product is $1.3 trillion. Approximately 37 percent of U.S. international trade flows through California. The 50,000 lane miles of state highway carry some 255 billion vehicle miles of travel annually out of 322 billion vehicle miles of travel statewide. There are 160,000 lane miles of local streets and roads. These figures indicate that the freeway sys- tem has to perform many different functions and serves local, regional, intrastate, interstate, and international traffic. The transportation system in California is institu- tionally complex. Many parties are individually or mutually responsible for providing transportation in the state. Hardly any one agency can unilaterally say yes to transportation investments, but many agencies can say no. Responsibility and accountability are diffused among numerous agencies. Federal and state agencies share responsibility with 19 MPOs, 26 regional transportation planning agencies, 58 counties, and 478 cities. There are 218 transit providers carrying 1.1 billion transit trips per year. The state also 2 7PERFORMANCE-BASED PROGRAM DEVELOPMENT AND DELIVERY 0% 50% 100% 150% 200% 250% 300% 350% Percent of Scheduled Time Used to Complete the Contract 0% 50% 100% 150% 200% Pe rc en t o f C on tra ct A wa rd Pa id FIGURE 10 Virginia Department of Transportation con- tracts with final vouchers paid in fiscal year 2004. 0% 50% 100% 150% 200% 0% 50% 100% 150% 200% 250% 300% 350% Percent of Time Used Pe rc en t o f A wa rd A m ou nt s FY02 FY03 FY04 Bubble size indicates sum of award amounts. FIGURE 11 Center of gravity for projects in fiscal years 2002, 2003, and 2004. 99395mvp9_38 12/13/05 12:24 PM Page 27

has major ports; commercial airports; and light rail, commuter rail, and intercity heavy rail operators. The state transportation funding and programming process has several unique characteristics. The Califor- nia Transportation Commission decides on the approved level of funding for safety, operation, and preservation projects. This amount comes off the avail- able funding for new projects. The remaining funding is divided uniquely, with 75 percent going to regions and 25 percent going to the state for interregional projects and rail system investments. By state law, all projects must be in the regional plans. I will briefly summarize the background to the current use of performance measures by Caltrans. The develop- ment of the Intermodal Surface Transportation Efficiency Act of 1991 required state plans in the mid-1990s and led to a follow-on effort to start developing performance mea- sures. The California Transportation Plan 1 Performance Measures Module developed in 1997 and 1998 included nine outcomes, indicators, and decision-making linkages. It established the need to integrate performance measures and long-range planning supporting decisions. It also noted that transportation improvement plans should demon- strate progress toward outcomes. The plan established the need for periodic monitoring and reporting. Tony Harris’s presentation at the 2001 conference focused on the development of performance measures between 1998 and 2001. An external system measures working group and policy advisory committee helped with the effort. Attempts to increase performance mea- surement resources were not successful. A system per- formance measure prototype was developed focusing on safety and security, mobility and accessibility, reliability, and environmental quality. At the same time, some MPOs in the state, primarily the Metropolitan Trans- portation Commission (MTC) in the San Francisco Bay Area and the Southern California Association of Gov- ernments (SCAG) in the Los Angeles area, began using both quantitative and qualitative performance measures independently. Since the first conference in 2001, Caltrans staff have continued the development of performance measures and monitored the national experience and research. During this time, performance measures entered the day-to-day activities in the traffic operation and the maintenance divisions. Strategic goals focused on safety, reliability, performance, flexibility, and productivity were developed to drive internal performance agreements. Some of the challenges from previous efforts included the lack of resources and the lack of management com- mitment. Linkage to real-world decisions was also unclear. In some cases, technology development and deployment issues had to be addressed. A number of factors influence the current efforts. The California Transportation Plan 2 provides the vision, goals, and policies that set the framework for perfor- mance measures. A number of external stakeholder groups were involved in the development of this plan. As you know, California went through a change in governor recently, with corresponding changes in appointed agency officials and agency directions. The new leadership has brought more of a business orienta- tion to the department. The Business, Transportation, and Housing Agency undertook a Performance Improvement Initiative. The initiative involved an expert review panel, whose recommendations included expanding relationships with external stakeholders and developing system performance measures. The primary intent of the current effort is to optimize transportation’s impact on the economy, the environ- ment, and equity by providing a framework for improved decision making by system managers and sys- tem users. There is a focus on improving collaboration and accountability and management of transportation systems and modes. Another intent is to streamline and improve business practices. A system performance measure team was appointed by the Secretary of the Business, Transportation, and Housing Agency in May 2004. The team is composed of public- and private-sector interests and state, regional, and local governments. The team focused first on out- comes and then on indicators. The team identified out- comes and indicators to pursue immediately, as well as others needing further development. The final report was delivered to the secretary on July 30, 2004. The following nine performance measures and defin- itions are included in the report: • Mobility, accessibility, and reliability: Minimize time and cost and maximize choice and dependability. Reach desired destinations within reasonable time and cost with choice, dependability, and ease. • Productivity: Maximize throughput or efficiency systemwide. • System preservation: Preserve the publicly owned transportation system at a specified state of repair or condition. • Safety: Reduce fatalities, injury, and property loss of system users and workers. • Environmental quality: Maintain and enhance the quality of the natural and human environment. • Coordinated transportation and land use: Ensure that transportation decisions promote and support job and housing proximity. • Economic development: Contribute to California’s economic growth. • Return on investment: Benefit–cost analysis or best return on investment, including life-cycling costing. • Equity: No person shall, on the grounds of race, color, or national origin, be excluded from participation 2 8 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS 99395mvp9_38 12/13/05 12:24 PM Page 28

in, be denied the benefits of, or be subjected to discrim- ination under any program or activity receiving federal financial assistance. Avoid disproportionate impact based on income and ethnic groups. Ensure equitable sharing of benefits. Ensure accessibility for people with disabilities. The plan also contains the following system perfor- mance measures or outcomes: • Mobility, accessibility, and reliability: Travel time in key corridors, on-time performance of various modes, trip time reliability, and accessibility for workers within specific minutes of jobs, and proximity to transit. • Productivity: Throughput for persons, vehicles, and goods movement, and effective use of capacity. • System preservation: Asset condition of highways, streets, and roads. Condition of transit and passenger rail facilities and rolling stock. Condition of airport runways and pedestrian and bike facilities. • Safety: Traveler safety, transportation worker safety, and crime statistics at transportation facilities. • Environmental quality: Air quality, noise, energy consumption, ecosystem health, and water quality. Performance measures or outcomes are still being developed for coordinated transportation and land use, economic development, return on investment, and equity. We are proceeding with the next steps including working with stakeholders to develop an initial version of a performance measures report by January 2005. A number of factors appear to have influenced the success of the department’s efforts to date. First, execu- tive management support has been critical. Second, the experience at MTC and SCAG, among others, helped support Caltrans’s work. Third, the 4 years of develop- ment work has started to pay off. Fourth, there was a recognition of the power of performance measures in making good programming decisions. Recognized support for business-to-business rela- tionships was an important factor. There was also a real- ization that shared responsibility for reporting is critical, since no one agency can or should do it all. Cal- trans is now working on organizational performance measures tied to strategic goals and system measures and regional strategies tied to outcomes. A number of challenges remain. First, technology and resources continue to be a challenge. Second, there is a cultural reluctance to deliver bad news. There is also some resistance to constraining local decision making. Transportation in California is a huge system of many modes and service providers. Maintaining a customer focus, not a technician focus, is important because pub- lic support is critical. It is important to report on perfor- mance measures that matter to the customer in a way the customer can understand. Keeping measures outcome- based and ensuring that the right projects and services are delivered efficiently and effectively are also critical. THREE DIMENSIONS OF PROJECT DELIVERY MEASUREMENT AND REPORTING Gregory Selstead I appreciate the opportunity to provide additional infor- mation on the use of performance measures at the Wash- ington State Department of Transportation. Yesterday, Douglas MacDonald, secretary of the department, shared his perspective on using performance measures in the political decision-making process. My presentation focuses on the internal use of performance measures in the department. After 12 years of no new revenues, the department now has an ambitious program based on the recently passed Nickel Tax. With these additional resources it becomes even more critical that the department deliver projects on time and on budget. Accountability within the department focuses on implementation and delivering on the commit- ments made to the public and the legislature. The department’s Gray Notebook and 2003–2007 Business Directions outline the performance expected within the department. Staff within the department are taking ownership of the performance measures included in the documents. It is important to set understandable goals. We also need to clearly show tangible benefits. Finally, performance measures need to be understandable, simple, and sustainable. The first section of the 2003–2007 Business Direc- tions presents the department’s mission and manage- ment principles. One of the management principles addresses delivery and accountability. This principle focuses on providing taxpayers and the legislature with the highest possible return of value. More detail is provided on defining delivery and accountability in the published measures section. Exam- ples of measures include planned versus actual expendi- tures, cost comparisons for contracts awarded, and cost comparisons for contracts completed. There is also an emphasis on the safety program. There is a statement advocating “frequent exchange and support between headquarters and regional staff to overcome obstacles to program delivery.” This state- ment supports the need to be proactive in addressing issues that may arise. Every quarter a team of head- quarters staff meets with regional staff to stress the com- mitment to the performance measures and to identify and address any issues. The first section of the Gray Notebook contains a report to the transportation commission, citizens, other 2 9PERFORMANCE-BASED PROGRAM DEVELOPMENT AND DELIVERY 99395mvp9_38 12/13/05 12:24 PM Page 29

officials, and the legislature on achievements, shortcom- ings, and challenges in the department’s performance. This statement initially caused concern among some groups within the department. It acknowledges that the delivery of transportation projects is difficult and that the department will be forthright in discussing any shortcomings and challenges. The Gray Notebook contains the universe of projects in the 10-year capital construction program. This pro- gram presents all the capital projects, including the preservation program, the improvement program, fer- ries, rail, facilities, advanced technology, local pro- grams, and special projects such as the Tacoma Narrows Bridge. A summary of advertised projects, awarded projects, and completed projects is published every quarter. Projects that are delayed or deferred are noted, along with an explanation of the reasons for the delay. Construction highlights are presented for every project. There is also a section on project delivery addressing proposed and potential adjustments to delivery plan- ning. In the past, the legislative appropriation was by program. With the passage of the Nickel Tax, a line- item appropriation process is now used. Each project now has a fixed funding amount and fixed milestones in the 10-year plan. This section presents any problems or issues, such as change orders, that may come up on a project. The opportunities and options section presents proposals that require legislative guidance. The Gray Notebook provides more detailed informa- tion on scheduled project advertisement dates. We have incorporated the construction season into this report. Previously, a project was listed as late if the advertised date was missed. Now, projects are not considered late if the advertised date is missed but the targeted con- struction season is met. The Gray Notebook is an important document for internal department use and for communicating with the legislature and the public. I look forward to further discussion at this conference on performance measure reporting. Thank you. 3 0 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS 99395mvp9_38 12/13/05 12:24 PM Page 30

3 1 ISSUES AND CHALLENGES IN USING EXISTING DATA AND TOOLS FOR PERFORMANCE MEASURES Louis H. Adams It is a pleasure to have the opportunity to participate in this conference. I thank the two coauthors of the resource paper, Frances Harrison and Anita Vandervalk of Cambridge Systematics, Inc., for their assistance. The purpose of this presentation is to highlight some of the technical issues associated with using existing data and tools for performance measurement at trans- portation agencies. Agencies rarely have the luxury of embarking on completely new data collection efforts and acquiring new information systems and analysis tools. Most agencies typically must rely on existing data and tools. This is not necessarily a hardship, however, since most agencies collect data on system condition, performance, supply, and demand. The challenge is to take advantage of existing data flows and tools. The resource paper identifies common challenges and offers recommendations to allow agency staff to anticipate and address these challenges in a proactive manner. My presentation focuses on five topics. I will begin by providing some suggestions for agencies just getting started in using performance measures. Second, for sea- soned practitioners, I will highlight a continuous improvement strategy. Third, I will cover issues associ- ated with specific types of performance measures. Fourth, three business processes will be examined. These business processes are defining measures, gather- ing data, and using information flows to make deci- sions. I will close by highlighting a few additional resources available to practitioners. If you are just starting the process of performance- based planning and budgeting, you have the luxury of learning from the experience of other agencies. Imple- menting data collection need not be costly, time-consum- ing, or resource intensive. A simple initial program can be established by using a single indicator that is of interest to an executive-level sponsor. Targets can indicate whether a performance rating is satisfactory or needs improvement. The program can evolve incrementally over time as the agency learns from the initial experience, and more per- formance measures can be added. Improvements in the sophistication of measures can also be incremental, as can further integrating the use of the measures into strate- gic and tactical decision-making processes and communi- cating results to a wider group of customers, partners, and stakeholders. This evolutionary model of perfor- mance measurement improvement is typical in agencies that have successful programs. Agencies experienced in performance-based planning and budgeting face challenges in the area of business process refinement. Typically, their continuous improve- ment strategy will require effort on multiple fronts: • Measuring the right things at an appropriate level of detail; • Taking advantage of current technologies and tools for data collection, processing, and analysis; • Making the best possible use of existing data and legacy systems; Data and Tools Required to Support Decision Making Sandra Straehl, Moderator, Montana Department of Transportation Louis H. Adams, New York State Department of Transportation Anthony Pietropola, GeoDecisions Jeff May, Denver Regional Council of Governments Mark Wolfgram, Wisconsin Department of Transportation Leonard Evans, Ohio Department of Transportation 99395mvp39_69 12/13/05 12:29 PM Page 31

• Enhancing tools over time to provide better deci- sion support; and • Building the staff capability and commitment required to ensure that quality information and analyses are actually used to make decisions. Infrastructure preservation measures are used by most state transportation agencies. A condition inven- tory is fundamental for this measure. Condition inven- tories provide a comprehensive summary of the number, age, and condition of capital assets by asset class. A sim- ple condition rating scale that is consistent across asset classes and agency programs is typically used. The scale identifies compliance with current standards, in-service failure rates, and annual maintenance requirements for each asset. Measures of functional deficiency are used to describe how well transportation facilities are serving their intended purpose. Measures of backlog or need can be derived on the basis of standards for condition or func- tional deficiency, or both. For these types of measures, it is important to establish precise criteria for what consti- tutes a need, so that identification of such needs can be an automated process. Varying definitions for what con- stitutes a deficiency should be accommodated, however, because agency policies change over time. Using remaining life as a performance measure is one approach that allows agencies to compare performance across different classes of assets. This approach requires reasonable estimates of the expected life of different types of assets under varying circumstances such as traf- fic, environmental conditions, construction methods, and maintenance practices. Monetary valuation indicates how well the agency is preserving the substantial investments that have been made in the infrastructure. Service improvements form another grouping for measuring performance. Exam- ples of safety measures include number and rate of fatal- ities, injuries, and run-off-the-road crashes. The timeliness and accuracy of police crash reports are improving rapidly as paperless methods replace hard copy documents. The establishment of reliable accident locations that can be correlated to highway design and condition attributes is becoming more widespread as a result of Global Positioning System technology. Increased atten- tion to safety data requirements and new levels of data integration will need to be pursued before the 2008 implementation target of a new federal strategic goal to reduce the number of annual deaths attributable to highway crashes by 9,000. Examples of mobility measures include the travel time index, which focuses on the average congestion level; the planning time index, which focuses on arriving on or ahead of schedule 19 times out of 20; and the aver- age travel rate, which was recommended by NCHRP Report 398. A reliability challenge is processing exten- sive data flows to determine the locations, magnitudes, and durations of disruptions to expected travel time pat- terns. A reliability recommendation is to use simulation and theoretical understanding to estimate performance. Most existing performance measurement efforts have focused on performance from the facility or supplier point of view. Interest in reflecting the customer point of view is growing, including measurement of customer perceptions concerning transportation services. Transportation user advocacy groups frequently use customer-related performance measures in their publica- tions. The Road Information Project and the American Highway Users Alliance express the backlog of needs in terms of excess cost of travel per highway user on an annual basis. The costs attributed to condition and per- formance shortcomings are likened to a hidden tax, which acts as a drag on regional and national economic efficiency in a globally competitive marketplace. Deciding how to measure performance should be based on what an agency is attempting to accomplish, the frame- work of policy goals and objectives that has been estab- lished, and the expectations of customers and partners, rather than the data now being collected. Consideration should be given to the information and the tools already being used to make decisions. I would not suggest suddenly replacing a well-understood measure with a new measure. Rather, use the old measure alongside the new measure until comfort with the new measure is established. Knowing whether goals are met is insufficient infor- mation for a leader to use in making decisions about how to improve agency performance. It is helpful to define a quantitative performance rating scale for each goal, with upper and lower bounds and ranges such as excellent, satisfactory, and needs improvement. It also is helpful to track whether the current performance rating is better or worse than an expected value and the value for the prior reporting period. Agencies have tended to rely on output-type mea- sures rather than outcome measures. Completing 500 lane miles of paving raises the question, “So what?” A better measure might be that this year 82 percent of cus- tomer travel is on smooth pavement, which is 4 percent better than last year and on target with our plan. Agencies need to demonstrate how expenditures of public tax dollars are in fact making things better for cus- tomers than they otherwise would have been. The avail- ability of trend data for the measures that are selected and the data sources to be used is also important, as is assur- ing that changes in measurement or computation methods have not affected the validity of trend lines. Consideration should be given to the impact of changing data collection methods or schedules on the agency’s ability to maintain valid time-series information. 3 2 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS 99395mvp39_69 12/13/05 12:29 PM Page 32

Where existing data sources are to be used, it is impor- tant to evaluate their accuracy, precision, timeliness, and consistency. If data quality improvements are warranted, it is suggested either to make incremental enhancements or to develop a strategic data plan for the agency. Ensuring the accuracy and consistency of fundamen- tal measures such as system mileage and vehicle miles of travel (VMT) is important. Before combining or com- paring data from different analysis tools or data sources, basic consistency checks on fundamental measures should be conducted. Location referencing and temporal integrity are essential in thematic mapping for project development. Spatial relationships among pavement condition, crash rates, and congestion hot spots must be valid. Temporal integrity is particularly important in map- ping current deficiencies from dated condition inventory records, because projects and work orders that have been completed since the condition survey act to remove cur- rent deficiencies from the map. An example of data inte- gration is the calculation of accident rates, which are derived by matching accident records with VMT. Challenges in data collection, processing, analysis, and distribution can seem either mundane or over- whelming, depending on a manager’s background and interests. However, addressing the challenges is critical to support performance-based decision making. Performance data must be managed as an enterprise asset. Data elements must have a data owner, a data ele- ment definition, a schedule for updating, and a fixed amount of precision. Precise data definitions are partic- ularly important where data from secondary sources are being used to derive performance measures. Processes should be developed for quality checking the raw data and turning the data into an aggregated and value-added information asset accessible to the whole enterprise. When data are collected from multiple pieces of equipment, by way of multiple methods or from mul- tiple sources, consistency of the measurement must be ensured. Location and temporal validity and integrity control systems for all enterprise-level data elements must be compatible. Many data elements are commodities and can be pro- cured from private-sector vendors by low-bid methods at unit costs that are competitive with costs for provi- sion of the same services by a public-sector work group. Examples include automated pavement condition sur- veys, periodic bridge inspections, and highway traffic counting programs. When the performance data are in place, they must be disseminated to decision makers in an effective man- ner, and analytical tools must be available to assist in development of performance targets and investment analysis. Most agencies focus on a small set of perfor- mance measures for external reporting and strategic budgeting. More detailed information is needed for decision making at the staff level. Ideally, the few high- level performance measures can be derived from more detailed measures for presentation to policy makers. Technical staff still have access to the detailed data and assumptions behind the aggregate measures. Use of a desktop geographic information system (GIS) application by well-trained staff can be an effi- cient means of preparing input data for a legacy system to process. The same is true for use of a desktop data- base application. Minor modifications to legacy system source codes are usually worth the effort to enable the system to produce comma-separated variable (CSV) output files of legacy system results. CSV files can be easily read into desktop spreadsheets and GIS applica- tions, which can then be used for analysis, presentation, and printing. Tools that predict future conditions should reflect work that is scheduled or programmed. Keeping infra- structure management systems in synchronization with program and project databases is often a challenge. Efforts are needed to ensure that data structures are con- sistent, so that information can flow between the man- agement systems and the program and project databases. At a minimum, specific work flow processes should be defined to update management systems as the program and project databases change. Tools that provide the capability to predict system performance as a function of investment levels use a variety of methods for identifying needs and determin- ing how the available budget is allocated. These meth- ods should be well understood in selecting or configuring a tool and in determining how to make the best use of an existing tool in the performance target–setting process. It is important to pay attention to costs in using tools to perform investment versus performance analysis. Tools and inputs should reflect proper use of discount- ing methods and should account for inflation. Budgets and work costs should be consistent. For example, if work costs do not include indirect costs, the budgets should be reduced accordingly. Fact-based what-if analysis of alternative funding scenarios and policy choices is a fundamental part of performance-based planning and programming. For many types of performance indicators, simulation tools can be used to help provide an understanding of how future performance may be affected by the quantity, tim- ing, and type of agency interventions and by variations in factors outside the agency’s control, such as growth patterns. Ideally, a technical champion should be desig- nated to run the simulation model and to make sure that it is producing reasonable results. In closing, I encourage you to read the resource paper. It includes examples focusing on each of four critical cat- 3 3DATA AND TOOLS REQUIRED TO SUPPORT DECISION MAKING 99395mvp39_69 12/13/05 12:29 PM Page 33

egories of measures: infrastructure, mobility, safety, and customer service. The examples highlight the use of read- ily available data and standard tools including GIS, desk- top applications, and management systems. In each case, the key to success is the way in which different data sources and tools are used in combination and the processes for establishing priorities and allocating resources. The resource paper is intended to help agencies suc- cessfully navigate the wide array of technical, process, and organizational issues that can be anticipated in using performance measures. A systematic approach to perfor- mance program design that considers the interrelated issues involved in measure definition, data management, and decision support can help agencies avoid major road- blocks and anticipate the nature and extent of the effort that will be required for success. Chapter 4 of NCHRP Report 446: A Guidebook for Performance-Based Trans- portation Planning provides additional guidance on the use of data and tools in performance-based planning. Thank you for your participation. TECHNOLOGY AND TOOLS TO ACCESS DATA FOR DECISION MAKERS: CASE STUDY OF STATE DEPARTMENTS OF TRANSPORTATION Anthony Pietropola It is a pleasure to discuss some of the approaches state departments of transportation are using to present per- formance measure information at this conference. My presentation will cover some of the lessons learned, as well as the historical perspective and the current experi- ence with the use of different techniques for presenting information on performance measures. I will describe the approaches being used at four state transportation agencies. I will highlight what I think are five lessons that have been learned from presenting performance measure information. First, keep it simple. Second, it has been suggested that if a picture is worth a thousand words, then a map is worth a thousand pictures. Using graphics to present key information appears to be effective. Third, everyone wants to use technology to enhance per- formance. Fourth, develop a good website or other com- munication method and people will use it. Finally, a catchy name and logo appear to help people find and remember your information. From a historical perspective, many state transporta- tion agencies began using route logs in the 1950s during the development of the Interstate system. The route logs were initially drawn by hand. During the 1980s, many route logs were converted to computer-aided drafting sys- tems. Route logs have been coordinated or replaced by management systems within a number of transportation agencies. Route logs typically contained a lot of information on the roadway system. Information on road surface types and depths, widths, curves, grades, structures, intersect- ing roads, and railroad crossings was included. Data on traffic volumes, accidents, traffic counter locations, and other elements were also included. The development and use of management systems required a lot of information. Many states have manage- ment systems for bridges, pavements, and other system components. Unfortunately, many management systems were developed around what I call islands of data—that is, the data in the different management systems are not integrated. Some states have developed information portals to integrate available data. Information portals provide business layers from many department of transporta- tion divisions and can be used as decision support tools. Some states have automated their road logs as part of their information portals, which allows many attributes of a road segment to be displayed. Information portals may also provide GIS-enabled intranet applications. They allow users to access many different types of data from multiple business functions simultaneously. Information portals also allow users to query, view, map, analyze, and report on data within the system. Information portals provide consistent informa- tion quickly so that more informed decisions can be made with regard to the transportation network. A number of benefits may be realized by using infor- mation portals. First, the decision-making process may be improved and accelerated through easier access to reliable data. Information portals may also improve consistency in responses to internal decision makers, the public, other state agencies, and legislators. The Information Network for Online Resource Map- ping (INFORM) at the Delaware Department of Trans- portation provides an example of an information portal. INFORM contains data on the road inventory, pavement conditions, highway safety improvement program, aver- age annual daily traffic (AADT), projects, bridges, envi- ronmental factors, and storm water conditions. Figure 1 illustrates the home page of the INFORM Internet site. Figure 2 highlights the layering capabilities of the INFORM system. As illustrated in Figure 3, the INFORM system provides users with the capabilities to drill down for more detailed project and site-specific data. The INFORM system is used for a variety of applica- tions. Future plans include providing additional report- ing and video logging capabilities. The department also plans to integrate the system with intelligent transporta- tion system elements including cameras, routing, and geocoding. Other future efforts include adding Highway Performance Monitoring System (HPMS) data reporting 3 4 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS 99395mvp39_69 12/13/05 12:29 PM Page 34

3 5DATA AND TOOLS REQUIRED TO SUPPORT DECISION MAKING FIGURE 1 Delaware INFORM home page. FIGURE 2 Delaware INFORM layers. 99395mvp39_69 12/13/05 12:29 PM Page 35

3 6 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS FIGURE 3 Example of INFORM drill down capabilities. FIGURE 4 Example of Vermont Agency of Transportation’s route selection tool. 99395mvp39_69 12/13/05 12:29 PM Page 36

and transit ridership and linking digital documents to the mapping layer. The Vermont Agency of Transportation’s automated route log system provides a second example. This Inter- net site provides a tool to create and view route logs for the entire state. The agency uses scalable vector graph- ics software and a GIS server. The system provides map view and selection and data linkages. Figure 4 illustrates an example of available route selection tools on the sys- tem. The agency also provides drill down capabilities for more detailed information. The third example is the Oklahoma Department of Transportation’s Geographic Resource Intranet Portal (GRIP). GRIP contains a wide range of data. Theamatic maps and reports can be developed for surface type, AADT, access control, functional class, number of lanes, terrain area type, and many other variables. Con- dition information includes surface condition rate, capacity adequacy, shoulder condition, foundation condition, total condition rate, total design rate, and total sufficiency rate. Data on accidents, pavement con- dition, bridge condition, and a number of other ele- ments are included. GRIP provides drill down capabilities for users. Information can be mapped. GRIP includes photographs of different elements of the transportation system. Finally, I will give a live demonstration of the Penn- sylvania Department of Transportation’s Transporta- tion Information Program (TIP) Internet site. Figure 5 shows the home page of the TIP Internet site. TIP has many of the same features as the systems just described. In summary, tools are available to link available data on the transportation system to provide agency techni- cal staff, as well as policy makers, with useful informa- tion. As I noted at the start of my presentation, keep the presentation method simple, use graphics, match tech- nology to need, develop a good website, and develop a logo and catchy name. Thank you. 3 7DATA AND TOOLS REQUIRED TO SUPPORT DECISION MAKING FIGURE 5 Pennsylvania Department of Transportation’s TIP home page. 99395mvp39_69 12/13/05 12:29 PM Page 37

ROLE OF REGIONAL TRAVEL MODELS IN DEVELOPING PERFORMANCE MEASURES FOR CONGESTION Jeff May It is a pleasure to have the opportunity to talk about the role regional travel models can play in developing and using performance measures. I will share some of our experience in the Denver area and describe the process we used. The Colorado Department of Transportation is responsible for the statewide performance measures and management systems. Two management systems—pave- ment and bridge—are well advanced and have been used for a number of years. The safety management system is in the intermediate stage of development and is begin- ning to show some results. The mobility or congestion management system is just in the initial stages of devel- opment. Like other states, Colorado has focused pri- marily on fixing and maintaining the existing transportation system, with expanding the system a lower priority due to lack of funding. Traffic congestion is a concern in the Denver metro- politan area. Congestion is the major transportation problem identified by front-range residents, including those in the Denver area. Traffic congestion results in lost time and lost productivity. Residents and businesses have made policy makers aware of their concerns about increasing levels of traffic congestion. The Colorado Department of Transportation and the Denver Regional Council of Governments (DRCOG) have been working on developing a funding allocation process for mobility projects for a number of years. In July 2004, the department and DRCOG agreed on a congestion measure for allocation of mobility funding to regions. For the urban areas, VMT on congested roadway segments will be used. Congestion is defined as a volume-to-capacity (V/C) ratio greater than 0.85 based on design hour, which is the 30th-worst hour of the year. The agreement also states that both agencies want to move toward more sophisticated performance mea- sures. The measures may include traveler hours of delay, freight delay, duration of congestion, and travel time reliability. There was also agreement, however, to move toward more sophisticated measures only if significant funding becomes available to address congestion prob- lems. Currently, the Colorado Department of Trans- portation has allocated $8 million statewide for congestion projects. More sophisticated measures would be used only if they would change decisions sig- nificantly. Finally, more sophisticated measures would be explored with a target of additional data collection and analysis costs being less than 1 percent of mobility funding dollars. The 1 percent figure reflects the amount spent on data collection for the pavement and bridge management systems. The challenge facing both agencies is how to improve Colorado’s mobility performance management system with existing or low-cost sources of information and analysis tools. We first examined available data and tools that could be used. The department’s files contain information on every link of the state highway system. Information such as AADT by state highway segment statewide is included. In the case of freeways, the aver- age daily traffic is based on a few permanent traffic counters extrapolated to estimates for the whole system, while arterial roadways are actually counted every 3 years. The department calculated the capacity of road- ways in the 1990s. These calculations have not changed, with the exception of links where lanes have been added. Another available tool is the four front-range urban areas’ regional travel models and the department’s I-70 model. These models cover the most congested loca- tions in the state. DRCOG also compiled samples of hourly traffic volumes from local and state sources. Data on speeds and vehicle classification counts are available. The Denver regional travel model was initially developed in the 1960s for use in planning the Inter- state system. In the 1970s and 1980s, the model was expanded for use in planning rapid transit. In the 1990s it was used for air quality analysis. In the 2000s, I see the model becoming a multimodal land use deci- sion-making tool. Almost $1 million has been allo- cated by the DRCOG board to improve the model to support multimodal decision making. The current four-step model calculates travel between destinations by mode and route. The model is calibrated through household travel surveys, external station sur- veys, and traffic counts and HPMS VMT estimates. Over the past year, additional speed and delay studies were conducted. Speeds and traffic counts were used to calibrate the model. Model outputs include estimates of traffic volumes and speeds on 25,000 highway segments by 10 time peri- ods of the day. The model calculates peak-period week- day V/C ratios. The model also provides congested hours of travel for current days and future year estimates. The model only deals with recurring congestion, not incident-related congestion attributed to crashes and vehi- cle breakdowns. Applying the Federal Highway Adminis- tration (FHWA) methodology to Denver freeways results in estimates of incident delay of about 110 percent of recurring delay. There is a good deal of variation among areas, ranging from 80 percent to 250 percent. The model focuses on the average weekday conges- tion levels. Weekend congestion associated with special 3 8 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS 99395mvp39_69 12/13/05 12:29 PM Page 38

events, shopping, and recreational opportunities is not modeled. We are working to improve the model credi- bility with Colorado Department of Transportation staff. Department representatives have been included in the long-range planning process and the process to enhance the model. The model produces estimates over time and shows geographically where congestion occurs. It can also be used to assess the impact of various improvement pro- grams. The model can reflect evolving definitions of congestion. The current definition of congestion used at DRCOG is V/C > 0.95 for 3 or more hours. Traffic congestion was not a major problem in the Denver area during the 1970s. In the 1980s, traffic con- gestion became a major issue on most freeways and road- ways in the area. This trend continued in the 1990s. The changes in congestion levels can be displayed graphically by using GIS. In conclusion, travel models can play a part in the development of performance measures for congestion. We need to do more work to supplement the model with other elements. Examples of these elements include being able to identify incident-caused congestion and weekend congestion. The Colorado Department of Transporta- tion is also exploring enhancing data collection and analysis tools for rural areas. Thank you. USING PERFORMANCE MEASURES TO IMPROVE HIGHWAY PROGRAM EFFECTIVENESS Mark Wolfgram I appreciate the opportunity to participate in this ses- sion and to share some of the Wisconsin Department of Transportation’s experiences with the use of perfor- mance measures for highway program effectiveness. My comments will touch on some of the topics discussed during this morning’s session and will describe what we have accomplished since the first performance measures conference in 2001. Figure 6 illustrates the department’s highway asset management process. The process is similar to those used in other states and is based on the department’s policies, goals, priorities, and highway program budget. The process begins by forecasting highway performance and needs. A program of projects is then developed to address the identified needs. After development of the program, the projects are constructed, and in turn they influence future highway performance and needs. The asset management process is guided by the department’s policies, goals, and priorities. High-level policies address conditions requiring pavement and bridge improvement as well as capacity, safety, and other elements of system performance. Our long-term goals focus on reducing pavement, bridge, safety, and capacity deficiencies. Infrastructure condition and safety are our two top priorities. Forecasting highway system performance and improvement needs requires a lot of data. Information on the system inventory, system condition, traffic volumes, and crash locations is needed. Such forecasts also require performance models. Our pavement management, bridge management, safety management, and congestion man- agement systems provide this modeling capability and, together, form what we refer to as our metamanagement system. The metamanagement system helps answer numerous program development questions. It helps iden- tify pavements that must be improved, bridges that are deficient, and safety and capacity issues that must be addressed, and it helps identify the appropriate way to address them. Once improvement needs are identified, the next step is to develop a program of projects. At the Wisconsin Department of Transportation, developing the project program is a biennial process. The goal of the program is to focus funding on delivering the right projects, in the right place, at the right time, to meet priority improvement needs. In Wisconsin, as elsewhere, our improvement needs greatly exceed the resources available. The program development process begins by allocating highway improvement funds to two subprograms within our overall highway improvement program. The first sub- program is designed to address needs on what we call our Backbone highway system. This system includes Wiscon- sin’s Interstate highways and other high-volume, multilane routes. The second addresses needs on all other state high- ways. The metamanagement system is used to identify pri- ority needs across the state and the cost of addressing them appropriately. The distribution of priority needs (i.e., bud- get-constrained needs) between Backbone and non-Back- bone routes determines the allocation of resources to the Backbone and non-Backbone subprograms. The distribu- tion of non-Backbone needs by district determines district allocations for addressing the needs on those routes. The Backbone subprogram is then developed and managed cen- trally, while the non-Backbone subprogram is developed and managed at the district level. 3 9DATA AND TOOLS REQUIRED TO SUPPORT DECISION MAKING Forecast Highway Performance and Needs Develop a Program of Projects Build Those Projects Policies, Goals, Priorities, Budget FIGURE 6 Wisconsin Department of Transportation asset management process. 99395mvp39_69 12/13/05 12:29 PM Page 39

This process has evolved slowly over many years. The overall vision was there from the beginning, but district understanding and acceptance were critical to success. Although the effort was led and staffed centrally, each step in developing the process was taken in partnership with district staff. District champions were identified, and the process was developed and implemented as everyone became comfortable with both the steps and the outcomes. Using the process to influence the allocation of resources between subprograms was accomplished early on. Next, the degree of influence was increased until it was direct and immediate. Finally, since our last confer- ence in 2001, several program performance measures were developed for program development staff in the central and district offices. The measures are focused on ensuring that funding is actually used at the right time, at the right place, to do the right thing. There are performance measures to address each of these three program goals. First, we consider whether funds are actually being allocated to projects in the right place. Performance data, policies, and goals are used to divide the transportation system into “needy” and “not needy.” We also identify priorities within the needy parts of the system. Currently, 74 percent of available funding is being spent on segments that are high-prior- ity needy portions of the highway system, 1 percent is being spent on lower-priority needy segments, and 25 percent is allocated to segments that, according to our data, are not needy. Our performance goal is to move toward spending at least 80 percent of available resources on high-priority needy projects. The 80 percent target was selected rather than a 90 percent or higher target for a number of reasons. First, there are limitations in the data and the models. Second, all portions of a logical project segment may not reflect the same high priority, which reduces the measure as cur- rently implemented. Third, in some cases, other factors may call for a project at this time. We also examine the timing of needs to determine whether projects are being undertaken at the right time. Approximately 75 percent of the current program focuses on projects that are needed right now, 19 per- cent addresses projects that are needed within a short time, and approximately 6 percent addresses distant needs. To help ensure that we focus resources on imme- diate needs, a performance measure was set calling for no more than 2 percent of available funding to be used for distant needs. To determine whether we are doing the right thing, we compare the project scope suggested by the meta- management systems with the scope of the projects actu- ally developed. Currently, some 55 percent of available funding is going to projects with scopes very close to those identified by the metamanagement system. Approximately 16 percent of our funding is going to projects that are overscoped, and 3 percent is going to projects that are underscoped. We have a goal of limit- ing resources for overscoped and underscoped projects to not more than 10 percent each. If central or district program managers develop pro- grams inconsistent with any of these goals, a peer-review process with other program managers is used to review the reasons for the decisions being made. Any group using funds on unjustifiable projects is violating the prin- ciples of accountability needed in program development and delivery. To help program managers improve their perfor- mance, geographically referenced data are made avail- able to them. By using GIS, they can easily identify projects on nonneedy portions of the highway system and focus on projects suspected of being improperly scoped. This allows them to explore the reasons for these potential discrepancies before the peer-review process is engaged. The use of performance measures is helping us meet program goals. Currently, 22 percent of our pavements are below the desired thresholds. Our models indicate that some 39 percent of pavements would be below the desired thresholds in 2008 without the program. With the program, 23 percent of pavements will be below thresholds. We still have a number of challenges in the use of per- formance measures. As with other state departments of transportation, we are working to link consequences to performance. These links might include an impact on future funding allocations for each district and an impact on compensation for key managers. Over time, it is our intention to improve our data and modeling processes, slowly raise the bar for program managers, and increase the performance delivered by our highway improvement program. Thank you. TRAPPED KNOWLEDGE, UNLOCKING KEY INFORMATION: WORKING WITH LEGACY DATABASES AND MANAGEMENT SYSTEMS Leonard Evans It is a pleasure to participate in this session and this con- ference. My topic addresses legacy databases and man- agement systems and how they can be used to support performance-based decision making. I will share some of our experiences in Ohio with the use of legacy databases to support performance measures. What is a legacy database? A simple definition of a legacy database is that once a system has been put into service, it becomes a legacy system. 4 0 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS 99395mvp39_69 12/13/05 12:29 PM Page 40

Legacy systems are constantly being created. During development, the programmer has complete control over data structures. After deployment, however, the programmer loses control as other systems interface with the database. We need to be aware that changes to data structures may alter dependent systems, so legacy databases should be continuously monitored. Numerous systems provide transportation data and support the transportation decision-making process. These information sources provide data on road inven- tories, bridges, pavements, signals, crashes, and speed limits. Other systems provide data on traffic control devices, detours, congestion levels, safety locations, and planning. Information on project design, construction, and accounting is also available. Most of these data sources were not originally designed to support asset management and performance-based decision making, however. The Ohio Road Inventory or road log is the base of the transportation information system. It was developed between 1947 and 1958. The road log includes the county, route, log mile, and physical roadway attrib- utes. The road inventory includes data on roads in all 88 counties, 1,309 townships, and 942 corporations in the state. Ohio is the 35th-largest state in the nation. The State Roadway System includes 20,000 miles; the County Roadway System, 30,000 miles; the Township Road- way System, 43,000 miles; and the Municipal Roadway System, 25,000 miles. The total roadway system in the state covers approximately 118,000 miles. One of the key challenges with any information sys- tem is maintaining the data. A process to update and maintain data is needed. With more than 50 years of information on the transportation system in Ohio, it is critical that we maintain the integrity of the data. We do not want to make investment decisions on the basis of incorrect information. In Ohio, formal roles and responsibilities have been defined through legislation and other policies. Ohio is a home-rule state, and the Ohio Revised Code (ORC) establishes the highway classification system in the state and the naming of the roads and bridges. The ORC also defines the duties and responsibilities of county engineers, county commissioners, and township trustees. Obviously, a critical element for a roadway log is a standard method to define transportation locations. The Base Transportation Referencing System (BTRS) provides the standard for identifying transportation locations in the state. The system links the GIS to dif- ferent databases. We encountered problems when we initially tried to map the locations of projects. The maps showed projects in the wrong locations. We orga- nized a team made up of representatives from various departments and districts to examine the problem. We found that we did not have all the necessary data or a standard method to map projects in the GIS correctly. The team developed a system that uses a 14-character Network Linear Feature identification and a three-digit decimal mileage reference. The latitude and longitude are also identified. The system provides the capability to integrate data- bases, which are used to develop mobility measures and performance measures. The BTRS link number provides the connection to information in other databases and management systems. Using computing power and stor- age readily available, we created an integrator file that breaks information down to log points at every 0.01 mile. This file has some 2.1 million records for the state system. Once the data are integrated, we can analyze a wide range of data. For example, our sufficiency index includes information on functionality geometrics, lane width, shoulder widths, curves, bridge and pavement conditions, and accidents. It has been an excellent tool for developing, analyzing, and monitoring performance measures. The system was a great benefit in the recent develop- ment of Access Ohio, the long-range transportation plan. We were able to access and map a wide range of information easily. We also used the system in scoring priority projects. In closing, I would like to note two research studies that have reports and information available on the Inter- net. The first is the FHWA-sponsored study Review of Data Integration Practices and Their Application to Transportation Asset Management, which was com- pleted in 2003. It is available at www.fhwa.dot.gov/ infrastructure/asstmgmt/diindex.htm. The second is NCHRP Project 20-64, XML Schemas for Exchange of Transportation Data (TransXML), which is under way. Information on this study is available at www4. trb.org/trb/crp.nsf/All+Projects/NCHRP+20-64. I look forward to discussing legacy databases and management systems in more detail in the sessions this afternoon. Thank you. 4 1DATA AND TOOLS REQUIRED TO SUPPORT DECISION MAKING 99395mvp39_69 12/13/05 12:29 PM Page 41

4 2 International Use of Performance Measures Michael Meyer, Moderator, Georgia Institute of Technology Michael Halladay, Federal Highway Administration Takayuki Oba, Ministry of Land, Infrastructure, and Transport, Japan Douglas MacDonald, Washington State Department of Transportation Connie Yew, Federal Highway Administration C. Michael Walton, University of Texas at Austin Bob Arnold, Federal Highway Administration Ken Philmus, Port Authority of New York and New Jersey Randy Halvorson, Minnesota Department of Transportation Jeff Price, Virginia Department of Transportation Gary White, Federal Highway Administration SESSION INTRODUCTION Michael Meyer The session this evening focuses on the results from the international scan on performance measures. Mike Hal- laday from the Federal Highway Administration (FHWA) will provide an overview of the international scan program sponsored by FHWA and the American Association of State Highway and Transportation Offi- cials (AASHTO). We will then hear from Takayuki Oba of the Japanese Ministry of Land, Infrastructure, and Transport, whom we met during the international scan. Mr. Oba is responsible for the performance manage- ment office in the transport bureau of the ministry and has been instrumental in implementing a comprehensive performance management program for Japan’s highway program. We are fortunate to have nine individuals who par- ticipated in the performance measures scan here this evening. Each scan member will highlight what he or she considers the most important observations and experiences from the scan. The international performance measures scan occurred in spring 2004. The primary purpose of the scan was to learn about the experiences of other coun- tries in using performance measures and performance management in planning and programming decision making for all modes of transportation. The scan team first visited Vancouver, British Colombia, Canada, and then traveled to Tokyo, Japan. In Australia we visited Brisbane, Queensland; Sydney, New South Wales; and Melbourne, Victoria. The scan ended in Wellington, New Zealand. During our visits, we met with representatives from national ministries of transportation, other transportation agencies, and provincial or state departments of transportation in each location. We also were able to meet with representatives of the Vic- toria Department of Infrastructure; rail and transit planning organizations in Vancouver, Queensland, Victoria, and Japan; and the police department in Victoria. Although the time was limited, the breadth of contacts provided the scan team with a good understanding of how other countries have adopted performance management in their investment and system management decision-making processes. A summary report on the scan will be posted on the FHWA Office of International Programs webpage international.fhwa.dot.gov. 99395mvp39_69 12/13/05 12:29 PM Page 42

FHWA AND AASHTO INTERNATIONAL TECHNOLOGY SCANNING PROGRAM: PERSPECTIVE ON HIGHWAY SAFETY Michael Halladay It is a pleasure to participate in this session and to share information with you on the FHWA and AASHTO inter- national technology scanning program. I will briefly describe the purposes and process of the scanning pro- gram and implementation efforts associated with it. I will also highlight key topics from the safety-related scans, including the findings and implementation activities. FHWA and AASHTO have been operating the scan- ning program for about 10 years. There are four major scanning program objectives. The first is to discover and explore technology used successfully in other countries and to learn from their experiences. The second is to increase our awareness of underlying policies, processes, and capabilities. The third is to determine the applicabil- ity of technologies and tools for use in the United States. The final objective is to foster implementation of appro- priate tools, technologies, procedures, and policies in this country. FHWA and AASHTO established a biennial program of 10 scans. Five priority topics per year are selected for scanning tours. The program began with a technical ori- entation, but the scanning tours conducted to date have covered a wide range of topics, including the use of per- formance measures. Topics of the 2004 and 2005 scans include infrastructure, operations, safety, and organiza- tional issues. I have had the opportunity to participate in a few of the scanning tours. A lot of work goes into selecting the team members, arranging the countries and cities to visit, conducting the scan, and summarizing the results. Each scan includes a chair or cochairs, a recorder, and team mem- bers. One key to selecting members is to include indi- viduals who can help champion the implementation of findings from the scans. Objectives and amplifying ques- tions are developed. Host countries are selected, con- tacts are established, and logistics are finalized. The trips are conducted and the experiences are docu- mented, including possible implementation actions. The core of a scan tour is meeting with representatives from agencies and organizations in the host countries. Detailed questions are provided to the representatives before the trip. Detailed presentations are given on the scan topic, along with site visits and field observations of technologies, operations, and research activities. Figures 1 and 2 provide examples of scanning tour presentations and site visits. There is a rich dialogue among scan tour members and representatives of host countries. I gained a much better appreciation for the different transporta- tion structures and cultures as a result of participating in the scanning tours. The culture and organization struc- tures, especially in the safety area, really set the countries we visited apart from the United States. Safety-related scanning tours have focused on policy, organizations, leadership, tools, and technologies. Other topics include bicycle and pedestrian safety, inter- section safety, highway safety, and intelligent trans- portation systems and safety. Most European countries make managing and organizing for highway safety a top priority. Highway safety information systems are a key part of this focus. The transportation system is also designed and operated with safety as a key element. Sig- nalized intersection safety is a priority, as is examining human factors in research and implementation. The signalized intersection safety scan in 2002 identi- fied five major implementation priorities. The first was developing a model photo-enforcement process and pro- gram. The second was identifying and implementing demonstration projects for enhanced dilemma zone detection. The third was developing a policy project to control speed through intersections by using a combina- tion of European practices. The fourth was to promote single-lane roundabouts as alternatives to signalized intersections. The fifth was developing guidelines and identifying pilot projects to enhance pedestrian crossings. 4 3INTERNATIONAL USE OF PERFORMANCE MEASURES FIGURE 1 Example of scanning tour presentation. FIGURE 2 Example of scanning tour site visit. 99395mvp39_69 12/13/05 12:29 PM Page 43

The human factors in research and implementation scan in 2004 identified practices that could benefit the U.S. trans- portation system. Human factors practices identified for implementation in the United States included the use of inter- disciplinary research teams, simulator research and field val- idation, and cognitive models. Human factors–based initiatives include self-explaining roads, the “2 plus 1” road configuration, and speed management strategies. The managing and organizing for the highway safety scan in 2002 covered Sweden, Germany, the Netherlands, and the United Kingdom. I had the opportunity to partic- ipate in the scan that focused on national safety goals and plans; processes, policies, and procedures to prioritize 3E (engineering, education, and enforcement) decisions; resources, tools, and legislative policies that guide and support 3E decisions and priorities; and examples and results of successful safety programs and activities. A number of themes emerged from this scan. First, highway safety is viewed as a public health issue in the countries visited. Second, national comprehensive and coordinated plans address safety. The plans provide a proactive approach, measurable and deliverable targets, leadership and financial support at the national level, and clearly identifiable safety slogans and philosophy. There is local participation in setting national targets. Local plans and targets are keyed to national plans, and monitoring and evaluating of effectiveness of plans occur at all levels. The plans contain a number of common elements. Among them are required seat belt use, low blood alco- hol content levels and zero tolerance, and speed manage- ment. Other common approaches include conducting road safety audits and addressing vehicle crashworthi- ness, vulnerable road users, and novice driver training and licensing. Other supporting activities include signifi- cant data collection and analysis efforts, along with strong research capabilities and technical expertise focusing on safety. The highway safety information systems scan in 2003 also identified a number of themes. First, safety drives the organizational structure in the countries visited. Data-driven strategic approaches, safety goals, and accountability permeate the organizational structures. Leadership supports safety, and needed resources are allocated for safety projects. Second, information on safety is shared among all agencies. Third, tools are available for efficient, accurate, and timely analysis sup- port. These tools include technology applications and human resources and brainpower that are dedicated to safety. I believe these scans have helped enhance the consider- ation of safety in planning, designing, and operating trans- portation facilities in the United States. We have seen action toward safety as a business driver through data-dri- ven strategic safety programs, improved tools and tech- niques, and legislative proposals. I believe there is also leadership awareness and support for safety programs. Planning for the Safety Summit II is under way. The scanning team members really become the imple- mentation champions. You will hear from members of the performance measures scan later this evening. They will share their enthusiasm and commitment to helping promote implementation of the best practices they encountered during the scanning tour. PERFORMANCE MANAGEMENT OF ROAD ADMINISTRATION IN JAPAN Takayuki Oba I greatly appreciate the opportunity to participate in this session and the conference. I will share with you some of the experiences with the use of performance measures at the Ministry of Land, Infrastructure, and Transport in Japan. My presentation focuses on three main elements. First, I will highlight the history of outcome-based road administration management in Japan. The use of per- formance measures in Japan includes analyzing the effectiveness of implemented policies and projects and using the analysis results to improve future projects, identify new approaches, and enhance budget projec- tions. Second, I will describe the process for developing performance plans for each prefecture, which is coordi- nated with local governments. Finally, I will discuss recent initiatives to enhance accountability. In August 2002, the Infrastructure Development Council provided a number of recommendations that resulted in the movement toward the use of performance measures. The council’s view was that it was important to shift to an outcome-based road administration. The council recommended the establishment of an evaluation system that uses outcome indicators. A number of activities were undertaken in fiscal year 2003 to start the process of road administration man- agement. These activities included establishment of an advisory committee for public management of road administration and establishment of the Performance Management Office in the Road Bureau of the Ministry of Land, Infrastructure, and Transport. The 2003 per- formance plan for road administration was released, and performance plans at the regional level were devel- oped. Finally, the cabinet approved the long-term plan for the development of social infrastructure in Japan. The cycle of management was put into practice in fis- cal year 2004. Key activities included linking budgets to outcomes through the introduction of performance- based budgets. The 2003 achievement report and the 2004 performance plan were also released. 4 4 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS 99395mvp39_69 12/13/05 12:29 PM Page 44

The Performance Management Office, which I repre- sent, was established in 2003. The office is located within the Planning Division. This organizational struc- ture allows the division to work directly with individu- als developing future plans and budgets. The performance plan for road administration includes numerical targets for 17 performance indica- tors associated with five policy themes. Five-year midterm numerical targets were established for each indicator in the Key Plan for Infrastructure Develop- ment. This plan was authorized by the cabinet in October 2003. The scope of the plan includes more than just highway projects to help ensure the integra- tion of all infrastructure plans. The first performance plan was issued in July 2003, and the 2003 achieve- ment report was issued in June 2004. The results were used to develop the budget request for 2005. In April 2004 we began the performance-based budgeting process. I will now highlight cooperation with regional gov- ernments in the use of performance measures. First, we discuss background data, such as the status of conges- tion, with each prefecture or state. This information is used to formulate a performance plan for each prefec- ture. The plan indicates the numerical targets and mea- sures and the projects for achieving these targets. It includes the projects undertaken at the national level. Many prefectures have worked out a succession of performance plans to disclose their numerical targets, as well as the measures and projects that address the unique features and needs of each region. For example, the Tokyo performance plan was developed by a coun- cil composed of representatives from national and met- ropolitan governments. Beginning in 2003, an outcome plan was published to meet the national goals and tar- gets. The plan links the policy targets to outcome indices and major projects. As of December 2003, the 22 prefectures had set up nearly 160 indicators. Approximately half of the indica- tors are original ones for the regions concerned and are different from those in the nationwide performance plan. For example, a prefecture in the northern part of Japan has an indicator addressing the ratio of sections of road where the driving speed is reduced during the winter. The Road Bureau holds regional road management workshops or hearings twice a year to discuss road man- agement strategies in each region. The first workshops were held in February 2004. Future workshops will be held in February and June of each year. In terms of accountability, the Road Bureau is work- ing to establish a system that is useful in actual manage- ment of the roadway system. It is also important for policy makers and the public to know what we are doing. Figure 3 provides a map of time lost due to con- gestion in the Tokyo area. Figure 4 illustrates examples of trouble spots on National Route 16, which is an arte- rial highway around Tokyo. The trouble spots are the interchanges with the national highways. We have found that three-dimensional maps like these are an excellent tool to communicate with policy makers and the public. 4 5INTERNATIONAL USE OF PERFORMANCE MEASURES Time lost due to congestion (million person -hours) Greater Tokyo (Metropolis + 3 Prefectures) 881 Within KenKen ’’ odoodo RoadRoad 793 Within Outer Ring Road 355 0 1000 2000 3000 4000 1 2 3Greater TokyoGreater Tokyo Within Within Kenen’odo Within Outer RingWithin Outer Ring 23% 21% 10% Proportion of nationwide time lost due to congestion by area FIGURE 3 Example of finding trouble spots by means of congestion loss data, Kanto area: time lost due to conges- tion by area (nationwide 3.81 billion person-hours; Kanto 1.239 billion person-hours). 99395mvp39_69 12/13/05 12:29 PM Page 45

Figure 5 provides an example of a map illustrating the benefits derived from a project. It shows the conditions on a national highway before construction of a new bypass and the conditions after the bypass was open. We use these types of graphics to illustrate the benefits of projects and policies. We are still in the early stages of using performance measures and performance-based management in Japan. I look forward to learning more about the approach in use here in the United States during this conference. 4 6 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS ( National Route 16 0.0 200.0 400.0 600.0 800.0 1,000.0 1,200.0 1,400.0 Ti m e lo ss d ue to c on ge st io n/ km (1, 00 0 p ers on ho ur/ ye arl y k m) Average for the directly controlled roads under jurisdiction of Kanto: 148.3 (1,000 person hour/yearly km) Kanagawa Tokyo Saitama Chiba EndStart Yokohama Machida Iwatsuki IC Kashiwa section Chiba section Yokohama Machida two - level crossing Tohoku Expressway Iwatsuki IC Chiba Kashiwa Line Kashiwa section) Chiba Kashiwa Line (Chiba section) FIGURE 4 Example of finding trouble spots by means of congestion loss data, Kanto, National Route 16. After openingBefore opening 15 million people -- hours/year 12 million people -- hours/year 3 million people3 million people -- hours/year New Section FIGURE 5 Analysis of the effect of an individual project in which data were used: change in congestion time loss. 99395mvp39_69 12/13/05 12:29 PM Page 46

INTERNATIONAL SCAN ON PERFORMANCE MEASURES Douglas MacDonald, Connie Yew, C. Michael Walton, Bob Arnold, Ken Philmus, Randy Halvorson, Jeff Price, Gary White, and Michael Meyer Douglas MacDonald: The scan tour was an intense and interesting experience. We visited a number of cities in Canada, New Zealand, Australia, and Japan over a 2- week period. The report to be distributed by FHWA on the scan will help share what we learned. Transportation agencies in the countries we visited seem to have used performance measures more effec- tively in their highway safety programs than we have. New Zealand and Australia have seen dramatic improvements in highway safety performance as a result of this focus. Safety-related performance measurement plays a major role in achieving these results. Perfor- mance measures are important to identify targets, ana- lyze needed changes, and track results. I believe most of us came back with a sense that the United States can do more in this area. In Tokyo, the scale of transportation systems is huge. We visited with representatives from Japan Railway and toured the Central Train Station, which is being rebuilt. Four million people travel through this station on a daily basis. A multibillion-dollar infrastructure replace- ment project is also under way at the station. Work on this project is done between 1:00 and 4:00 a.m., when trains are not operating. In New Zealand we visited an agency charged by the central government with producing a plan to cut road use by 20 percent. The agency was struggling with how to accomplish this goal while meeting other goals related to mobility and sustainability. I found the scan to be beneficial, and I greatly appre- ciated the opportunity to participate. There is much to learn from the experience in other countries. I believe all the team members came back a little changed by the experience. Connie Yew: One of the themes we encountered at many of the agencies we visited was an interest in pro- viding greater accountability and visibility to the public. This interest in communicating performance measures with customers relates directly to many of the topics dis- cussed at this conference and was the focus of a breakout session earlier today. A few agencies we visited are fairly advanced in the use of performance management. These agencies link planning, decision making, and project delivery. Agen- cies in Queensland, Australia, are using sophisticated planning and reporting methods. Planning in Queens- land is more accountable and visible to the public. The agency prepared five reports to document the status of planning and project delivery. The first presents a top- level policy plan that identifies the desired outcomes. The second is a strategic plan that outlines the perfor- mance measures and the strategies to accomplish the desired outcomes. The third presents a 2-year road investment plan. The fourth highlights the annual accomplishments. This annual report presents financial performance measures and major activities during the year. The final report addresses the management and performance of Queensland’s roads. This report pre- sents performance measure results in an easy-to-under- stand way. I found this final report interesting and useful. It provides a model for transportation agencies in the United States to consider. I also found interesting a report published by Aus- troads, an organization similar to AASHTO, on national performance indicators for Australia and New Zealand. The report has been published annually for the past 7 years. It provides important time-sensitive information to the transportation community and the transportation industry. The report acknowledges the differences in performance measures, data collection techniques, and operations in the various states in the two countries. Supporting information is provided to explain and address these differences. The report pro- vides a benchmark to facilitate discussions at the local, state, and national levels in Australia and New Zealand. I believe we can learn from these and other examples from the scan on enhanced methods to communicate performance measures with the public and policy mak- ers. The agencies are interested in being more account- able and visible. The examples also highlight the connections among the various reports. The examples can be used as models to enhance the way we reach out to the public, which is important in communicating our message. The priority placed on safety was also evident. We can learn a lot about the approaches used in the countries we visited to improve safety on our roadways and other transportation facilities. Thank you. C. Michael Walton: I greatly appreciated the oppor- tunity to participate in the scan tour. The composition of the team provided an excellent mix of expertise and areas of interest. The chemistry among team members was outstanding and made the entire experience more rewarding and enriching. Mike Meyer did an excellent job as the reporter and was responsible for developing the final report. I thank everyone who participated in the tour, especially Mike for his work on the report. I believe everyone takes pride in it. I have a few observations. First, as noted by other team members, safety is clearly a high priority in the countries we visited, and the results from the implemen- tation of various strategies in reducing the number of fatalities are impressive. 4 7INTERNATIONAL USE OF PERFORMANCE MEASURES 99395mvp39_69 12/13/05 12:29 PM Page 47

Another issue that struck me as important was the ongoing organizational and institutional changes in many government agencies. Agencies are separating policy and management from the delivery of services and projects. The changes occurring within most agencies are a top- down process being driven by financial concerns and bud- get limitations. These changes are resulting in a dramatic reduction in public forces. The issues of accountability and transparency are key drivers of change. A related point was the transition of the culture at many public agencies, given the changing role of public institutions. I had the privilege of visiting some of the countries 5 years ago and noted the progress, or at least the changes, that have occurred during the intervening period. There is a greater recognition of the importance of data—qual- ity and quantity—in the ultimate choice of performance measures. This is particularly important for agencies undergoing significant changes in outsourcing, privati- zation, or downsizing. Knowing what you are getting for your investment is deemed critically important. Many of the agencies we visited are moving aggressively toward design–build–finance–operate, design–construct– maintain, and build–operate–transfer approaches. There is a growing realization that data are critical in gauging how well these approaches are working. The agencies recognize that it is important to have a strategy to collect needed data and to maintain these systems. Stewardship was another term used frequently by representatives at many agencies we visited. Intelligent transportation systems are another major focus among many of the transportation agencies vis- ited. Technology is being considered to enhance the operation of the transportation system. For example, alternatives to loop detectors are being explored and tested in Japan. Reliance on loop detectors has become problematic given their maintenance experience, relia- bility issues, and the high traffic volumes on the Tokyo expressways. Alternative technologies, such as the use of probe vehicles, are at various stages of deployment. Another procedural requirement is the use of cost–ben- efit analyses and before-and-after studies in the evalua- tion of alternatives and in determining the effectiveness of an investment in achieving expected results. Finally, the agencies we visited are interested in defin- ing environmental performance measures. However, they are struggling to identify appropriate measures, data collection techniques, and reporting methods, as are transportation agencies in this country. Bob Arnold: I enjoyed the opportunity to participate in the scan. I believe all of the team members learned a lot from the countries we visited. There were a number of things that I found of interest in the site visits and in our meetings with agency personnel. First, performance measures are being used by trans- portation agencies and are being monitored in the coun- tries we visited. Many of the performance measures are similar to those in use by transportation agencies in the United States. There were some differences, however, especially in the areas of safety, the environment, and quality of life. Second, I was surprised that the insurance industry is active in transportation in many of the countries we vis- ited. The insurance industry is involved in transporta- tion projects and uses performance measures to select and fund some projects, particularly in the safety area. We can learn from the experience with the use of per- formance measures in some of the difficult-to-measure areas in the countries we visited. Many transportation agencies in the United States are struggling with this. Certainly, we can learn from the performance measures used in the safety area. Ken Philmus: I appreciated the unique opportunity to participate in the scan. I also appreciate the recognition by AASHTO and FHWA of the importance of tolling as a method of raising funds not only for added improve- ments and capacity but also for traffic and congestion management. We saw this approach in the four countries we visited. One of the things I noted was the involvement of the public in the transportation performance measures process. The public is our customer, and it was interest- ing to see how agencies in other countries involved the public in developing performance measures and report- ing the results of ongoing monitoring efforts. Too many times managers and policy planners in this country think we know and understand what the public wants without asking. We are often surprised that we really do not know what the public wants when we actually ask for opinions. In all of the countries we visited the public was involved in some way, either through user satisfac- tion surveys or participation in meetings to help develop performance measures. I believe we can learn from the methods used in the countries we visited to involve the public in setting performance measures more directly. Coming from New York City, I deal with major traf- fic congestion on a regular basis, and I was not totally overwhelmed by the traffic in Tokyo. I must admit that I was overwhelmed by the multimodal approach in Tokyo, however. I am responsible for one of the busiest bus terminals in the world, and it pales in comparison with the Central Train Station in Tokyo. The Tokyo sta- tion was busier at noon than our station is during the peak periods. It was just an amazing place to see. The agencies we visited are using congestion perfor- mance measures and reliability performance measures. These measures were not focused solely on passengers. Too often we think of the commuter and travel in per- sonal vehicles, but the movement of freight and goods is equally important. Trip-time reliability is also impor- tant. We all will put up with a certain amount of con- 4 8 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS 99395mvp39_69 12/13/05 12:29 PM Page 48

gestion. While we want to reduce congestion levels, most commuters tell us that trip-time reliability is more important. Trip-time reliability and other related mea- sures are being used in other countries, in addition to measures addressing congestion. It was also interesting that tolling is used in the coun- tries as a way to reduce congestion as well as to raise funds. Not only is electronic tolling used, but open road tolling is used in some areas and is planned in other areas. Open road tolling means there are no tollbooths and enforcement of toll payments is done through the use of cameras and other advanced technologies. I was also interested in regional and local agency involvement in the state performance measures. I believe it is critical that local entities and state agencies be aligned. We saw some good examples of coordination among state, regional, and local agencies in the devel- opment and use of performance measures. We still struggle with this type of coordination, especially in coordinating transportation and land use. Finally, there is one thing that I did not see in the countries we visited. As you know, the Port Authority of New York and New Jersey was terribly disrupted by the events of September 11, 2001. We constructed, owned, and operated the World Trade Center. Our offices were in the World Trade Center, and when those buildings collapsed our agency came to near collapse. We have been rebuilding our agency ever since. My job has changed from being a transportation expert to being more of a security expert. As are agencies in the United States, those we visited are struggling with how to measure security. How do you know that you are spending the right amount of funds in the right places to accomplish the right kinds of things when the only measure you have is that a terror- ist attack did not happen? I did learn that the risk and vulnerability studies we are conducting at the Port Authority seem to be on the cutting edge, as are the changes we are implementing to reduce risk and vulner- ability. No one in the world is using these approaches, and I answered numerous questions about our efforts. Thank you. Randy Halvorson: I believe all the team members, myself included, greatly appreciated the opportunity to participate in the scan. We joked why people in their right minds would agree to travel 25,000 miles on 12 flights over 18 days, however. The answer is found in the 21 observations contained in the final scan report. I will highlight four of the observations. I will note how these observations relate to what we are doing at the Minnesota Department of Transportation and how some initiatives that are under way have already been influenced by the scan. The first observation relates to safety. As other speak- ers have noted, safety is given a high priority in the countries we visited. The breakthrough in reducing fatalities in Australia is especially dramatic. Minnesota’s first comprehensive highway safety plan was still in the development stage when I returned from the scan. We were able to use some of the information from the scan in the development of the final plan. I am pleased to say that the highway safety plan includes 10 emphasis areas—four in infrastructure and six in behavior. Before we incorporated what we learned from the scan, the emphasis areas probably would have been eight in infrastructure and two in behavior, and the behavior performance measures would have been included as an afterthought. The lessons we learned in Australia showed the power of behavioral interventions in reducing fatalities on the transportation system. The second observation relates to the power of per- formance measures in educating both elected officials and the public. I would take that one step further. I believe performance measures can actually be used to change the nature of the public debate. The third observation relates to the use of visualiza- tion. How we present performance measure informa- tion and display data to elected officials is important. There is a new National Cooperative Highway Research Program (NCHRP) project in the 8-36 series called Effective Organization of Performance Measures. Part of this project will examine visualization techniques for presenting performance measures. It will include best practice examples. My final observation relates to benchmarking, which is being used extensively in Australia. An NCHRP panel in the 20-24 series is currently examining the use of bench- marking in this country. I believe we have a lot to learn about benchmarking in the United States. While some groups are concerned about the use of benchmarking, I believe it is worth pursuing. I appreciated the opportunity to participate in the scan and look forward to helping champion these and other observations. Thank you. Jeff Price: Many of the things that I found most inter- esting during the scan have already been mentioned by other members of the scan team. As other speakers have noted, the scan provided a wonderful opportunity to visit with staff from the different agencies and to see the vari- ous transportation facilities. The same basic principles and approaches to performance measurement that we have been talking about at this conference are being used in many of the agencies we visited. It was interesting to see how they are taking the same principles and applying them to their circumstances. I believe many of the differ- ences in areas of interest and different applications of per- formance measures reflect differences in the culture and in the political structure or the current political agenda. For example, it was interesting to see the different approaches being used to address customer satisfaction, 4 9INTERNATIONAL USE OF PERFORMANCE MEASURES 99395mvp39_69 12/13/05 12:29 PM Page 49

especially those aimed at providing a pleasant and satis- fying experience for public transportation passengers. In Tokyo, passengers are alerted when a bus is approaching a congested area. In Melbourne we saw variable message signs that gave the expected travel times for automobiles versus buses. In combination with bus-only lanes and a bus priority initiative, showing expected travel times has been a good stimulus for people to use public trans- portation. In general, I was impressed with the emphasis placed on encouraging the use of public transportation. While this trend is in part due to many of the organiza- tions we visited either owning or operating the transit systems, it also reflects the different attitude they have toward public transportation. I believe you will find the final report to be of great value and use. It is well done and contains the observa- tions from the team. As other speakers have noted, we can learn from the experiences in the countries we visited. Gary White: I greatly appreciated the opportunity to participate in the scan. As other speakers have indicated, the use of safety performance measures was one of the most notable observations from the scan. The safety per- formance measures in these countries provide examples that are transferable to the United States. Another important point that we heard several times related to starting small and building on success. Rather than waiting for the perfect performance measure to emerge, develop an initial set of performance measures, adjust them as needed, and learn as you monitor and eval- uate performance over time. If you wait for the perfect measures, it will never happen. Another observation concerns keeping the use of per- formance measures simple. Rather than starting with too many performance measures, identify a few key measures in the beginning. Focus on collecting the data needed to monitor and evaluate these key measures, and expand to more measures as appropriate. There were a few performance measures that I found of interest. One internal performance measure was the percentage of employees who had professional develop- ment plans. The Virginia Department of Transportation has a similar internal performance measure, but what I found of interest was the use of a percentage of the agency payroll allocated to a strategic training plan. Another performance measure that I found of interest related to the percentage of projects that include the provision of walking and cycling. Michael Meyer: My colleagues have covered many of the highlights from the scan. As the last speaker, I will focus on a few observations that I consider to be impor- tant, without repeating what we have already heard. In particular, I want to emphasize lessons learned that relate to this conference. Many of you have heard me talk previously about the importance of vertical integration in information flow and decision making within organizations. The paper I presented at the first national performance mea- sures conference in 2001 included a figure that helped illustrate this concept. The figure was a triangle that had different tiers of decision making with arrows going up and down connecting these different levels, representing the important connections in information flow and deci- sion-making accountability that need to occur within an organizational decision-making structure. The arrows illustrated how performance measures could cut across different levels of decision making and provide impor- tant information for decisions made at different levels in an agency. I was greatly surprised and impressed that we saw this concept of performance measurement in Australia and New Zealand. Using a pyramid diagram, agencies in the Australian states and in New Zealand portray performance measures as starting at the highest level of an agency’s decision making and flowing down. Perfor- mance measures are also used to provide consistency in decision making among other agencies, traffic manage- ment centers, and private contractors. The cases studied provided excellent examples of vertical integration. Per- formance measures need to be placed in the context of a performance management structure that integrates an organization’s decision-making process. I was also impressed with the widespread use of before-and-after studies in the countries we visited, especially Japan and Australia. Before-and-after studies are conducted on 10 to 15 percent of all the projects implemented in some of the Australian states every year. The results of these studies provide a feedback loop to help evaluate the benefits of the projects and the useful- ness of specific performance measures. Most trans- portation agencies in the United States do not conduct extensive before-and-after studies of projects. I believe we can learn about the benefits of these types of studies from the countries we visited. Benefit–cost analysis was used both before project implementation and after projects have been in place for 2 years. The before benefit–cost analysis is compared with the after benefit–cost data to develop what is called an achievement index. If the achievement index varies, which indicates that the benefit–cost analyses do not match, or in other words that the initial attempt at ben- efit–cost analysis was faulty, an assessment is conducted to determine what factors caused the difference. The results of this assessment are linked to the initial assumptions and performance measures. As other speakers have noted, agencies in the coun- tries we visited are struggling to develop performance measures for environmental quality and other hard-to- measure features. Performance measures addressing environmental quality tend to be similar to those used in the United States. While there is an interest in develop- 5 0 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS 99395mvp39_69 12/13/05 12:29 PM Page 50

ing better measures, most agencies have not progressed any further than have agencies in this country. We did see an emphasis on rural transportation issues in a few areas. Brisbane is the major population center in Queensland, Australia. However, the transportation needs of the rural parts of Queensland, especially iso- lated communities, are considered to be an important equity issue for the Queensland government. Providing truck and rail transportation to these communities is the means of providing connections to the rest of the state, and performance measures are used to determine how effective these rural transportation services are. Some agencies in the four countries had performance measures focusing on quality-of-life issues. For exam- ple, in Japan one performance measure relates to ensur- ing that pregnant women are within a threshold travel time of a maternity hospital. In other words, the perfor- mance measure focuses on making sure that women having babies can get to the hospital quickly. It was interesting to see the use of performance measures to address issues such as this. Finally, it was interesting to see the link between per- formance measures and the political process in the coun- tries we visited. In most of the countries visited, performance measures are developed at the government level first and then flow down to agencies. For example, the national parliament in Japan provides the initial direction on performance measures. The state govern- ments or parliaments in Australia also provide direction to state transportation agencies. These general policies are used to develop more specific agency performance measures. My sense is that in the countries we visited there is less concern about trying to get the attention of the politicians because they are already actively involved from the beginning. I greatly appreciated the opportunity to participate in the scan. Team members who were not able to be here tonight include Hal Kassof from Parsons Brinckerhoff, Doug Rose from the Maryland State Highway Adminis- tration, and John Baxter from the FHWA Office of High- way Safety. Tony Kane from AASHTO also participated in part of the scan tour. 5 1INTERNATIONAL USE OF PERFORMANCE MEASURES 99395mvp39_69 12/13/05 12:29 PM Page 51

5 2 Measuring Performance in Difficult-to-Measure Areas Michael Meyer, Moderator, Georgia Institute of Technology Timothy Lomax, Texas Transportation Institute Randy Halvorson, Minnesota Department of Transportation Barbara Ivanov, Washington State Department of Transportation CONGESTION AND RELIABILITY PERFORMANCE MEASURES Timothy Lomax It is a pleasure to have the opportunity to talk about per- formance measurement issues related to traffic congestion. My comments focus on the types of decisions congestion measures might be used with, possible congestion and reli- ability performance measures, data considerations, and research needs. It is important first to examine what we are trying to measure and why we are trying to measure it. Perfor- mance measures should be considered within the con- text of a variety of urban goals. Goals addressing improved mobility might include lower travel times and more reliable travel conditions. Mobility goals may focus on the movement of both people and freight. Con- sideration may be given to a variety of time-of-day peri- ods and to regular and special events. Urban and rural situations should be considered, as should different modes and possible pricing structures. It is also important to remember that congestion per- formance measures are of interest to a wide range of groups, including the public, travelers and shippers, the business community, elected and appointed officials, agency leaders, and agency staff. We need to be able to communicate with all these groups. Different communi- cation techniques and messages are typically needed with these various groups. We also need to look at the potential actions or end results with the use of congestion performance mea- sures. Influencing changes in travel might be one out- come. Actions by agency staff to address problem areas identified through the use of specific measures may be another result. Business location decisions may be influ- enced by congestion levels in a corridor or an area. Trav- elers and freight shippers may make routing decisions on the basis of congestion. Survey results from around the country indicate that the public is aware of congestion issues and congestion levels. Figure 1 tracks the results of an annual public perception survey in Houston, Texas, and the hours of delay per peak-period traveler in Houston. The line with the square boxes indicates the percentage of survey respondents ranking traffic congestion as Houston’s major problem. The line with the diamonds shows the delay per traveler value for Houston. As Figure 1 indicates, congestion was a major prob- lem in Houston until the mid-1980s, when the city expe- rienced a major downturn in the economy at the same time that there was a significant expansion in trans- portation system capacity. Congestion levels began to increase in the mid-1990s as Houston’s economy recov- ered. The public’s perception of congestion, as recorded through the annual surveys, mirrors the trends seen in the traffic data. I believe that the public does understand congestion-related performance measures. The chal- lenge is to match measures to issues of interest to the public. We also need to think about the types of decisions agencies and travelers can make on the basis of infor- mation from performance measures. Those related to congestion and reliability might influence decisions 99395mvp39_69 12/13/05 12:29 PM Page 52

about what to do, what to spend, when to act, and what to change. We also need to consider how important or immediate the decisions and related actions are. Congestion performance measures are related to a variety of other measures and issues associated with the quality of life in an area. Some of these issues are acces- sibility, air quality, business productivity, economic development, safety, schools, health care, libraries, and housing. It is important to think about congestion per- formance measures in a competitive sense. Congestion- relieving projects are not just competing with other transportation projects for limited funding; they are also competing with other urban priorities. We need to have performance measures that work in real time. We need short-term or day-to-day perfor- mance measures and long-term measures for annual reporting and future estimates. We also need to set tar- gets for the various performance measures. We can anticipate that these targets will change as people’s expectations change. The targets will also change by location and by time of day. Most people do not expect to be able to drive 50 mph on a freeway in the morning peak hour, but they do on a weekend. It is important to talk about the benefits as well as the costs of various strategies. There are a number of sources of unreliability related to travel time and trip reliability. The Future Strategic Highway Research Program Reliability Research Proj- ect identified seven sources of unreliability: traffic inci- dents, weather, work zones, traffic variation, special events, traffic control problems, and bottlenecks. Trans- portation agencies can address many of these sources of unreliability and may have more success in reducing the day-to-day differences in travel time than in reducing average congestion levels. Performance measures have been used with pavement quality and bridges for many years. Applying perfor- mance measures to operations is a recent activity. Opera- tions performance measures should be related to the goals and the vision of the region and the various parts of the transportation system. They should be easy to use and easy to communicate to stakeholders. They should be based on cost-effective data collection that is compatible with legacy data to the extent possible. Measures should allow for use of estimating and modeling future trends. They should also allow for modal and program trade- offs. There may be some good private-sector examples that can be used. Another topic of discussion is how to measure improvements or successes. Lower travel times and less congested time and space are two approaches that have been used. Given the variability in travel time from day to day, focusing on acceptable travel times may be a more realistic measure. Providing more options for trav- elers may be appropriate. We might also be able to do a better job of predicting travel times. A number of factors should be considered in selecting appropriate mobility measures. For example, being able to show the effects of potential solutions is important. Performance measures that respond to land use changes are needed. Performance measures should address both person and goods movement. Targets for measures should be based on realistic expectations. Consideration needs to be given to what we should measure. I believe we need to measure trip and segment travel times and link, section, and route speed. We need to be able to identify variations. Some people have sug- gested that we measure bad days or days that are worse than normal. The value of travel and delay allows us to place a monetary value on congestion. Matching expec- tations and schedules is another possible measure. Mea- sures are needed that focus on a single mode, but we also need measures addressing multimodal and intermodal trips. A number of measures are appropriate for considera- tion. Measures that appear to resonate with the public and policy makers include travel time, travel time varia- tion, and delay. The travel time index at the area level also appears to be understood by these groups. The buffer index that Jeff Lindley talked about on Sunday and the misery index that is targeted toward the number of bad days are two other possible measures. Being able to place a monetary value on delay and variation is also important. Research indicates that there is a strong relationship between congestion and reliability. As congestion increases, reliability decreases. In metropolitan areas that have implemented more aggressive operations strategies, unreliability does not appear to increase as sharply. Figure 2 illustrates the buffer index measure. The buffer index, which is the difference between the average travel time and the planned travel time, can be used in a number of ways. We can also match the results of cus- tomer satisfaction surveys with reliability performance measures. We know intuitively that travel conditions vary greatly on a daily basis. Graphing daily values shows that there is wide variation in the upper and lower bounds. 5 3MEASURING PERFORMANCE IN DIFFICULT-TO-MEASURE AREAS 0 10 20 30 40 50 60 1982 1986 1990 1994 1998 2002 1.20 1.25 1.30 1.35 1.40 1.45 1.50Biggest Problem is Congestion Houston Travel Time Index Value Tr av el T im e In de x Pe rc en t o f R es po nd en ts FIGURE 1 Congestion and public perception in Houston. (Source: Stephen Klineberg, Rice University.) 99395mvp39_69 12/13/05 12:29 PM Page 53

The influence that turning off the freeway entrance ramp meters in the Minneapolis–St. Paul, Minnesota, area had on delay at the entrance ramps has been exam- ined. Congestion levels increased and travel reliability declined significantly during the period the ramp meters were turned off. It is appropriate to consider different performance measures to address a range of needs. Real-time maps of urban traffic may be related to specific trips. Measures addressing average travel times, historic travel times, and variation in travel time are also needed. Performance measures for annual strategic decision making might include the different indices, elements covering all travel, average and actual travel time, and condition-specific reporting. A wide range of data sources may be available for use with performance measures. Point detectors and travel time collection are two of the most common approaches. It is important to identify the critical data collection ele- ments and to maintain them over time. Travel time stud- ies, data from private firms, and continuous Global Positioning System–based data collection may all be used. Key data collection resources should be identified and maintained over time. It is difficult to maintain 2,000 loop detectors or 2,000 radar microwave detectors. As illustrated in Figure 3, we are seeing a migration to direct monitoring over the long term. Other potential data sources will be needed, including legacy data sys- tems and customer satisfaction, travel behavior, and goods movement surveys. Surveys of travelers, truckers, and bus riders can be an important source of information. Surveys of high- occupancy vehicle (HOV) lane users over the years indi- cate that they report greater travel time savings than are measured by travel time runs or other data collection techniques. It could be that these commuters are indi- cating that they do not need to plan on a longer travel time, since they can depend on the trip-time reliability provided by the HOV lane. In using congestion performance measures it is important to have an understanding of other factors that may influence the performance of the system. These factors include weather, collisions, breakdowns, mal- functions, traffic or loading variation, new develop- ment, and special events. The limitations of data collected through various means should be recognized. Understanding the differ- ent data collection devices and maintenance needs is critical. Data quality issues, as well as other uses of data, should be understood. Partnering with other agencies can reduce the cost of data collection and can increase the availability of data. In conclusion, I believe the strategies being consid- ered in most urban areas focus on both providing more transportation system components and increasing choices for system users. Agencies are examining ways to improve the implementation process and to provide better products and services. Performance measures have a role to play in these strategies. Performance mea- sures can help provide linkages between choices, fund- ing, and actions. We also need to continue to experiment with both performance measures and strategies to improve the transportation system. Thank you. TOOLS FOR UNDERSTANDING CUSTOMERS IN MAKING INVESTMENT DECISIONS Randy Halvorson I will start my presentation with a story about linking customer satisfaction, performance measures, and inci- dent management, and communicating with elected officials. Three years ago, the Minnesota legislature was facing a significant budget shortfall. A legislator sug- gested that the Minnesota Department of Transporta- tion’s budget could be reduced by eliminating the Highway Helper Program. It was suggested that people could change their own tires. 5 4 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS 12 AM 6 AM 12 PM 6 PM 12 AM Planning Time Average Time Time of Day In de x Va lu e Buffer Index 1.50 1.40 1.30 1.20 1.10 1.00 FIGURE 2 Planning time and buffer index. Data Sources • HPMS • ITS Deployment Tracking • CMS Methods • HPMS/HERS Delay Models • Incident Models • IDAS Relationships Data Sources • Real-Time Travel and System Performance Data Methods • Quality Control Methods • Analytic Processing Near -Term (modeling/estimation) Long -Term (direct monitoring) )Migration Path FIGURE 3 Migration to direct monitoring for data collec- tion. (HPMS = Highway Performance Monitoring System; ITS = intelligent transportation system; CMS = congestion management system; HERS = Highway Economic Requirements System; IDAS = ITS Deployment Analysis System.) 99395mvp39_69 12/13/05 12:29 PM Page 54

The department was able to show the legislature both the importance of the program to congestion management and the importance travelers place on it through ongoing customer surveys. The result was that the legislature increased the budget for the Highway Helper Program that year. The department has conducted annual omnibus sur- veys of citizens since 1987. The department was also one of the first state transportation agencies to employ mar- ket research personnel. It adopted a statewide perfor- mance-based plan in 2003. Performance measures are being integrated into the budget process. We are commu- nicating with members of the legislature concerning the key elements of the plan and the performance measures. The Minnesota Department of Transportation col- lects and analyzes information on customers in a variety of ways. First we use market research techniques, such as surveys, focus groups, and other methods, to collect quantitative information about customers with scien- tific sampling techniques. Our public involvement processes also collect information on our customers. These forums, especially public meetings, do not scien- tifically sample or randomly select participants. Never- theless, they are an important source of input from citizens and stakeholders. The department is in the process of converting an 11- mile HOV lane on I-394 into a high-occupancy toll (HOT) lane. The department used market research when the HOV lanes were being developed in the mid- 1980s. We are also using market research, including focus groups, as part of the HOT lane project. Issues discussed by the focus groups included the hours of operation and the tolls for HOT lane users. Finally, user impact analysis measures the effects of our investments on roadway users in terms of travel time sav- ings, accident reductions, and vehicle operating cost sav- ings. These assessments are typically performed through techniques such as benefit–cost analysis or life-cycle cost analysis. My comments today focus on using market research information for investment decision making and perfor- mance measurement. Developing performance measures requires three steps. First, an outcome statement describing the result we hope to achieve for our cus- tomers is developed. An example of an outcome state- ment is reasonable, predictable travel times between regional trade centers. Second, a quantitative measure of that outcome or objective is developed. These mea- sures may be proxies for the outcome, on the basis of existing data. Several measures may be needed to cap- ture a single outcome. An example of a quantitative measure for an objective is a specific average travel speed in miles per hour. Third, a quantitative target or goal is set for each time period. An example of a goal is achievement of an average travel speed of 60 mph on 88 percent of miles of high-priority interregional corridors by 2005. In developing performance measures by using cus- tomer information, the importance of transportation services to our customers should be understood first; only then should we examine how satisfied they are with these services. Understanding the reasons for their satis- faction or dissatisfaction is also important. Performance measures and targets combine this information, with more aggressive targets set for the most important ser- vices. Products that are not important may not be tracked with performance measures or may be selected for a reduction of resources. Figure 4 illustrates how the Minnesota Department of Transportation analyzes customer information on the importance of and satisfaction with specific products and services. We focus resources on those products and services that customers rank as most important. If they are already satisfied, we try to maintain that service. If they are dissatisfied, we try to improve the service. Figure 5 provides an example from the department’s market segmentation study of how customers assess the importance of our roadway maintenance activities and their satisfaction with those services. As shown in the upper right corner, our customers rank snow and ice removal as very important, and they are satisfied with those services. In the upper left corner, customers are also very satisfied with our roadside planting, but these services are not highly valued by our customers. The department may be able to reduce resources in this area and still maintain customer satisfaction. The lower right corner identifies areas that customers value but where they find the department’s current level of service unsatisfactory. Road striping is important to customers, but they are not satisfied that the stripes are clearly visible. The department may place increased effort in this area. Finally, as noted in the lower left cor- 5 5MEASURING PERFORMANCE IN DIFFICULT-TO-MEASURE AREAS Action Planning Redirect Resources Keep It Up ! Limit Resources Improvement Area High HighLow Se rv ice S at isf ac tio n Service Importance FIGURE 4 Customer satisfaction with products and services. 99395mvp39_69 12/13/05 12:29 PM Page 55

ner, customers think our roadsides are too weedy, but they do not rate this as an important roadway feature. This information provides a clear picture of which products and services are of more importance to our customers. Those services directly provided on the road- way are consistently ranked most important by all of our customer segments. These services include salting, sanding, maintaining roads and bridges, building roads and bridges, removing debris, creating and maintaining speed and road signage, managing traffic, and providing features such as lights and striping. The two highest- ranked services are snow and ice removal and road smoothness. The Minnesota Department of Transportation uses customer satisfaction information in conjunction with importance rating data to set a limited number of perfor- mance measures directly. We make limited use of these direct satisfaction measures for a number of reasons. First, budget limitations do not allow us to satisfy all cus- tomer priorities. For example, we know our customers are dissatisfied with traffic congestion. The department does not have the funds to reduce this growing problem significantly, however. Second, many of the services we measure are not vis- ible to customers, so they would not have a sound basis for reporting satisfaction or dissatisfaction. For exam- ple, the structural condition of bridges is important to the safety and efficiency of the highway system. The department measures and reports on the number of structurally deficient bridges each year. However, we do not ask customers whether they are satisfied with our performance, since bridge structural problems are not typically visible to drivers. We do use customer satisfac- tion measures directly in the areas of roadway mainte- nance, including snow and ice removal, and reliability of department information. The Minnesota Department of Transportation con- ducted focus groups several years ago to obtain a better idea of customer satisfaction with different levels of snow and ice removal. The focus group participants were shown different photographs of roadways after a snowfall. Participants were asked whether they would be satisfied if the road looked like the photographs after different lengths of time. The focus groups were strati- fied to provide feedback by roadway type, from high- volume urban freeways to low-volume rural two-lane roads. Targets were established on the basis of the focus group results for the number of hours after a snowfall a roadway should be cleared for each roadway type. Fig- ure 6 shows the systemwide average. The target for all roadways is shown. The bars are the department’s actual performance for the past 5 years. The department also conducts an annual survey of customer satisfaction with roadway maintenance. Fig- ure 7 illustrates the overall satisfaction ranking from the surveys. While the department’s snow and ice removal times are good, the survey results indicate that satisfac- tion levels are declining. This measure covers several 5 6 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS Lo w H ig h Sa tis fa ct io n Ra tin gs Possible Overcommitment by the Department: Plants, grasses, and flowers by roadside look good Customer Irritants: Weeds on roadside eliminated Areas for Improvement: Road surfaces smooth and comfortable to drive on Road stripes, markings clearly visible Strengths of the Organization: Roads clear of ice and snow High Importance Rating Low • • • • • FIGURE 5 Customer satisfaction with roadway maintenance activities. 18 H ou rs 16 14 12 10 8 6 4 2 0 98 –9 9 99 –0 0 00 –0 1 01 –0 2 Winter Season 02 –0 3 03 –0 4 04 –0 5 05 –0 6 10.3 Hrs.9.0 9.0 10.5 7.3 5.0 7.8 4.1 Hrs.Ta rg et R an ge 03 –0 4 Significantly Under Expectation Under Expectation On Target Over Expectation FIGURE 6 Snow and ice removal: statewide annual average hours to achieve bare lanes after a snow or ice event. (Source: Office of Maintenance, Minnesota Department of Transportation.) 7.3 7.0 7.1 6.3 6.3 6.4 1994 1996 2000 2001 2002 2003 2004 2005 Sa tis fa ct io n Ra tin g 1010 1 9 8 7 6 5 4 3 2 1 Ex tre m el y G oo d Av er ag e Ex tre m el y Po or Department Target ≥ 7.0 FIGURE 7 Minnesotans’ satisfaction with maintenance of roadways: state highways—average on a 10-point scale. (Source: Omnibus Study 2003/2004, Minnesota Department of Transportation.) 99395mvp39_69 12/13/05 12:29 PM Page 56

dimensions, including both snow and ice removal and roadway smoothness. Ratings of road smoothness have declined as pavement conditions have deteriorated in the past 4 years. This trend has overshadowed our per- formance in snow and ice removal. Customer satisfaction with the department as a reli- able source of information is also measured. Customers consistently rank providing timely and accurate infor- mation as one of our most important services. Each year through our omnibus survey, we ask citizens if they are satisfied with us as a reliable source of information. As illustrated in Figure 8, we have been below our target of 60 percent of customers regarding us as a reliable source of information for the past 4 years. The department uses market research information on customer satisfaction indirectly to validate some of its performance measures. For example, the department sets a target for pavement smoothness rating (PSR) at the level that minimizes the life-cycle cost of maintain- ing the pavement. However, we use customer satisfac- tion information to validate that target. Figure 9 shows the department’s PSR data. We have set a target that 70 percent of roads will have a PSR rat- ing of 3.3. Keeping pavements at that rating minimizes the cost to maintain them over their life. Figure 10 shows customer satisfaction with road smoothness. When the department was meeting the pavement targets, the average satisfaction score was 6.7 to 6.6 on a 10-point scale. When pavement conditions started to decline in 2000, however, so did customer sat- isfaction with road smoothness. The department’s investment decisions have changed in reaction to customer information. On the basis of roadway maintenance information in 2000, the depart- ment reduced expenditures on roadside mowing and weed control. These resources were redirected to snow and ice removal, which our customers value more. The department has recently directed districts to increase pavement preservation investments to reverse declining PSRs and to help districts meet pavement targets by 2014. The Minnesota Department of Transportation has learned a number of things from these efforts. In the pri- vate sector, market research is either implicitly or directly related to product price. In contrast, we do not directly price highway services, with the exception of toll roads. As a result, it is important to structure questions care- fully to get at customer preferences and priorities. We often ask customers to rank a series of services or fea- tures. Another technique we use is to ask customers to allocate $100 among different transportation invest- ments or services. These approaches help eliminate the tendency of the average citizen to say “I want it all.” All of our major customer segments generally report similar priorities for services, with those tied directly to 5 7MEASURING PERFORMANCE IN DIFFICULT-TO-MEASURE AREAS 56% 49% 51% 52% 60% 0% 10% 20% 30% 40% 50% 60% 1999 2001 2002 2003 Target Pe rc en t R at in g 7, 8 , 9 , o r 1 0 FIGURE 8 Customer regard for Minnesota Department of Transportation as a reliable source of information. Good/Very Good Target = 70 percent or more Poor/Very Poor Target = 2 percent or less 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Pe rc en ta ge o f R at ed R oa dw ay M ile s PSR ≥ 3.1 76.5% 73.9% 73.8% 71.0% 76.1% 76.1% 80.5% 76.7% 72.0% 63.6% 61.3% 60.0% 60.7% 64.1% PSR £ 2.0 1.2% 1.6% 0.7% 2.4% 1.3% 1.3% 0.7% 1.5% 2.0% 2.6% 4.0% 7.9% 10.9% 11.1% 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 Best case scenario, based on the 2004 -2007 program FIGURE 9 Customer ride quality serviceability rating and pavement smoothness rating. (Source: Office of Materials, Minnesota Department of Transportation, February 17, 2004.) 99395mvp39_69 12/13/05 12:29 PM Page 57

the roadway ranking highest. The segment in Minnesota with significant differences is farmers. This group tends to rate roadside services higher than other groups because farm equipment frequently uses the shoulders. We have found that our customers are perceptive and accurate about changes in road conditions. For exam- ple, in 2002 our customers reported that congestion was improving. This response surprised us until we exam- ined the annual extent of congestion data. Congestion actually did decline that year. The state was in a reces- sion in 2002, with a loss of some 45,000 jobs. The 45,000 fewer commuters made a measurable, although short-term, impact on congestion. In conclusion, customer information is important to the Minnesota Department of Transportation in priori- tizing investments and allocating resources. The depart- ment uses customer satisfaction ratings for performance measures only in limited areas, however. The lack of resources and price signals makes it difficult to satisfy customer needs fully. Also, many important measures are not directly assessable by our customers, such as bridge conditions. Customer satisfaction measures alone do not provide enough information for investment decisions. We need to make sure that we are working on the products and services that are most important to customers—and focus our investments on improving satisfaction in those areas. In the 1980s, a decision was made to enhance the department’s market research capabilities. The depart- ment currently has two full-time staff, which is down from a high of five in the late 1990s. The department spends about $250,000 annually on market research projects. There is strong management support for market research within the department. While market research is used throughout the department, the Minneapolis–St. Paul metropolitan district probably uses it the most. PERFORMANCE MEASURES FOR FREIGHT SYSTEMS Barbara Ivanov It is a pleasure to provide a perspective on freight perfor- mance measures at this conference. Performance measures for freight can assist state departments of transportation in making program and investment choices. Performance measures are part of a data-driven, problem-solving approach at the Washington State Department of Trasnsportation. Freight performance measures focus on customers and are grounded in data and analysis. They attempt to find root causes of issues and concerns. Freight performance measures should be based on cus- tomer requirements. In 2004, the department’s Freight Office methodology included the following elements: • More than 150 one-on-one interviews with high- volume shippers and freight carriers, • Focus groups with key public and private partners, • Voice surveys of another 350 statewide customers, • Literature reviews of freight-dependent industries’ requirements, • Truck surveys providing origin–destination data on major statewide corridors, and • Additional existing regional research. Performance measures help show the benefits of var- ious programs and services. For example, the depart- ment is able to document the time and cost savings from the Commercial Vehicle Information Systems and Net- works program and the Weigh-in-Motion program. This information is directly related to motor carriers’ interest in on-time performance. To create state-level performance measures, we first examined how the freight system supports three major customer groupings in Washington. First, it supports international and national trade through Washington ports, airports, and other gateways. Second, it supports Washington State’s own producers and manufacturers. Finally, it supports distribution, wholesale, and retail functions. Washington producers and manufacturers are impor- tant customers of the freight system because they gener- ate hundreds of thousands of jobs and make a significant contribution to the gross state product. Food and agri- cultural products accounted for $5.6 billion in revenues in 2002. Transportation is especially important for Washington agriculture because the state produces about three times as much food—and for some commodities up to 20 times as much on a tonnage basis—as it con- sumes. The state is also separated by long distances from the majority of the nation’s consumers. Manufacturing gross business revenues were $88.3 billion in 2003. This figure accounted for 21.3 percent 5 8 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS 1994 1996 Year 2000 2001 2002 2003 8.4 8.2 8.0 7.8 7.6 7.4 7.2 7.0 7.0 6.8 6.6 6.4 6.2 6.0 M ea n R at in g P os iti ve N eu tra l 6.7 6.6 6.5 6.2 6.3 6.4 FIGURE 10 Customer satisfaction with road smooth- ness: rating on a scale of 1 to 10. (Source: Office of Investment Management, Market Research, Minnesota Department of Transportation, February 2004.) 99395mvp39_69 12/13/05 12:29 PM Page 58

of the total state gross business income. Manufacturing is thus a key freight-dependent industry. Washington’s regions have distinct economies based on industry and agriculture, and freight customers’ problems vary from region to region. Documenting cus- tomer requirements of the freight system is critical. To develop state freight performance measures, we began by analyzing current freight system performance in terms relating to customer requirements. Here are two examples of how the Washington State freight system serves regional economies. The first is southeast Washington, and the second is the central Puget Sound region. Southeast Washington is predominantly rural and is known as a wheat producer for the world. Washington ranks third among U.S. wheat-producing states, with 130 million bushels grown on 2.7 million acres. Approximately 85 percent of the wheat grown in the state is sold to inter- national markets. Wheat contributed $1.8 billion to the state’s economy in 2002, mostly in eastern Washington. By tonnage, food and food products outweigh other water- borne outbound commodities shipped from the state. About 53 percent of growers say that the price of freight service matters most to them, and 20 percent say adequate storage at the right location is the most impor- tant element. Only 50 percent of wheat growers are highly satisfied with the current performance of the state freight system. There are a number of significant transportation issues in this part of the state. Maintenance and preserva- tion of the Columbia River and Snake River channels and locks are critical. Second, improvements are needed on I- 90 at Snoqualmie Pass to avoid winter weather closures. Third, year-round freight corridors on the county–state road system are needed. Fourth, adequate grain storage is needed at the right locations. The Columbia River and Snake River transportation system is important to wheat growers because 92 percent of southeast Washington wheat is shipped to Columbia River ports. Approximately 51 percent is transported by truck/barge, 19 percent moves by bulk rail, and 30 percent travels by truck to storage facilities or nonbulk rail. Risks face each of these modal choices. Barge traffic may be influenced by environmental issues and federal policies. Truck weights are restricted on local roads during the spring thaw. Rail branch lines and short-line rail with low volumes often cannot recoup capital costs, and service may be curtailed or the line abandoned. The central Puget Sound region is urban and is the center of the state’s manufacturing sector. In 2003, 4,434 manufacturing firms were located in the region. Survey results indicate that 65 percent of south Puget Sound region manufacturers and 63 percent of eastside manufacturers are very satisfied with current freight sys- tem performance. However, only 50 percent of central Puget Sound trucking companies are very satisfied with the current performance of the freight system. The survey also provides important information on freight customers’ goals. Some 56 percent of eastside manufacturers rate on-time performance as most impor- tant, while 20 percent rate predictable trip times first, and 17 percent identified price as most important. In comparison, 52 percent of South Sound manufacturers rated on-time performance, 31 percent rated price, and 7 percent rated predictable trip times as most important. Customer deliveries are mostly made by truck. The state transportation system is also important for wholesale and retail distribution. Up to 80 percent of truck trips are operated on the local distribution system, which handles an enormous variety of goods. Products include food and groceries, fuel, pharmaceuticals and medical sup- plies, office supplies and documents, trash and garbage, and retail stock, from furniture and appliances to clothing and books. Survey results from the Washington State distribu- tion sector indicate that on-time performance is the main goal of about half of the respondents. About 35 percent identified price and 12 percent identified reli- able trip times as most important. On-time performance is critical to this sector, with 100 percent of high-volume distributors requiring delivery in less than a 3-hour win- dow. Approximately 25 percent defined it as less than a 1-hour delivery window, while 63 percent defined it as within 1 to 2 hours of the appointment time, and 12 percent identified a window of 2 to 3 hours. A freight congestion data pilot test was conducted in the central Puget Sound region by the Transportation Research Center at the University of Washington to obtain a better idea of the influence of congestion on truck travel. Trucks of volunteer firms were equipped with Global Positioning System devices, and their move- ments were tracked to travel times on different routes at different times of the day. Freight industry performance is achieved through effi- cient and effective logistics. State departments of trans- portation need a better understanding of logistics and should acquire a skill base in this area. The Council of Logistics Management defines logistics as “that part of the supply chain process that plans, implements, and con- trols the efficient, effective flow and storage of goods, ser- vices, and related information from the point of origin to the point of consumption in order to meet customers’ requirements.” Operational improvements require multi- ple data points at short time intervals to adjust a system, guide strategic investment, and analyze supply chains to determine and influence drivers of performance. Both short-term fixes and longer-term solutions are needed to address freight transportation system issues. Remedies include preventing customer dissatisfaction while a problem is being solved, addressing symptoms, and focusing on detection. Long-term solutions include ensuring customer satisfaction, reducing or eliminating root causes, and preventing problem recurrence. 5 9MEASURING PERFORMANCE IN DIFFICULT-TO-MEASURE AREAS 99395mvp39_69 12/13/05 12:29 PM Page 59

6 0 Closing Comments Lance A. Neumann, Cambridge Systematics, Inc. Thank you all for participating in this conference.The past 2 days have been productive. My closingcomments focus on a few summary observations from the sessions; the conference follow-up activities; and opportunities for continued involvement with per- formance measures research, technology transfer, and outreach. As I noted in my introductory comments on Sunday afternoon, the first observation is that the use of per- formance measures has become widespread in many transportation agencies. We have learned a great deal about the application of performance measures since the first conference in 2000. The speakers over the past 2 days have reinforced this point. Clearly, we have moved from performance measures as a concept to per- formance measures as a practical management and decision-making tool. We have seen a tremendous maturing in the use of performance measures to guide transportation investment decisions. The final session this morning added another dimen- sion. That dimension is including our customers in the process of developing performance measures and in iden- tifying where investments should be made. Obtaining the perspective of our customers in defining performance and priorities is important. The second observation is that while using perfor- mance measures to guide budget and program decisions is a critical step, we miss the mark if we do not also focus on program delivery. Once we decide where funds should be allocated, we also need to monitor program delivery and ensure that funds are spent appropriately. The practical reality is that agencies are never able to deliver 100 percent of the promised program. Perfor- mance measures can help determine the factors influ- encing the delivery of the program. So the second observation is the recognition that we need to extend the use of performance measures to include program delivery. At many agencies, it is the pressure to deliver projects on time and on budget that drives the use of performance measures. The third observation relates to the three critical stages of overall program management. These three stages are program development, program delivery, and program monitoring. The discipline to monitor the impacts of what is actually delivered and to compare the actual impacts with those estimated during the planning process is an important step. As noted during the Mon- day night session, before-and-after studies are routinely conducted in many other countries. Many speakers dis- cussed the notion that the program management cycle is not complete until we learn what performance was actu- ally generated. We need to realize that some projects will not generate the anticipated results for a number of reasons. We should acknowledge both our successes and our failures in estimating project impacts. Another observation relates to the notion of perfor- mance measures as a communication tool. The effective- ness of performance measures as a decision-making tool is dependent on our ability to communicate effectively with policy makers, the public, and staff at other agencies, as well as internally. The point was made that performance measures can be a powerful tool in helping integrate an agency vertically as well as in building partnerships with other agencies. 99395mvp39_69 12/13/05 12:29 PM Page 60

The final observation relates to the data, tools, and analysis techniques needed to support the use of perfor- mance measures. While data and tools are needed to main- tain and analyze performance measures, a strong message was presented that the lack of data should not be a barrier to starting to use performance measures. Clearly, better tools and data collection techniques are needed, but it is encouraging that it is possible to start using performance measures with available tools and data and expand to more sophisticated measures. The difficulty of developing mean- ingful performance measures related to environmental quality and social concerns was also noted. In closing, I would like to thank all of the sponsors— the Transportation Research Board (TRB), the Federal Highway Administration (FHWA), the Federal Transit Administration (FTA), and the American Association of State Highway and Transportation Officials (AASHTO). The conference planning committee did a great job, as did the TRB staff. All of you participating in the confer- ence deserve thanks. Your participation and willingness to engage in discussions and share your ideas determined the success of the conference. I look forward to working with you on future con- ferences and other activities of the TRB Performance Measurement Committee. The committee will use the results from the breakout groups to help develop research problem statements and identify opportunities for technology transfer and information sharing. The committee will also work with FHWA, FTA, AASHTO, and other groups to coordinate these efforts and other outreach activities. Thank you all again for participat- ing in this conference. Please continue your involvement in ongoing activities of the Performance Measurement Committee and other groups. 6 1CLOSING COMMENTS 99395mvp39_69 12/13/05 12:29 PM Page 61

6 2 Breakout Session Summaries George Gerstle, Colorado Department of Transportation Kristine Leiphart, U.S. Department of Transportation Joe Crossett, TransTech Management, Inc. Craig Secrest, Wilbur Smith Associates Gregory Selstead, Keith Cotton, and Barbara Ivanov, Washington State Department of Transportation Robert Romig and Brian Watts, Florida Department of Transportation Leonard Evans, Ohio Department of Transportation Mark C. Larson and Randy Halvorson, Minnesota Department of Transportation Anthony Pietropola, GeoDecisions Kent Barnes and William Cloud, Montana Department of Transportation Robert Winick, Motion Maps, LLC Jeff May, Denver Regional Council of Governments Louis H. Adams, New York State Department of Transportation Riju Lauanya, University of California at Irvine Timothy Lomax, Texas Transportation Institute Steven M. Pickrell, Cambridge Systematics, Inc. Michael Meyer, Georgia Institute of Technology Mark Hallenbeck, Washington State University MAKING IT WORK: OVERCOMING INTERNAL ORGANIZATION CHALLENGES George Gerstle, Moderator Kristine Leiphart, Recorder Topics Discussed Participants discussed a wide range of topics associated with building internal understanding and support for using performance measures and performance-based management. Participants provided examples from their agencies and organizations of techniques to introduce performance measures, change organizational culture to focus on performance, overcome staff concerns, and incorporate performance measures into day-to-day management. Participants discussed the need to obtain input from staff in the field and throughout an agency, to explain goals and performance measures, and to obtain their buy-in. Suggested topics were meeting with field teams and showing them how the measures would help solve problems and help with their responsibilities. Perfor- mance measures must be conveyed to an array of audi- ences, including new staff and those who have been with an agency a long time. Planners, administrators, and elected officials rep- resent the three levels of cultures. Working with each level to coordinate outcomes is important. One approach to building support is assigning a high-level champion, assigning the champion three or four staff members, and providing ongoing support to the champion. For example, the Minnesota Department of Transportation made numerous presentations tai- lored to the location, time, and need of the audience 99395mvp70_128 12/13/05 12:36 PM Page 62

to communicate the importance of meeting perfor- mance targets. Staff are frequently excited to have a strategic plan, but they are not necessarily committed to implementation. Participants discussed the need to provide adequate resources to support the development and use of perfor- mance measures. A critical mass is needed, with enough people to carry out the change and affect every level of management. There must be top leadership support, but support and understanding must trickle down to all staff levels, including maintenance and field workers. Participants noted that performance measures at some agencies were developed a number of years ago and may no longer be relevant today. Performance mea- sures should be kept current with changes in the trans- portation systems and modifications in the roles and responsibilities of transportation agencies. Leadership is key to these outcomes. Participants noted that some state departments of transportation and city agencies are beginning to link management pay and bonuses to strategic directives and agency performance. The British Columbia Min- istry of Transportation has linked its performance tar- get goals to individuals’ development plans. There can be a disconnect between where the funding is being allocated and agency objectives. It is important to link short- and long-term objectives. With limited resources, priorities are often set for the measures that are most important. Performance is not always linked to the amount of funding a program receives. If additional funding is available, it is important to show what can be delivered with the additional resources. Tension may come from unmet expectations. Agency staff may believe that they are being monitored for performance they cannot reach. The example in Portland shows that higher perfor- mance can be achieved even when less funding is avail- able. Although agencies track where funds are spent, the follow-up of the benefit received is sometimes lacking. The need to assess performance measures periodically was discussed. In most cases, agencies will not meet every performance goal. Tracking performance mea- sures is important, as are periodic discussions of their status. Feedback along the way is a key reason for using performance measures. Making the measures important and understandable for staff at all levels is also critical in obtaining and sup- porting performance measures. Celebrate successes and revisit shortfalls. Participants discussed the need to move from the the- oretical phase of performance planning to the empirical level. In the public sector, there may be a lack of com- mand and control to meet performance measures that may be more evident in the private sector. Keeping things simple and measurable and showing that man- agement has confidence in what agency staff are doing are important practices. Participants discussed how some police departments use performance measures effectively. Personnel are tar- geted toward the outcomes they want to accomplish. For example, the police put more resources into decreasing murder rates, and a decline in murder rates was accom- plished. Transferring best-case scenarios from other orga- nizations to transportation is a good approach. Leadership is important in meeting performance targets. The law enforcement society has leadership that drives the performance of the top two to three performance measures. Obtaining a consensus is not the primary focus for meeting performance in the law enforcement society. Research Needs • Identify and document methods to measure orga- nizational capacity to meet performance targets. • Identify and document methods agencies have used to communicate with staff concerning the use of perfor- mance measures. Provide examples of best practice case studies. • Examine models from nontransportation agencies and businesses for addressing internal barriers to the use of performance measures. Document best practice case studies. • Examine how changes in the governor’s office or the state legislature have influenced the use of perfor- mance measures for planning and project delivery by the state transportation agency and the specific mea- sures used by different agencies. Provide approaches state transportation agencies may consider in communi- cating performance measures and performance-based management to new elected officials. • Identify and document analytical tools to help implement agency goals and to encourage managers to enhance agency performance. • Examine approaches to making the monitoring and feedback loop more effective, and ensure that resources are being allocated to meet performance tar- gets. Document practice case studies and identify other possible approaches for agencies to test. • Conduct a study to assess how states, counties, and cities are currently aligning performance measures to coordinate project delivery. Identify methods to standardize performance measures so that what is being measured is common across state departments of transportation and local transportation agencies. • Examine methods and techniques to engage policy makers proactively in the project delivery process and in establishing performance measures. Document best practice case studies and identify other possible approaches. 6 3BREAKOUT SESSION SUMMARIES 99395mvp70_128 12/13/05 12:36 PM Page 63

TURNING EXTERNAL BARRIERS INTO OPPORTUNITIES Joe Crossett, Moderator Craig Secrest, Recorder Topics Discussed Participants in this breakout session discussed a range of external influences that affect the use of transporta- tion performance measures. The discussion focused on the following questions: • Who or what are potentially significant external influences and barriers to performance measurement? • In what ways may external forces create barriers to the development, selection, or implementation of measures? • What successful examples, lessons learned, or ideas do participants have about how to address external per- formance measurement barriers? Participants discussed additional research needs to address these issues. Participants discussed sources of external influences and barriers. It was generally agreed that items on the following list, developed in advance by the moderator and recorder, represented most of the external influ- ences or sources of barriers to various aspects of perfor- mance measurement: • Interest groups, • Other state agencies, • Local governments, • Metropolitan planning organizations (MPOs), • The public, • The state legislature, • The federal government, and • Media. There was general agreement that none of these groups stand out as a major external influence; rather, they are all influential. Participants discussed the types of external influences and barriers. They generally agreed that items on the following list, developed in advance by the moderator and recorder, represented the general types of external influences or barriers to various aspects of performance measurement: • Management of performance measures, • Balancing needs and demands of nontechnical and technical audiences, • Negative ramifications, • Timing issues, and • Other considerations. Some participants were interested in how the perfor- mance measurement activities of departments of trans- portation are interrelated with external stakeholders concerned with environmental issues. Environmental groups were perceived to have a powerful influence over the project development process, and failure to develop project solutions that respond appropriately to their con- cerns may create barriers to attaining mobility, accessi- bility, and other performance targets. Some participants suggested that measurement results for mobility and other performance measures can be skewed because of events and activities that transportation agencies cannot control. Participants identified the following potential strategies to address the impact of environmental delays on performance measurement: • Build in longer project development time frames based on assessment of delay risks. • Identify potential sources of project development delays as early as possible in the project development process—build relationships and establish clear expecta- tions to help avoid delays. Several participants indicated that states with unique demographic compositions often face particular chal- lenges in meeting performance targets that are influenced by driver behavior, such as alcohol-related accident and fatality rates, or unique concerns of such groups. Poten- tial strategies participants identified to address the impact of these issues on performance measurement were as follows: • Develop dual sets of measures to deal with differ- ent populations. • Sift through performance-related data to eliminate outliers and prevent them from skewing performance results. Participants discussed how state legislatures, state executive offices, and state and local agencies can impose performance measures on an agency that are redundant or that conflict with the agency’s existing or desired measures. Potential strategies participants iden- tified for handling negative implications of externally imposed measures were as follows: • Work with the outside parties to educate them about the agency’s internal performance measurement needs and to find mutually satisfactory measures. • Maintain a broader set of measures that meet both internal needs and externally imposed mandates. Participants discussed how stakeholders often tend to be “balkanized” in their desire for performance mea- 6 4 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS 99395mvp70_128 12/13/05 12:36 PM Page 64

sures—they only want performance measures that relate to their particular areas of interest and neither under- stand nor care about the need for trade-offs between competing performance targets. This can make it diffi- cult for state departments of transportation to arrive at measures that appeal to all external audiences. As a solution, participants emphasized the importance of closer coordination with external groups in establishing measures, but they recognized that this may be difficult to achieve in practice. Some participants noted that external audiences may not understand that performance measurement is an evolutionary process. They suggested that external audi- ences are sometimes inclined to be unduly critical of departments of transportation if the agency does not “get it right” the first time. Participants discussed how external interest groups can be inconsistent in advocat- ing for outcome versus output types of measures, regardless of their validity or usefulness. In addition, external interests may have a poor understanding of the data challenges associated with many types of measures. Participants discussed the problems that can occur if external interests feel threatened by the real or perceived influence of performance measures on important deci- sions, such as funding allocations. These problems can create challenges for state departments of transporta- tion using performance-based decision making. Participants noted that development of their own measures by state departments of transportation could create unrealistic or unattainable performance expecta- tions among external audiences and thus make it diffi- cult for state departments of transportation to manage. Participants noted that externally driven perfor- mance reporting needs often differ from internal report- ing needs with respect to detail, timing (e.g., frequency and reporting cycles), terminology, and presentation, and that there are no clear guidelines for how to merge internal and external timing needs. Research Needs • Develop strategies and best practices for engaging external audiences and managing their expectations, specifically in creating and implementing performance measures and performance targets. • Develop strategies and best practices for educating external audiences about the connection between policy challenges and performance measurement needs. • Investigate how external influences have actually improved the use of performance measures at state departments of transportation. • Complete case studies of how state departments of transportation have sought and obtained support for their measures from external groups, particularly in the areas of freight and environmental quality. THE ACCOUNTABILITY CHALLENGE: EFFECTIVE PERFORMANCE COMMUNICATION Gregory Selstead, Moderator Keith Cotton, Recorder Topics Discussed Participants in this breakout session discussed a variety of topics related to communicating information on per- formance measures effectively. Participants first identi- fied a list of ideas for discussion. Techniques for reporting performance measures, methods for determin- ing what the public wants to know, internal communi- cation techniques, and providing information on tax increases were discussed in more detail. Participants identified the following possible topics for discussion: • Turning data into external reporting and decision making—public relations or information perspective; • Benchmarking and performance for comparison with other departments of transportation; • Providing information to local elected officials; • Engaging with the public; • Converting data from advanced systems into use- ful information; • Using information to communicate and drive deci- sions; • Communicating more effectively and increasing understanding; • Consulting with and obtaining information from clients on what those clients think about communication; • Formalizing the process—it is not the measures, it is what you need to say; and • Identifying which performance measures are easy to communicate. Participants discussed how to report and communi- cate performance measurement. Examples of strategies and techniques were provided. Performance measures are internally oriented tools that allow you to tell your story externally. Agencies gain credibility and engage- ment from citizens, legislators, council members, and others when you connect their experiences to the story you are telling and the information you present. It is important to connect to the audience’s local interests, such as districts for legislators. Audience members should see themselves in the story you are presenting. It can be challenging to translate measures into an understandable story and to get people to focus on what is important. It is important to collaborate and bring in people to work with the technical staff. We often ask the people who monitor and analyze the data “to try to speak English” rather than involving professional com- 6 5BREAKOUT SESSION SUMMARIES 99395mvp70_128 12/13/05 12:36 PM Page 65

municators and others who are experts in communica- tion. Agencies should consider either providing support and training for the technical staff or having others pro- vide the messages for external groups. It is essential to build a conversation and communi- cation process around the measures. Using this approach gives you both the story and the means to manage what is happening. The person collecting and analyzing the data does not typically know all that is happening on the system. Facilitating communication among all groups helps you learn from each other and improve processes. One example of a way to build trust with the legisla- ture is to bring legislative staff into quarterly project review meetings so they can see the problems firsthand. Also, explaining to the commission or other policy groups the background behind the specific numbers is important. You can then drill down to more detailed information. The first-level information needs to be a simple message. Managing expectations is critical, especially with regard to funding. A circus analogy was described. At the circus, rather than telling the audience how good he is and doing the hard stuff right away, the trapeze artist builds the tension over time. The audience feels the ten- sion building, and as the routine grows more difficult, the audience does not know whether the person can pull it off. Then the artist does a perfect, death-defying stunt and the crowd goes wild. The analogy: we must do the drum roll and let people know that project delivery is hard and that we might fall the first time, but that we will get up and keep trying. Once in a while when we meet expectations the crowd goes wild. We must communicate that this work we are doing is difficult. Everyone in the process needs to be educated about the constraints of project delivery. Agencies will not meet project targets 100 percent of the time. The issue is bal- ancing high expectations with realistic steps to manage problems and do the best job. From the maintenance perspective, in Los Angeles the complaints from citizens often center on garbage and graffiti rather than congestion or pavement condition. Speculation is that citizens in the area expect congestion but want the road to look nice. The public complains to legislators about aesthetics, who then tell the department to clean up areas and focus resources on aesthetics, which is a low priority for the department. Yet for the agency, the emphasis needs to be on improving and maintaining the system. The public does not necessarily associate transporta- tion services with costs. Participants discussed the need to explain the costs clearly, what it takes to respond to complaints, and what is not being done when resources are diverted to aesthetics and other items. An article in the newspaper to give the department perspective on what is most important was suggested as one technique. The Seattle, Washington, example of tracking customer comments on incident response services, which is very well received, provides a huge political benefit. Participants discussed different audiences for perfor- mance measure information. It was suggested that the initial audience needs to be the legislators, since they control funding. Participants discussed how to measure the costs of what agencies do on a daily basis, especially work in response to special interests. It is important to report how much things cost per unit, or how much it costs to perform certain functions. A clear, concise graphic showing the costs and benefits of certain items can help tell the story. Participants discussed what the public wants to know and different techniques for gathering this information. Agencies have used customer surveys, including tele- phone surveys, and found that the public’s priorities did not always match department priorities. The most visited Federal Highway Administration web- site is National Traffic and Road Closure Information— the public wants traveler information. Listen to talk radio. Urban myths abound, especially on talk radio. Listening may be painful, but under- standing the points of view presented is important. Opportunities for intervention are there, especially since ratings depend on the conflict/controversy. A local agency used a town hall meeting with community lead- ers to hear issues and prioritize what they discussed. One state department of transportation has a radio show where people can call in and ask questions. Talking to your neighbors was suggested as a good approach. Listen to what they say, understand, and care about. You need to be able to talk to them with stories; otherwise you are not really talking to anyone. With talk radio, you get the bias of a segment of peo- ple and can learn what they think. Monitoring letters to the editor and opinions is important—the commu- nications group in an agency should be monitoring, and potentially responding to, opinions that can spread misinformation. Participants discussed whether performance mea- surements have been embraced by rank-and-file staff within agencies and how far down the measures cascade into organizations. It is important to link different mea- surement processes and initiatives. Measure what you need to manage, but make the link follow through to the administration. There is often employee resistance to using perfor- mance measures as a stick—for example, transferring resources from good to bad performers. This issue leads to lack of buy-in and resistance. Reward good perfor- mance and communicate the background to document the external factors and forces influencing performance. 6 6 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS 99395mvp70_128 12/13/05 12:36 PM Page 66

As you drill down into statewide numbers, agency staff can make their own performance assessments and appraisals. Good information used fairly will liberate information you would not have obtained any other way at the top. Performance measures may set false expectations. You may achieve on-time and on-budget targets, but are they the right projects? Look at trends and system out- comes rather than numbers and percentages. You can deliver projects in a bad plan. You need to work with others in the organization to develop good plans and projects. Using measures to manage is essential. Providing information to the right people in the right format inter- nally is also important, as is understanding the data that are being used. If information is not getting to the right people, you do not know how you are doing or what to change. The importance of iterative and adaptive measure- ments was discussed. This approach can help ensure that you do not get locked into an unintended effect. One example is measuring project advertisement dates. A project may be rushed just to meet a date and not done well. Measures should not be easy to manipulate. Participants discussed different approaches agencies have used to communicate with the public on tax increases. Providing accurate information on why the increased funding is needed and how it will be spent was stressed. Research Needs • Identify and document techniques to use perfor- mance measures as carrots and sticks to influence, encour- age, and reward behavior. Explore approaches to provide flexibility to change the process and use measurement iterations to arrive at the proper outcomes. • Explore approaches for communicating problems encountered with meeting performance measures. In the public sector, we cannot gamble with taxpayers’ money. We have to be successful. Identify ways to provide a more realistic assessment of the difficulties associated with project delivery. • Identify techniques to provide clear information on performance measures to various audiences. There is a delicate balance in using performance measures as a communication tool. You are either putting out propa- ganda—putting a spin on “objective” performance mea- sures—or you are trying to communicate the difficulties and risks of delivery. At the same time, to manage effec- tively you need your measures to be accurate. Explore how much explanation of the risks is appropriate. • Explore marketing and communication techniques for use by transportation agencies. Marketing is a central piece. Listening, especially with the right mind-set, is cru- cial to marketing. If you just say “we are meeting our goals” and are inflexible, you are not connecting with the citizens. If the public does not believe that we are meeting their goals, we are not meeting the goals we should be meeting. Be flexible listeners with the right mind-set, and do not use performance measures as an excuse. HOW TO SET PROGRAM TARGETS AND MAKE TRADE-OFF DECISIONS Robert Romig, Moderator Brian Watts, Recorder Topics Discussed Participants discussed the issues typically associated with setting performance measure targets and making trade-off decisions. A major focus of discussion was per- formance measures addressing preservation versus adding capacity. Other topics included regional versus statewide priorities, transportation and economic impacts, trucking and freight, and funding issues. Participants discussed different approaches to estab- lishing performance measures and setting targets for preservation versus capacity projects. It was noted that Florida has standards and targets established by legisla- tion, while other states have struggled with setting poli- cies in this area. In Washington, a system to prioritize and implement projects was developed with input from private and public interests. The experience from the international scan indicates that setting targets for preservation and capacity is being addressed in other countries. The basic goals are set at the national and state levels, and everything flows from the high-level outcomes. This approach helps encourage understanding among local and statewide groups. Participants discussed the following approaches to addressing regional versus statewide transportation priorities: • Florida is trying to move state projects to 75 per- cent statewide system and local projects to 25 percent. • California is focusing on comprehensive corridor planning and on working with partners to obtain fund- ing and planning support. There is a need for a clear assignment of roles and responsibilities among state agencies and regional and local agencies in this process. • Virginia is examining long-term planning in corri- dors. This approach may save funding in the long run. Participants discussed the possible economic impacts of the transportation system, including approaches to defin- 6 7BREAKOUT SESSION SUMMARIES 99395mvp70_128 12/13/05 12:36 PM Page 67

ing return on investment and measuring the economic influence of different transportation improvements. The importance of measuring how improvements to corridors influence economic conditions and the local economy was noted. Decision making should be based on facts, including consideration of the gross domestic prod- uct and other economic factors. The need to be informed and to examine the entire picture, rather than being limited to database decision making, was suggested. California and Washington have investment boards to help examine these economic impacts and manage resources and direct projects. Leveraging private investments can affect public investment. California focuses on the economic impact of trans- portation projects, including identifying costs and bene- fits of projects and the number of jobs created by projects. It is difficult to determine the contribution of a transportation improvement to job creation. There is a need for better tools to estimate these types of factors. The model of Regional Economic Models, Inc. (REMI), may be of use in this effort. High-occupancy vehicle data have supported the use of these types of facilities. These data add to the argument for better management of capacity. When this type of information is available, it is helpful in selling initiatives to policy makers. Multimodal issues generally seem to be regional concerns. For example, consideration of mass transit versus more highway capacity is typically a corridor issue. Regional governments and state agencies may have different approaches to the corridor solution. Participants discussed possible trucking and freight performance measures; sources of data on freight move- ments; and coordination among public agencies, truck- ing companies, and businesses. Several participants noted that it is often difficult to obtain data from the trucking industry. There is no standard data reporting from trucking industry organizations. The lack of data makes analysis, measurement, and decision making associated with freight challenging. An example was provided of high truck volumes on a specific freeway. The state department of transportation met with shipping companies, merchants associations, and other groups. One of the recommendations was to provide incentives to companies to ship in off-peak hours. This approach received positive feedback, and progress has been made in moving some truck traffic to off-peak periods. The example of the raising of tolls on the Ohio Turn- pike was described. The turnpike is separate from the department of transportation. When the turnpike raised tolls on larger trucks, many truckers changed routes to use state roadways. Truck volumes increased on the other highways, which caused safety and capac- ity concerns. Participants noted that large volumes of commercial vehicles may cause other operational issues. Incident management and clearing of accidents may be more dif- ficult. More information is needed on the causal factors associated with truck accidents. Usage charges and fee structures for trucks for capacity management were dis- cussed. A little improvement at the right time of day can have a large impact overall. These examples focus on reliability, which is a key measure. Tolling can be an answer in managing capacity and maintaining the flow to improve reliability. This approach translates into the ability to add capacity on other roads. Participants discussed the following topics related to funding issues: • In California gasoline and sales tax revenues have not kept up with highway funding needs. It appears that existing legislation provides regional governments with the responsibility to address these funding problems. • Tax revenues and the political nature of funding have a big impact on transit and public transportation. For example, tax initiatives have had a negative influ- ence on transit in Washington State. Transit fares typi- cally pay for 40 to 50 percent of transit operating costs at best. In New York State, tolls subsidize some transit operating costs. User charges may be more politically acceptable than general taxes. Setting targets by corridors may make sense in some cases. This approach would allow improvements to be prioritized in local areas. Most corridors include more than just highways. Rail, transit, and other modes are frequently funded within major travel corridors. Research Needs • Develop best practices and case studies of perfor- mance measures addressing operations versus adding capacity. The case studies would examine the efficiency of existing facilities versus the expansion of facilities. • Explore performance measures that address trans- portation and land use. • Examine the trade-offs associated with mobility versus security and performance measures addressing the two. This study would evaluate the impact of new security procedures on mobility. Long-term issues would be explored along with approaches to balance mobility and security. The experience in New York could be used as one case study example. The study would also examine trade-offs in investments. • Examine the potential to extend the REMI model or develop other economic models to include consider- ation of the economic impact of investments in other modes. Explore how these models can be used in set- 6 8 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS 99395mvp70_128 12/13/05 12:36 PM Page 68

ting performance targets and evaluating project impacts. • Examine the use of high-occupancy vehicle facili- ties for freight movement. • Examine the influence of transportation on eco- nomic development in the state as a whole, as well as regionally. Identify performance measures to address economic development. • Examine bringing a supply chain analysis to cor- ridors. • Examine how performance-based management can address trade-offs in investment in small communi- ties needing transportation improvement to encourage economic development versus congested areas where demand exceeds capacity and the lack of transportation improvements is impeding economic expansion. • Examine underserved communities from the stand- point of economic growth versus economic development. Document approaches for addressing these issues. • Develop tools and techniques for land use and site decisions. These tools would focus on economic consid- erations in the plant and industry location decision- making process. WORKING WITH LEGACY DATABASES AND MANAGEMENT SYSTEMS Leonard Evans, Moderator Mark C. Larson, Recorder Topics Discussed Participants in this breakout group discussed the issues and opportunities associated with legacy databases and their use with transportation performance measures. The limitations of legacy databases were examined, along with techniques to enhance the use of these systems. Areas for additional research were identified. Participants noted that there are some problems asso- ciated with updating project management systems. The use of old and new parallel systems at the same time can be a problem. Frequently the old system is not shut down. Washington State established a separate project management system database because the old system was inadequate. Both systems are being used because the legislature did not provide funding for the new system and required that the old system be used for reporting milestones. It was noted that several states are exploring and using data warehouse technology to improve data qual- ity and enhance coordination at reasonable cost. Clear accountability and business rules for project management systems are needed. Ohio provides a link to accounting data. The system is web based and is inte- grated with pavement data, which allows queries of pavement condition values. Legislation in Ohio includes an individual career manager performance accountabil- ity system. A manager’s performance is linked to pro- motions and potential demotions. District performance values are included in manager performance plans. Action plans are required if there are deficiencies. It was noted that quality tends to increase if data are used on a regular basis. In Virginia, demand for regular project status reports to the commission has driven an improvement in data. Data from a number of sources must often be integrated. The age and inadequate capacity of legacy databases in some states cause problems and concerns. Legacy databases in many states are 30 to 50 years old. Prob- lems may be encountered with temporal data. The via- bility of project management systems, which are often used in reporting project milestones to legislatures, managers, and other stakeholders, is important. Participants discussed funding needs and noted that obtaining funding from state legislatures for database systems can be an issue. California has legislative restric- tions on information technology projects exceeding $500,000. The Washington State legislature terminated funding for the new project management system. There are also limits on the use of federal funds. Participants noted that there is often a lack of cost information early in the project development process. The lack of good cost estimates may cause problems in later phases of a project. Research Needs • Participants noted that developing shared geographic information system (GIS) and management system stan- dards for state departments of transportation may be ben- eficial. This approach would encourage vendors to offer standard applications that are lower in cost rather than customized systems for individual states. NCHRP Project 20-64, XML Schemas for Exchange of Transportation Data, could be used to provide guidance in this effort. • Identify and document the formats for manage- ment reports used by various agencies and organiza- tions. Identify formats and approaches that appear to best communicate key information to policy makers and other external groups. • Identify and document techniques to develop part- nerships to increase funding for system enhancements. The partnerships could focus on increasing funding at all levels of government, as well as exploring other avail- able resources. • Some participants noted that setting standards for GIS at all levels of government would be beneficial. 6 9BREAKOUT SESSION SUMMARIES 99395mvp70_128 12/13/05 12:36 PM Page 69

Identify and agree on common benchmarking and other standards to provide for comparisons among states. • Identify and document approaches to address state regulations relating to the use of information technol- ogy and database systems that create barriers for state departments of transportation in sharing systems with local governments, contractors, and other key cus- tomers. Provide best practice examples. NEW TECHNOLOGIES FOR DATA COLLECTION Anthony Pietropola, Moderator Kent Barnes, Recorder Topics Discussed Participants in this session discussed new technologies to assist with data collection needs associated with per- formance measures. Participants discussed the issues and opportunities associated with various technologies. Participants discussed a number of new technologies that are being used to collect data about different aspects of the transportation system. These technologies include the Global Positioning System (GPS), weigh-in- motion, virtual weigh-in-motion, and electronic toll col- lection (ETC). These technologies can provide data on trucks, commercial vehicles, and buses. ETC and other onboard systems may also be available for collecting data on passenger vehicles. Issues associated with pri- vacy and vehicle and infrastructure integration may need to be addressed. For example, it is difficult to esti- mate bus travel times by using GPS because of all the stops buses make to pick up and drop off passengers. One possible approach for addressing this concern is to use changes in the overall bus travel times. Participants noted that new travel forecasting models may also be of use in assessing performance measures. The TRANSIMS model and other advanced models may eventually be used, but many of them are still in the development stage and thus are not as user-friendly as they ultimately will be. Participants discussed possible technical barriers that may need to be addressed with new technologies. For example, some of the older data collection techniques cannot meet the quality standards needed with the new analysis. Equipment frequently needs upgrading for expanded use. For example, closed-circuit cameras can now be used for many more purposes, but the older equipment may not be adequate for the expanded appli- cations. Additional maintenance or better equipment may be needed. There is a desire to avoid two layers of data collection, but they may be needed at least in the short term. Participants also noted that institutional issues may need to be addressed with the introduction and use of new data collection and analysis technologies. There may be a need to inform and educate potential users about the benefits of new technologies and to justify additional costs to policy makers and funding sources. Asking the right questions is important to all groups, as is having the information needed to answer the questions. In addition, participants noted that problems with data collector reliability remain a concern in most met- ropolitan areas. Research Needs • Research technologies for obtaining vehicle occu- pancy information, including infrared and other advanced technologies. Explore vehicle infrastructure integration needs and the expanded use of transponders for all types of vehicles. • Identify best practices to present information to policy makers and the public on the use of new tech- nologies. Examine approaches to address privacy issues, funding concerns, and other possible issues. STANDARDS FOR DATA AND PERFORMANCE MEASURES William Cloud, Moderator Robert Winick, Recorder Topics Discussed Participants in this breakout session discussed issues and opportunities associated with standards for data and performance measures. They discussed the experi- ences and challenges with different approaches. They also identified areas for future research. Participants noted that some state agencies have for- mal enterprisewide standards for data and performance measures. Islands of data and databases that are not integrated were noted as problems at some agencies. Participants discussed the importance of data quality standards. Sharing information among states about establishing and maintaining data quality standards was suggested. Participants discussed benchmarking. The Interna- tional Roughness Index was discussed as one measure that many states use. Sharing information on what other states are doing with regard to data standards and data collection issues and procedures would be of benefit. Participants discussed possible secondary uses of data. Issues with secondary uses were noted. For exam- 7 0 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS 99395mvp70_128 12/13/05 12:36 PM Page 70

ple, secondary users must be considered. A change in culture within a department related to data ownership may be needed—a change in thinking from “my” data to “our” data. Standards development and the use of different stan- dards were discussed. The internal organizational struc- ture for dealing with standards was discussed. Some agencies use committees, while some have one individ- ual or group in charge. The importance of addressing data security and backup was noted. Top management support is needed in the develop- ment and use of standards. People who use the standard on a day-to-day basis should be involved in standards development. Participants discussed whether standards that deal with dynamic changes associated with some types of data are needed. For example, while road geometry data are mostly static, such data can change slowly over time as improve- ments are made. Other data can be dynamic, such as progress in snowplowing and removal or crash-related data. Work related to intelligent transportation system (ITS) data archiving is dealing with aspects of data for roadway oper- ations and use that change frequently over time. Participants discussed the need for data audits. Data audits need to check for omissions as well as having too much data. Participants discussed the need to start with realistic expectations for data standards. This approach means starting small and building on successes. Identify a few critical standards as a starting point. Participants discussed providing access to data and possible roles related to being a data broker that state departments of transportation can play. This approach might help minimize duplication of effort and reduce the costs of data collection. The fact that data and technol- ogy are not information was also discussed. Information is needed for performance measurement. Research Needs • Examine staff capability and resource needs asso- ciated with developing and maintaining performance measure database systems. Identify potential issues with staff training and staff retention. Identify and document approaches to address these issues. • Examine the current use of benchmarking by state transportation agencies. Identify and document what information is being collected, how it is being collected, and how it is being reported. Identify possible benefits from benchmarking across all state transportation agen- cies. Examine methods to ensure that agencies are col- lecting and analyzing the same data in the same way so that benchmarking can occur and accurate comparisons can be made. • Conduct surveys of database needs and links between databases and operating systems. PERFORMANCE FORECASTING: FINANCIAL ISSUES, SCALE, AND IMPLEMENTATION TRACKING Jeff May, Moderator George Gerstle, Recorder Topics Discussed Participants discussed a variety of financial, scale, and implementation tracking issues associated with perfor- mance measures. Issues associated with the scale of per- formance measures and concerns related to financial forecasting were discussed. Participants discussed financial forecasting and the influence of poor estimating on project cost and effec- tiveness. How much to spend on performance-related data collection was discussed. It was noted that the Motor Vehicle Safety Administration spends 5 to 10 per- cent on management systems data. Data collection approaches used by different agencies were discussed. The data collection of the San Diego Association of Gov- ernments (SANDAG) is oriented toward highways. Data collection for lower-classification roadways is much less extensive. The Sacramento Area Council of Govern- ments collects data as grants come in for system perfor- mance monitoring. MPOs typically conduct less direct data collection than state departments of transportation. Participants discussed the value of data and how it will influence decision making. Decision makers should be educated about the importance of data, the costs associated with data collection, and the uses of data. Participants discussed risk assessment and how it relates to project costs. Some agencies conduct peer reviews of project costing before it is included in the transportation improvement plan. Participants also dis- cussed techniques for communicating risks to the pub- lic. Use of design–build can shift risk to contractors. Financial forecasting is different from planning fore- casts. Elements to consider in financial forecasting include noncompete clauses and effects on parallel roads. Participants discussed different scales for perfor- mance measures, including statewide, metropolitan areas, and corridors. Possible integration of measures for these different scales was discussed, as was the need for comparisons among different levels. Coordination among agencies at the different levels is needed, as is consistency in the approaches used. California has started trying to coordinate with MPOs in the state. 7 1BREAKOUT SESSION SUMMARIES 99395mvp70_128 12/13/05 12:36 PM Page 71

Participants discussed approaches for evaluating the effectiveness of capacity projects given latent demand. Techniques for showing long-term benefits were dis- cussed, including measuring economic benefits and reduction in vehicle- and person-hours traveled. Research Needs • Complete a synthesis of MPO data needs for per- formance monitoring and management. • Complete a synthesis of data collection needs for performance measures associated with quality of life, environmental quality, and other related measures. • Conduct and document case studies and a synthe- sis of performance measures and data collection needs related to economic development, return on investment, equity, and transportation and land use. • Explore techniques for relating project purpose and need to performance objectives, especially the more intangible performance measures. • Conduct a synthesis of techniques for incorporat- ing risk assessment and uncertainty into project costing. • Evaluate private-sector performance measures on toll facilities and how they relate to public measures. • Conduct and document case studies of parallel performance data coordination. • Conduct a study of the technology of evaluating effectiveness of capacity projects given latent demand and methods to document long-term benefits. • Analyze factors that may influence cost increases after project advertisement and project scope creep. LINEAR REFERENCING SYSTEMS—OR WHERE IN THE WORLD IS THAT BRIDGE? Louis H. Adams, Moderator Riju Lauanya, Recorder Topics Discussed Participants in this breakout group discussed a variety of topics associated with linear referencing systems. The fol- lowing seven questions provided the basis for discussion. Participants also identified areas for further research. • What technology changes are in your immediate future, and what benefits do you foresee as a result of the forthcoming implementation? • What advances in linear referencing and dynamic segmentation state of the practice are needed to support or expand capabilities for measuring, reporting, and mapping of transportation system performance? • What are your suggestions for technology transfer or research? • What has worked or been implemented success- fully, and what are some key factors that resulted in that success? • What has not worked or has experienced major implementation problems, and what insights should other practitioners consider as a result of your experience? • What can agency managers do to address any lin- ear referencing and dynamic segmentation barriers that constrain expanded use of performance measures? • What is the relationship of linear referencing to performance-based decision support? Participants discussed national standards for linear referencing systems. Transportation is one of seven lay- ers on the nationwide map, but it is not finalized yet. Participants discussed some of the issues associated with using linear referencing systems, including those associated with bridges. For example, capacity is an issue on a bridge. Under a bridge, clearance is the issue. Clearance may vary along the distance of a bridge, so a bridge can be treated as a single segment (value) for capacity purposes, but that same bridge may need to be divided into several segments to define its height. Similar issues may be encountered with freeway ramps. Participants discussed various hardware and soft- ware issues. It was suggested that the ITS community is on a track different from the infrastructure manage- ment system. The need for border states to coordinate with Canada was noted as creating more problems. Joint naming was noted as a problem. GPS cannot see the lower road in cases where one road runs over the other. Two points having the same x- and y-coordi- nates but being different physical entities may also be a problem. Adding multimodal facilities becomes more complex. Participants discussed cost issues and the need to conduct financial analyses. Participants noted that e-government is on the presi- dent’s agenda. Movement in this area may help trans- portation agencies. Participants discussed the possibility of remote sens- ing for updates of GIS instead of physically driving on the road. Participants discussed links to inventory systems. Research Needs • Conduct a synthesis of the current status of linear referencing systems. • Explore the use of remote sensing and other related approaches for updating linear referencing systems. 7 2 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS 99395mvp70_128 12/13/05 12:36 PM Page 72

CONGESTION AND RELIABILITY Timothy Lomax, Moderator Steven M. Pickrell, Recorder Topics Discussed Participants in this group discussed a wide range of issues associated with developing and using perfor- mance measures related to congestion and reliability. Topics included current performance measures, possible barriers, expansion of the use of congestion and relia- bility measures, and other possible measures. Partici- pants also identified areas for further research. Participants discussed positive experiences with con- gestion measures. The following elements were identified: • Provide the public better access to better trip data (e.g., travel speeds), such as the San Francisco Metro- politan Transportation Commission website. • Travel information systems (511 systems) are working, though getting the word out to the general public is a challenge. • Make the connection between safety-related improvements and congestion improvements to help sell congestion improvements. Comparing safety analysis with congestion analysis might provide agreement between the two problems. Investing in a solution for one might help the other. “Turf wars” between safety and congestion factions in agencies and among external stakeholders may be reduced if the two needs are not viewed as competing. Removing a bottleneck that causes accidents as well as delay was provided as one example. • Provide side-by-side comparisons of travel times for automobiles versus transit on the roadside. The example came from Australia, where transit and auto- mobile travel times in a particular corridor are shown on a message sign. • Communicate the overall improvement (e.g., total capacity in person trips, average delay, average travel time) to system capacity when a new facility is opened, rather than just communicating the improvement for the new facility. This was suggested as a method to counteract the problem of latent demand quickly absorbing all new capacity in an improved corridor, which could make it appear as though the investment was for naught. • The use of real-time data has helped one agency understand and explain recurring and nonrecurring congestion; it has also allowed the agency to better dis- tinguish between the two when such data are used for diagnostic purposes. • The California Department of Transportation’s (Caltrans’s) Los Angeles district is compiling a log of traffic data specifically linked to special events and inci- dents to improve its understanding of the impact of such events on traffic conditions. • Implementing variable speed limits and aggressive, automated speed enforcement (Australia) has allowed an agency to improve management of the effective capacity of a congested corridor. By gradually reducing drivers’ speeds as congestion mounts and using video enforcement, the agency is able to delay or avoid the onset of a breakdown in flow rates. This approach reduces collisions at the back of the queue and allows freeway lanes to flow more smoothly and accommodate relatively high speeds for longer periods. Participants discussed possible barriers, concerns, and questions with regard to congestion and reliability. How do agencies identify their role in congestion prob- lems and solutions? What can agencies actually do about the problem? What can we demonstrate we are doing to make things better? Several responses were provided by the group: • Examine data from operations centers to develop congestion patterns and data and identify potential solutions. • Ensure proper traffic signal maintenance and operating status. • Municipal agencies should be a part of regional multiagency groups that examine problems and solu- tions. The traveling public and freight movers do not always know which agency operates what components of the facility or system—it’s just “transportation” to them, and they expect seamless, coordinated service delivery. • Speed and travel time data are important, espe- cially at decision points between alternate routes. The 511 system is beginning to help where it is deployed. Agencies can help by making better data available to the 511 system and to the general media, which distribute a lot of information via television, radio, and so forth. • The congestion problem begins with a lack of coordination between land use and transportation plans. It is difficult to address transportation in a com- prehensive manner without an effective land use plan. Agencies can strive for a better land use–transportation linkage in all of their planning and development-related activities. – The poor link between land use and transporta- tion undercuts congestion and reliability efforts. As a result, latent demand soaks up new capacity. Agencies have a limited ability to affect land use decisions; there should be a better link between land use goals and actions and the causes and measures of congestion. – What is the ideal mode split for a given corridor or region? The opinion was expressed that different 7 3BREAKOUT SESSION SUMMARIES 99395mvp70_128 12/13/05 12:36 PM Page 73

regions all have an optimal mode split, that dispro- portionately spending too much money to push mode split far beyond the optimal range is inefficient, and that benefits do not justify costs. There is a need for research or peer exchange on different optimal mode splits for different areas, system types, and related factors. – Different customer market segments have dif- ferent definitions of reliability as well as different expectations in terms of reliability. There is a need to clarify the terms and to understand the different expectations. – Selling the benefits of operational improve- ments is still difficult. An example was presented by a highway district of a great package of operational improvements that it would like to fund, but the dis- trict has had no success in selling the package. It can- not make a compelling argument for operational improvements. In response, participants indicated that measures and data need to be refined to commu- nicate the importance and value of improved system management and operation. – “Telling the story” is the challenge. Do not focus so much on the data or the measures, but rather on how you use them to make the point. – Why has the rapid spread of telecommunica- tions to home and office not led to more telecom- muting, shift of trips to different times of day, and other related changes? Peak periods still experience major congestion despite the increased ability to use computers, the Internet, and other technologies to conduct work and perform jobs. Participants discussed methods to expand the use of congestion and reliability measures in transportation decision making. • More funds are needed for traffic operations cen- ters to post more trip information, including more accu- rate, up-to-date information. • Define and clearly communicate the additional system (highway and transit) operational demands and burdens created by new development. Demonstrate more fully the system impact from development and the impact that is not offset by any mitigation fee. • Models should be improved to incorporate different values of time—for example, time spent in congested ver- sus uncongested travel conditions (similar to in-vehicle and out-of-vehicle time values used in transit modeling). • Monetize the benefits of agency actions (such as incident management) that improve operations by pre- serving capacity. This will better demonstrate the value of such programs and of allocating agency resources to activities such as freeway patrol. (Refer to NCHRP Research Results Digest 289 for more information.) • Similarly, monetize and demonstrate the cost of incidents to help sell increased funding for improved system management practices. Communicate the prob- lem and the magnitude of the costs and benefits to other agencies that play a role in system management, such as police and fire. For example, communicate the high value of incident management in reducing secondary accidents. This is something that police and other emer- gency response agencies can value. • Identify data on smaller improvements that have been implemented with good results. These have been successful in some areas in reducing congestion and showing the public a good return on spending (high ben- efit-to-cost ratios). • State departments of transportation need to think and operate more multimodally, from a system perspective. Participants discussed reliability and the relationship to congestion. • SANDAG is using the concept of increased relia- bility to promote its managed lanes program as well as light rail service. Users enjoy greater reliability than mixed-flow lane travelers. • Hedonic modeling can demonstrate the value placed on reliability, as expressed in terms of real estate valuation. Would increased reliability have an effect on land values and location choices? • Reliability seems to be a key metric or concept for the future. Congestion reduction or average travel time may not be the primary measures of success because of the great difficulty in actually reducing congestion with- out more draconian measures such as dynamic conges- tion pricing. For example, airlines compete on reliability of on-time departure and arrival rather than on trip duration. Yet the question was raised, does the promise of a more reliable system with slower travel times “work” in a regional improvement plan? Do leaders and the public agree that this should be the goal? • Participants discussed measures related to conges- tion and reliability that need further examination. • Ensure that whenever possible, performance mea- sures are comparable across regions, states, and time frames. Comparisons are made between data items that appear similar. If the data are not comparable, that should be clearly noted in footnotes and explained in text. • Develop lost system capacity (throughput) graphs, like those presented in the opening session. • Does relating future congestion levels to “what would have been” or “what was projected” provide per- spective on the effect of growth? Congestion may be worse, but was it predicted? Were projections and deci- sions based on accurate data? • Compare user costs of congestion problems with the benefits of living in large metropolitan areas. Com- 7 4 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS 99395mvp70_128 12/13/05 12:36 PM Page 74

municate the message that you gain something in return for enduring higher levels of congestion by choosing to live in an urban area or congested location. • Communicate measures that demonstrate the impact of congestion and unreliability on business—in particular, those that provide or rely on local collection and distribution of goods and packages. Comments in an earlier session indicated that up to 80 percent of truck trips (not miles) in an urban area relate to rela- tively local collection/distribution. Congestion and unreliability have a big impact on the productivity of these services and thus the businesses that rely on them. • Express the value of time and cost of congestion in terms that are relative to other regions (benchmarking) rather than solely in absolute terms. • One state department of transportation has changed to measuring whether actual congestion is less than 85 percent of the projected value. It is not using reliability measures at present. Research Needs • Examine available information on latent or induced demand that fills transportation facilities. Robert Cervero’s article in the Journal of the American Planning Association in 2003 provides a good summary of the subject, but more research on the topic is needed. • Identify and document methods and techniques to improve the way we tell our story to the media and the public. This study would focus on developing a narra- tive that describes the data and findings, not on “spin- ning” the data. • Examine and document the effects of different types of land use and land use patterns on congestion problems. • Examine and document factors that influence the mode split and mode share. Among the elements to be considered are land use and city and region size. The study should also consider whether there is an optimal mode split. • Identify and apply techniques for estimating safety and travel time benefits. • Identify and document guidelines or plans con- cerning how to operate systems on a regional basis. Freeway managers can operate the main lanes at rela- tively high speeds or high throughputs, but the influence of this approach on local streets should be examined. In addition, how a corridor analysis identifies the benefits of transit and good operations options should be con- sidered. • Identify and document approaches to use technol- ogy more aggressively in data collection devices. Possi- ble technologies include in-vehicle navigation systems, cell phones, and GPS-equipped personal data assistants. The potential to use the same infrastructure to increase distribution and utility of traveler information would be examined. • Examine and document the data needs for a broad range of facilities and conditions—urban and rural, weekday and weekend, persons and freight. These data needs go beyond the traditional peak-period work trip focus. • Examine and document the effect of congestion and reliability on the activity patterns of American households and businesses. It appears that there have been gradual but significant changes in household and business activity patterns due to congestion, but more research in this area is needed. • Examine techniques to forecast speeds and travel times for future years or alternative strategies. Models are not generally calibrated to speed. This capability should be improved, and models that address this need should be developed. • Examine available hardware, tools, and data needed to estimate recurring and nonrecurring delay percentages. Develop techniques, tools, and methodolo- gies to improve estimates of delay percentages. • Examine techniques for microsimulation of opera- tions improvements. Identify available tools, data needs, and benefit estimation programs. Develop better tech- niques to assess changes in traffic flow and traffic vol- umes from operations improvements. • Examine and document the influence of changes in fuel costs on congestion levels. Examine how eco- nomic slowdowns affect congestion. ENVIRONMENTAL Michael Meyer, Moderator Topics Discussed Participants in this breakout session discussed a variety of environmental issues and possible performance mea- sures for addressing these concerns. Topics included human health, ecosystem health, environmental quality, and land use. Participants discussed potential human health issues associated with the transportation system, which are becoming more important concerns in many areas. This interest may miss other causal factors besides trans- portation, however. There is a need to examine a range of options and outcomes linked to specific issues. One example is health effects related to automobile emis- sions related to compliance with permits. The direct link with transportation may decline over scale. If there is a link, the question becomes what can 7 5BREAKOUT SESSION SUMMARIES 99395mvp70_128 12/13/05 12:36 PM Page 75

we do about it and is it a transportation issue? For example, could roads be moved away from sensitive locations? There may be areas that should be monitored where some populations might be at risk. A better sys- tem for forecasting air quality concerns may be needed. A better mechanism is needed to link transportation and strategy effectiveness. There are strong possibilities for collaboration in high-risk areas, including the use of Congestion Mitigation and Air Quality Improvement program funds for projects. Participants discussed transportation and ecosystem health issues. It was noted that in California special studies have been conducted for habitat and species preservation. Development interests have been involved in some of these efforts. However, a number of smaller areas do not have these types of resources. The Califor- nia examples appear to be the exception rather than the norm. Participants discussed possible links and coordina- tion between state departments of transportation and state resource agencies. Transportation agencies have funding and have a formal planning process. A frame- work may be lacking for resource agencies. A frame- work may be missing for transportation agencies to evaluate transportation’s role in ecosystem health issues. Participants discussed possible performance measures and how to measure and monitor ecosystem health. For example, is it possible to develop a measure of corridor permeability or acreage developed for culverts and fish habitats? The issue of state-level responsibilities versus local land use decisions was discussed. Transportation agen- cies typically have a large-scale perspective. Transporta- tion agencies have a responsibility to ensure that the right things are accomplished. The degree to which transportation played a role in achieving ecosystem health would be important to highlight. Participants discussed possible roles for state depart- ments of transportation in all of these issues. Should transportation agencies be the facilitator for all these types of issues? Is it transportation’s responsibility to pay for all of these things? There is a need to facilitate a meeting of the minds early on rather than waiting until the end. In this way, large mitigation costs might be avoided. It might be useful to identify tools or methods to measure the benefits of mitigation strategies. Being able to tell the story about the value of transportation strategies in environmental projects would be of benefit. Care should be taken to incorporate dollar values of environmental costs into estimates of return on invest- ment. Better estimates of external costs are needed. However, these are really value issues and cannot be quantified. Participants discussed the need to quantify benefits and costs of environmental strategies. This information could help determine whether a project is no longer the right project. A Washington State Department of Trans- portation study of environmental mitigation strategies and costs shows that costs varied widely. It is difficult to quantify roadway relocation projects and environmen- tal mitigation. What do you call amenities mitigation and what is needed to produce a good design? Participants discussed transportation and land use issues. Land use is a larger issue that involves numerous public- and private-sector players. Addressing urban sprawl is a complex undertaking. More information is needed on the secondary and the cumulative impacts of sprawl. Transportation documents should consider the issue of sprawl. Research Needs • Conduct a study to analyze the influence of changes in fuel types on health, especially for individu- als living near major roadways. Examine the potential benefits to improved health from changes in fuel types. • Conduct research to develop a better understand- ing of the relationship between air quality strategies and performance outcomes. • Examine the experience in states where natural resource agencies have strategic planning frameworks, and explore the role transportation plays in these frameworks. Examine the approaches to addressing issues with habitat connectivity. • Examine the impacts of transportation and com- munities on ecology. • Identify and define the costs associated with envi- ronmental mitigation. Help define environmental stew- ardship and document the benefits of transportation and the improvements it makes as part of environmen- tal mitigation measures. • Conduct a study identifying elements of good design versus mitigation versus amenities. Identify com- mon approaches among states and the experience of implementing agencies. • Conduct follow-up work for NCHRP Project 20- 20, including examining the costs associated with addi- tional environmental strategies. Convening a panel of designers might be one approach to address some of these topics. • Conduct studies focusing on the impact smart growth has had on various measures. Examine the expe- riences in Maryland, Florida, and other states with major smart growth initiatives. • Conduct research on the factors that cause urban sprawl, including local governments’ incentives for developers. Denver, Colorado, which is promoting infilling and transit-oriented development, might be one case study. 7 6 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS 99395mvp70_128 12/13/05 12:36 PM Page 76

• Conduct scenario planning on the costs of infra- structure and environmental preservation. CUSTOMER SATISFACTION Randy Halvorson, Moderator Mark C. Larson, Recorder Topics Discussed Participants in this breakout session discussed tech- niques for obtaining information on customer satisfac- tion with the products and services provided by transportation agencies. The use of customer satisfac- tion information with performance measures was also discussed. Participants discussed use of the Internet for cus- tomer surveys. It was noted that the methodology used and the response rate are important for the results of Internet surveys to be meaningful. This approach is effective for some population groups but not for the general population. Internet surveys can also be used effectively to survey agency staff and other agencies. The Internet has worked well for specialized trade-off surveys at some transportation agencies. Participants discussed internal employee communi- cations tools. The Minnesota Department of Trans- portation used 25 employee meetings to communicate the new performance-based state plan. Surveys con- ducted with managers in Oregon focus mainly on their satisfaction with internal management services, such as human resources and finance. Approximately half of managers are surveyed each year. Automated Outlook survey software is used. This software provides for anonymous responses. The entire state government is now doing this in some fashion. The surveys also pro- vide the opportunity for narrative feedback. The Washington State Department of Vehicle Services conducts 2-day forums to convey key messages to man- agers. The director and assistant director travel through- out the state and have one-on-one meetings with staff in a structured program focusing on relationship building. Another approach is to have employees rate how well they believe they are delivering services and how well they believe they are able to do it given constraints or other issues. This approach can be Internet based and is used extensively in the private sector. The use of comprehensive service and accomplishments reports for organizations and a customer report card for all city services was discussed. Seattle, Washington, and New York City were identified as using this approach. More information is available from the Federal Highway Administration performance and accountability reports. Questionnaire design was discussed. It was noted that questionnaire design is crucial in conducting cus- tomer surveys. Be sure to test the survey first. Draw on existing knowledge, with a special focus on what cities and counties have done. They are fairly advanced, and many have done benchmarking. The use of market research to validate measures and set targets was discussed. The Caltrans maintenance department has a web-based customer complaint process. Some 800 responses were received last year in Los Ange- les County. Caltrans monitors the patterns and the high- frequency issues. Complaints are answered within 2 weeks. Letters and phone calls are also received by the department. The Caltrans maintenance department meets with travel information staff monthly to improve public information and obtain feedback. It was noted that surveys of legislators have been con- ducted by some state transportation agencies. Concerns about nonprofessionally designed surveys were discussed. The sequencing and frequency of sur- veys were also discussed. It was suggested that annual surveys of the same user groups may be too frequent. The purpose of customer satisfaction surveys is to deter- mine whether new investments make a difference. Sur- vey methodologies should also provide for comparisons between different regions of a state. Research Needs • Consider conducting a national symposium or peer exchange focusing on customer research practices and experiences. This symposium would focus on shar- ing experience and advancing the state of the practice with transportation customer surveys. • Compile and make available the basic types of questions that should be included in transportation cus- tomer satisfaction surveys. The book by Gallup, Now Break All the Rules, may be of use in this effort. • Identify and document the analytical methods available for use by transportation agencies interested in conducting customer satisfaction surveys. • Identify and document how market research is linked to the decision-making process, including invest- ment decisions. Provide best practice case studies. • Conduct research examining the factors that influence customer decisions related to what modes they will use. • A scan of all customer surveys in use by state departments of transportation is being conducted. States can buy into extra samples in the National Travel Survey this fall. Examine the opportunities for benchmarking versus national average or peer states. Explore waiving match requirements if states use state planning funds. 7 7BREAKOUT SESSION SUMMARIES 99395mvp70_128 12/13/05 12:36 PM Page 77

FREIGHT Barbara Ivanov, Moderator Mark Hallenbeck, Recorder Topics Discussed Participants in this breakout session discussed a variety of issues related to freight performance measures. Topics included identification of bottlenecks and problem areas for freight transportation, identification of appropriate roles for state transportation agencies and other public agencies, safety and security issues, techniques for obtaining information on freight movements and needs, and the involvement of freight stakeholders in the plan- ning and performance measures development processes. Participants discussed the need for better freight data. Many state transportation agencies do not have exten- sive information on freight. Information concerning the volume and direction of freight movements, problem areas or bottlenecks, and the needs and concerns of the various freight stakeholders is often lacking. Other information that would be of benefit relates to the influ- ence of just-in-time delivery needs on the weight of loads per truck and the number of trucks on the roadways. Most public transportation agencies do not have exten- sive expertise in freight or logistics. Better information is needed at the technical level within transportation agencies and for presentation to decision makers. Participants discussed institutional issues related to freight performance measures. There may be institutional issues that influence freight movement, including those associated with international borders or cultural differ- ences. For example, natural resources and manufacturing capabilities in Quebec, Canada, may miss opportunities for customers in the eastern United States because of bor- der crossing issues or bottlenecks between the countries. Participants discussed techniques to involve freight stakeholders in the transportation planning process. Many areas would benefit from examples and sugges- tions concerning techniques to enhance involvement of freight stakeholders in all aspects of transportation plan- ning, policy development, operations, and evaluation. Possible performance measures were identified and discussed by participants. Potential freight performance measures include freight mobility indicators, bottle- necks affecting freight movements, and freight travel and delay times. Participants discussed potential relationships between freight and economic growth. A few areas are exploring this relationship. For example, the New York State Department of Transportation has a project to simulate the freight network and intermodal connections. Participants discussed safety and security issues asso- ciated with freight movement. There are trade-offs between security and efficiency issues. Many trans- portation agencies are examining what these trade-offs are; what role transportation agencies can and should play in helping facilitate discussion of these trade-offs; and what technologies, techniques, and policies can be used to enhance both safety and efficiency. ITS and other advanced technologies provide opportunities to address safety, security, and efficiency concerns. Research Needs Participants noted that there were research or information- sharing opportunities associated with each of the topics discussed. The following research needs were suggested to help advance the state of the practice with freight perfor- mance measures. • Conduct a study of the tools and technologies for obtaining, collecting, and analyzing freight data. Case study examples would focus on identification of data needs; collection and analysis of freight data; and meth- ods to present information to stakeholders, policy mak- ers, and other groups. • Examine institutional issues that may influence the movement of freight, including international border crossings. • Identify and synthesize best practice case studies of methods to involve freight stakeholders in the trans- portation planning, policy making, and project develop- ment process. • Conduct a study to examine current freight per- formance measures used by transportation agencies and to identify other possible measures. Best practice studies would be included. Direction on more extensive freight performance measures would be provided. • Examine the relationship between freight and eco- nomic growth in a synthesis or a more detailed study. • Conduct a study examining freight safety and security issues as they relate to the development of performance measures, including trade-offs with efficiency. 7 8 PERFORMANCE MEASURES TO IMPROVE TRANSPORTATION SYSTEMS 99395mvp70_128 12/13/05 12:36 PM Page 78

Next: Resourcre Papers »
Performance Measures to Improve Transportation Systems: Summary of the Second National Conference Get This Book
×
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

TRB’s Conference Proceedings 36, Performance Measures to Improve Transportation Systems: Summary of the Second National Conference are the proceedings from a conference held on August 22-24, 2004, in Irvine, California. The purpose of the conference was to explore the implementation and use of performance measures and to discuss how to monitor the impact of performance measures on the delivery and quality of transportation services. The proceedings include summaries of presentations made in each conference session and of resource papers. These summaries highlight a variety of agencies' experiences with the use of performance measures and identify research that could improve the use of performance measures. The resource papers prepared for the conference are also included in the proceedings.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!