National Academies Press: OpenBook

Transportation Performance Management: Insight from Practitioners (2010)

Chapter: Chapter 3 - Use Performance Management to Help an Organization Focus

« Previous: Chapter 2 - Performance Management Structure
Page 21
Suggested Citation:"Chapter 3 - Use Performance Management to Help an Organization Focus." National Academies of Sciences, Engineering, and Medicine. 2010. Transportation Performance Management: Insight from Practitioners. Washington, DC: The National Academies Press. doi: 10.17226/14384.
×
Page 21
Page 22
Suggested Citation:"Chapter 3 - Use Performance Management to Help an Organization Focus." National Academies of Sciences, Engineering, and Medicine. 2010. Transportation Performance Management: Insight from Practitioners. Washington, DC: The National Academies Press. doi: 10.17226/14384.
×
Page 22
Page 23
Suggested Citation:"Chapter 3 - Use Performance Management to Help an Organization Focus." National Academies of Sciences, Engineering, and Medicine. 2010. Transportation Performance Management: Insight from Practitioners. Washington, DC: The National Academies Press. doi: 10.17226/14384.
×
Page 23
Page 24
Suggested Citation:"Chapter 3 - Use Performance Management to Help an Organization Focus." National Academies of Sciences, Engineering, and Medicine. 2010. Transportation Performance Management: Insight from Practitioners. Washington, DC: The National Academies Press. doi: 10.17226/14384.
×
Page 24
Page 25
Suggested Citation:"Chapter 3 - Use Performance Management to Help an Organization Focus." National Academies of Sciences, Engineering, and Medicine. 2010. Transportation Performance Management: Insight from Practitioners. Washington, DC: The National Academies Press. doi: 10.17226/14384.
×
Page 25
Page 26
Suggested Citation:"Chapter 3 - Use Performance Management to Help an Organization Focus." National Academies of Sciences, Engineering, and Medicine. 2010. Transportation Performance Management: Insight from Practitioners. Washington, DC: The National Academies Press. doi: 10.17226/14384.
×
Page 26

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

This chapter describes insights from practitioners on how performance management programs can help transportation organizations focus on their unique challenges and particular problem areas. At the Kansas DOT, staff members noted that they face challenges keeping performance measures on their agenda when other significant issues (i.e., economic stimulus or developing a new program) take up attention and time. The following insights all address the idea that many successful performance management initiatives are born out of specific agency challenges, rather than simply an overt desire for performance management, and that an incremental, responsive, and transparent approach is crucial. As these agencies have responded to specific challenges, per- formance management has become the lens through which all challenges are viewed. Performance management is a powerful tool for diagnosing and solving problems or avoid- ing problems in the future. In many cases, establishing a performance management program means imposing sweeping changes to an agency’s business practices, and such changes are likely to be met with resistance. The first and best approach to overcoming such resistance is to develop a program that directly addresses the agency’s most salient challenges. Often this will be obvious if an agency is facing a financial crisis or has recently endured potentially embarrassing failures such as project delivery problems, ethical issues, or spikes in traffic accidents. Regardless of the particular area of concern, it is far more important to address that issue thoroughly than to address all issues adequately. Maintaining focus is the key to achieving desired outcomes. For example, “improving safety” may sound like an admirable goal for a DOT, but without greater specificity the agency might spend its resources reducing property damage crashes by 30 percent while fatalities continue to rise. If that outcome does not match the concerns of the agency and its customers, then those resources were allocated poorly. Once broad agency challenges are identified, performance measures can help illuminate their underlying causes. Selecting the correct measures is a crucial step. The quality and specificity, of the measures selected is far more important than the overall number or breadth of available measures. The right mix of performance measures both identifies the underlying causes of par- ticular problems and measures the agency’s progress in mitigating them. An effective and responsive strategic planning process can help to identify short- and long-term goals that are rel- evant, achievable, and appropriate measures. Finally, agencies should be prepared to share both positive and negative results, including an examination of factors that may not be within the agency’s control at all. This ensures that the agency continues to understand its own performance in the context of the “big picture” and is adequately responsive to the needs and priorities of its customers. 21 C H A P T E R 3 Use Performance Management to Help an Organization Focus

3.0 Initiate a Performance Management Program to Identify and Address or Avoid a Compelling Problem Performance management is not an end in itself, but a means to focus an agency on specific priorities. Although these priorities can be deter- mined at multiple levels, the initial focus should be on broad agency goals. A performance management program will not sell itself on its own merits, but instead needs to demonstrate how it can help an agency address spe- cific challenges. Begin by taking on the agency’s most pressing challenges and potential problems. Many, but not all, of the programs evaluated as part of this research effort began as a response to a clear and present problem faced by the agency, rather than a broad desire for performance management by virtue of its own merits. However, a crisis is neither necessary nor sufficient, nor even desirable; rather, it highlights the need for better, more informed decision- making that could be achieved by other means. Popular support and an agencywide appreciation for the value of performance management greatly increase the likelihood of successful implementation of a program. To gain traction, a proposed program should initially highlight significant areas of concern that an agency wants most to address, and these areas of concern should be important to employees throughout the agency, not just the top executives. Budget shortfalls, chronic project delivery issues, lack of confi- dence in agency effectiveness, or highly publicized agency missteps are all potential drivers that might lead to the desire for a performance management program. Use performance measures as agents for change. Just as performance management is most effectively implemented in response to specific chal- lenges, the underlying measures themselves are most useful when they pro- vide a laser-like focus on a DOT’s most challenging problems. At the Virginia DOT, difficulty consistently delivering projects on-time and on-budget led to the agency’s first “Dashboard,” signaling the Virginia DOT’s reengagement with a long-underutilized performance measurement program. The Virginia DOT has since greatly expanded its dashboard program, but its initial success was based on using performance measures to tackle a timely and potentially embarrassing agencywide challenge. In order to avoid the risk of “diluting” the original objectives of perfor- mance management, agencies should resist attempts to institute performance management in every aspect of an agency at the same time, especially early in program development. Furthermore, within a particular agency function, the specific measures used should be tailored to the challenges at hand, even if the result is that certain aspects of that department or function are not measured. Organize around a bold leadership initiative. Effective leaders either bring a performance management philosophy to their position or imprint their own leadership style on an existing program. Particularly when per- formance management is implemented in response to concerns about agency accountability, it is often in conjunction with changes in leadership. To emphasize that a fundamental change is taking place in the way the agency does business, it is helpful to accompany these changes in philosophy 22 Transportation Performance Management: Insight from Practitioners At the Washington State DOT (WSDOT), the original impetus for a strong agencywide emphasis on performance management was an accountability crisis that occurred during the late 1990s in response to funding shortfalls, rapid growth in investment needs, voter repeal of a key revenue source, and erosion of trust between the department and the legislature. A new transporta- tion secretary was brought in specifically to improve the agency’s accountability through performance management. He emphasized a cul- ture of performance management, accountability, and transparency. At the Ohio DOT, growing consen- sus on the need for performance management arose out of an agencywide concern about possible large-scale outsourcing and privati- zation of agency responsibilities. In 1992, the governor launched a management audit of all state agencies, which was conducted by private-sector volunteers. Ohio DOT staff and management both recognized at that time that their jobs were in serious risk of privati- zation. In response, the Ohio DOT supported the Total Quality Man- agement initiative, and manage- ment and staff jointly worked on a partnership to improve quality. Quality Service through Partnership has been written into the Collective Bargaining Agreement since 1992, and the Quality Office is a direct partnership between Ohio DOT management and the Ohio Civil Services Employee Association.

and leadership style with a bold and identifiable new agency program or initiative. These initia- tives must involve both leaders and staff. Many agencies that have successfully implemented per- formance management have done so through the introduction of a new agencywide initiative specifically aimed at promoting performance and accountability. Examples of this include the Total Quality Management program at the Ohio DOT (see sidebar), the introduction of dash- boards at the Virginia DOT, and the widely cited and often-emulated Baltimore CitiStat program, which was initiated, led, and overseen by Mayor Martin O’Malley. In the interest of longevity and long-term consistency, it also is important to design programs that are insulated against the inevitability of top-level administration turnover, discussed further in Chapter 6. Focus on initiating a performance management program, not on completing it. No agency examined as part of this research had a finished performance management program. In fact, it is doubtful that a “complete” performance management program is even possible or desirable. Rather, successful systems build on initial successes and continually refine their program, performance measures, and supporting data. Continuous improve- ment includes changes to goals and objectives based on data analysis, improve- ments in data collection and use, changing fiscal or political constraints, and others. 3.1 As a Program Develops, Use Measures to Diagnose Problems Performance management programs should always begin by addressing fundamental challenges and the outcomes that are sought in mitigating those challenges. Once the agency, with the help of performance measurement, identifies problem areas and desired outcomes, a detailed review of the under- lying reasons for the problems is needed. Some issues can be solved quickly with creative thinking by DOT staff, while others may be more severe and require a more comprehensive evaluation. Seek to understand the causal relationships that underlie an agency’s pro- grams. Once a performance management program is initiated, it is important to move beyond high-level outcomes to more detailed, problem-specific out- comes. Agencies should seek to build a logical understanding of the connec- tion between the inputs and outputs supporting each outcome and how they help inform decision-makers. A well-developed complement of performance measures serves to illustrate that connection, especially over time. Many DOTs, including Florida and Arizona, have developed predictive models for their maintenance programs that are used to demonstrate the relationship between future investment levels and expected performance. The Kansas DOT has used performance management to generate new thinking about problems, such as a new bridge classification system that is easier to understand. Measure and manage performance at multiple levels. There are many lev- els within any agency or state government, and approaches to reporting and using performance data will vary among them. For example, legislators or the public may only be interested in the “tip of the iceberg” (the most critical and easily understood output- and outcome-oriented performance measures), while district managers should apply the same performance management principles to drill down into more specific functions of the organization, building a better understanding of where their problems lie. Use Performance Management to Help an Organization Focus 23 Prior to 2004, the Missouri DOT (MoDOT) experimented with a sys- tem of performance “dashboards,” but interviews with staff members indicated that the agency’s leaders were not focused on performance management, and the dashboards received only limited attention from staff and managers. Since recommitting to comprehensive performance management in 2004, MoDOT has constructed a program that centers on “Tangible Results,” using simple language to describe agencywide strategic goals in terms of desired outcomes. MoDOT’s measures are designed to be as clear and direct as possible, main- taining a clear connection to the Tangible Results on which they are focused. The primary measure for safety, for example, is the absolute number of deaths on the road each year, rather than more commonly used—but also more abstract—measure of fatalities per 100 million VMT, which can fall even as fatalities are rising. By maintaining a strict focus on the original area of concern, MoDOT realized its goal of an absolute reduction in fatalities, even as total crashes, a secondary area of concern, increased.

Agencies also should be prepared to address the inherent biases or under- lying agendas of certain performance management stakeholders, particularly at the highest levels. There may be pressure to publicize or highlight positive measures, and likewise there may be concerns about negative measures, par- ticularly those that highlight issues not under the direct control of the agency. Track changes over time. This is the key to successful analysis of progress. Agencies should employ a balance of leading indicators (signals of future per- formance) and lagging indicators (measures of existing or past performance) to inform the resource-allocation process. To better calibrate assumptions about the impact of future funding levels, performance measures should link system inputs, needs, and outputs. Ideally, performance measures should be sensitive and focused enough to show the impacts of allocation decisions on specific policies and programs of allocation. Time-series data also may show diminishing returns on a particular strategy or program, indicating that it has become less effective over time and that a change in strategy is needed. This was the case in Washington, where the Washington State’s DOT’s (WSDOT) analysis of travel times on HOV and non-HOV lanes showed that the HOV travel time advantage was lessening over time, leading the agency to develop an action plan to address the problem. 3.2 Support Performance Management with a Nimble Strategic Planning Process Strategic planning is the foundation of performance management, but the focus is too often on the product (a plan) rather than on the result (a strate- gic direction for the agency, with concrete actions). Strategic planning should help the agency focus on a limited set of short-term outcomes that are tied to specific performance measures. The strategic plan should make the agency’s focus clear to all staff. As with the measures themselves, the strategic plan that first and foremost identifies the agency’s most compelling problems and lays out a path to address them is far more useful than one that is simply “com- prehensive,” giving equal treatment to all facets and divisions of the agency. Use strategic planning and performance management to navigate through uncertain times. Financial and regulatory uncertainties are becoming the norm for many DOTs, and this trend does not appear likely to reverse itself any time soon. Strategic planning and performance management not only help an agency to operate more efficiently as resources become increasingly scarce, but they also empower the agency to better anticipate external changes and to be more responsive to them. Focus on a limited set of high-level outcomes. A nimble strategic planning process is inclusive but not all-encompassing. On the contrary, it should help the agency to focus on outcomes that can be addressed within a short time- frame (typically 1 to 2 years). Being focused means that effective strategic initiatives have explicit start and end dates, and are supported by clear leading (output) and lagging (outcome) measures. Recognize when excessive planning is getting in the way of decisive action. Although strategic planning is the basis for effective performance 24 Transportation Performance Management: Insight from Practitioners Oregon DOT is in the process of implementing a performance man- agement program for its Highway Division. As part of this process, the DOT has identified both lead- ing and lagging indicators for each of its major functions. Lagging indicators were defined first, to examine fundamental outcomes. Leading indicators were developed to help predict how fundamental outcomes might change over time. For example, in the safety area, Oregon DOT uses total fatalities as a lagging (outcome) measure. The leading measure is the backlog of safety needs. Addressing these needs should help reduce fatali- ties, but it may take years to see the change or even evaluate updated data. Since 1996, Ohio DOT has produced a biennial state-of-the-system report and business plan. Each edi- tion of the plan contains strategic objectives to be achieved within the biennium, longer-term goals, and performance measures. In early edi- tions of the business plan, man- agers developed strategies for their individual business units without consideration of the department. More recently Ohio has focused on developing a consensus around a small number of strategic initia- tives, selected by a core group of senior managers for the entire agency. Each initiative is linked to a handful of specific action items, including assigning responsibilities to specific divisions or individuals.

management, there comes a time when an agency must move the focus from planning to measuring and changing behavior. Organizations that spend con- siderable time developing a series of interrelated business plans are not more likely to use them, as the process to develop them can become burdensome and exhaustive. Those agencies that hone in on a limited set of priority out- comes are more likely to address them quickly and effectively. The Florida DOT has required all business units and districts to develop business plans. However, most have not, and those that have been developed have not often been used. Although business plans can be useful to the extent that they focus the agency, requiring all units of an agency to develop them may not have the intended impact. 3.3 Use Performance Management to Improve Agency Transparency In sharing performance data, it is more important to sell the process than the agency. The goal of performance management is to tackle difficult prob- lems and improve the agency, not to provide public relations material. Agen- cies must work through resistance to presenting negative results and must recognize that performance measurement will not only highlight improve- ments, but also may uncover chronic problems. Often, short-term pressing problems will lead staff and managers to resist measuring performance because they do not feel they have the time to address longer range process improvements. Also, improvements in data collection techniques, the fre- quency of data collection, and the completeness of the dataset, may give the impression that things are getting worse, when in reality they are only being more accurately measured. In other cases, negative performance data may not reflect poorly on the agency itself at all, but rather on an outside factor such as inadequate funding or rapidly increasing demand. As part of focusing the agency, it is important not to avoid these difficult challenges. Performance management should highlight existing and potential prob- lems. Performance management will never result in a perfect agency. Rather, the measure of success of a performance management program is whether an agency is developing a deeper understanding of the nature of its problems and developing programs to address them. Performance management is not the end of the process, but it is a crucial beginning point. If an agency does not understand the nature of its problems, it is in a poor position to correct them. In an environment of constrained resources, tradeoffs are inevitable, and the agency that best understands the nature and consequences of these trade- offs will be most effective going forward. High-quality performance data that link leading and lagging indicators not only help the agency to prioritize its needs, but also highlight the overall unmet financial need. For example, at Pace Suburban Bus Service, officials indicated that although improved analytical capabilities driven by their Intelligent Bus System have made it easier to iden- tify problem areas and the steps necessary to solve them, resource constraints mean that these solutions are not necessarily implemented. However, armed with this knowledge and the data to back it up, the agency can make smarter use of scarce resources and can present a stronger, more convincing, and more credible case for increased funding in the future. Use Performance Management to Help an Organization Focus 25 The Indiana DOT (INDOT) organ- izes its performance measures by function—external (reported quar- terly); tactical (same measures as the external group but reported by District); and operational (more detailed, organized by INDOT’s organizational chart, and reported monthly). The measures are stored in the Management Information Portal (MIP) that provides standard reports for various audiences and access to performance data for INDOT staff. Development of the Gray Note- book was part of a larger effort at the WSDOT in the 1990s to address an accountability crisis and to repair an erosion of trust between the department and the legisla- ture. The WSDOT owes much of its improved public stature to increased accountability and trans- parency within the project delivery process. The WSDOT made on-time and on-budget project delivery a priority, seeking to avoid surprises by combining quantitative project data and results with detailed, can- did narratives. Project delivery per- formance is reported every quarter on project cost, scope, and sched- ule. Deadlines are set early in proj- ect development, and project teams are required to keep within these parameters or explain why they have deviated. The Gray Note- book identifies projects with sched- ule, scope, or budget concerns, holding project managers account- able for on-time and on-budget performance of their projects.

Don’t just report the numbers. Especially when dealing with measures that indicate negative trends, it is vital that the measures be presented in the appropriate context. Reports should use a narrative description to ensure that key points are communicated. In many cases there may be unmeasured externalities or unusual circumstances at play. For example, Washington State DOT includes substantial narrative with the performance measures presented in their Gray Notebook (see sidebar), to provide clear explanation and analysis of results. Washington State DOT is discussed further in the sidebar case study in this section. Several officials interviewed as part of this research also indicated that improved data collection techniques and more complete data sets actually gave the impression that conditions were wors- ening, when in fact they were simply being measured more thoroughly. This is a perfect example of the type of anomaly that could be addressed through a thoughtful and thorough narrative explanation. Remember the big picture. DOTs (and all government agencies) are often hesitant to include measures of factors that are outside of the agency’s direct control. Among these are “Societal Outcomes” that measure quality- of-life. Some of these factors might be neither directly caused by nor directly responsible for agency performance but are important because they are the most direct measures of the real circumstances experienced by citizens. Ide- ally, the agency looks at both its own performance and the circumstances of its community, and relates one to the other to affect real positive outcomes. For example, while air quality is not directly controlled by DOT activities, it is a factor of great importance to residents of urban areas throughout the country, and is undoubtedly related to many DOT activities and resource allocation decisions. In many cases the appropriate level for the develop- ment of societal level measures might be at a higher level of government (i.e., overall state government instead of inside the DOT). As governmental entities develop broader performance management and measurement ini- tiatives, transportation agencies can link to a broader set of societal goals that the transportation agency is supporting through their investments. The New Jersey DOT has developed an approach for its Capital Investment Strategy (a performance-based effort to guide capital decisions) that incorporates measures from the New Jersey DOT, New Jersey Transit, and two public toll road operators. This strategy allows major decisions to be made that reflect a coordinated approach to transportation challenges, rather than a more narrow focus on an agency’s limited sphere of influence. 26 Transportation Performance Management: Insight from Practitioners In King County, Washington, the county government’s “AIMs High” (Annual Indicators & Measures) program integrates Performance Measures of agency effectiveness with Community Indicators that represent overall conditions in the region. The goal of this integration is to demonstrate the complex relationship between the agency’s decisions and the overall condi- tions in the community and the influence that these two realms have upon one another. King County’s Performance Measures and Community Indicators span nine different topic areas, includ- ing Natural Resources, Health, Transportation, and Governance. Within each topic area, the two categories of measures are dis- played alongside one another, but always distinguished, and each includes a discussion of each fac- tor’s determinants, and the county’s role in influencing it.

Next: Chapter 4 - Performance Management Must Engage with Employees »
Transportation Performance Management: Insight from Practitioners Get This Book
×
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

TRB’s National Cooperative Highway Research Program (NCHRP) Report 660: Transportation Performance Management: Insight from Practitioners explores the concept of performance management and examines how other agencies bring performance management into the decision-making process.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!