Click for next page ( 30


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 29
Volume II: Guide for Performance Measure Identification and Target Setting The basic documentation can be established as the performance measurement program is launched. Updating should occur whenever new measures are added or procedures are modified. Separate doc- umentation (i.e., metadata) should be maintained on specific performance results--for example, to note special factors that influenced the results (e.g., sharp rise in gas prices or severe weather), or devi- ations from standard procedures (e.g., statistical procedures used to compensate for missing data). If and when the agency decides to change the performance measures that it uses or to modify the details of how a given performance measure is calculated, the documentation for the old measure should be archived along with the performance results that exist. This documentation will be invalu- able for future construction and interpretation of performance trends. 3.4 Establishing Performance Targets Setting performance targets involves a mix of considerations: Financial--Targets need to reflect a realistic projection of available funding levels. Policy--Targets need to reflect current policies and priorities, as well as input from executives, legislators, and customers. Technical--Targets need to be achievable based on current and forecast conditions or performance. Economic--Targets should be set (where feasible) based on economic efficiency considerations, including minimizing life-cycle costs and maximizing the benefit obtained for a given level of investment. This section describes steps for establishing performance targets by considering all of these factors and using them as an integral part of the agency's resource allocation process. Step 1: Define Contexts and Time Horizons The first step is to make sure you have a clear picture of how the performance targets will be used and what time horizons they will cover. Performance targets may be used in a variety of contexts: In long-range transportation plans. In regional or corridor plans. In short- or medium-range (e.g., 1- to 5-year) internal business planning processes. As part of the agency's capital or operating budget process (including TIP/STIP development) involving communication to legislative bodies and/or transportation commissions. As part of fulfilling requirements for the GASB 34 modified approach to asset financial report- ing. If you are setting targets for GASB 34 reporting, you will need to establish targets for the condition of eligible assets, disclose these targets publicly, and verify each year that the current performance values meet the stated targets for the year. 29

OCR for page 29
Volume II: Guide for Performance Measure Identification and Target Setting The contexts you identify will help to clarify over what timeframes targets are needed. Many agencies set both long- and short-range targets, using different processes. Short-range targets are more firmly based on financial constraints, whereas long-range targets can be based more on technical, economic, and policy considerations. However, for targets to be credible, care should be taken to make targets set for different timeframes and in different contexts consistent with one another. Step 2: Select Scope of Measures for Targets For each of the target-setting contexts identified in Step 1, you will need to identify which measures are to be included. Not all performance measures are amenable to quantitative target setting because some are qualitative in nature and some agencies do not have sufficient control over their value. Some performance measures may be candidates for quantitative target setting, but do not have suf- ficient baseline or trend data established to allow for reasonable targets at present. Others may have baseline data available, but tools or methods are not available to reasonably predict their value. It may be helpful to label performance measures as "mature" (baseline data and targets exist), "emerg- ing" (baseline data are available, but no target has been set), and "developmental" (neither data nor targets exist yet). Where targets for multiple measures are being considered for a given policy objective (e.g., pave- ment preservation), technical analysis should be conducted to see if one of the measures acts as a "controlling target." For example, if targets for both "average condition" and "percent in poor con- dition" are set, it may be the case that achieving one target (average condition) will always auto- matically result in achieving the other (percent in poor condition). In this case, you may want to consider setting a target for only one of the two measures. The potential for conflicts or competition across the targets should be anticipated. Some of these conflicts relate to resource allocation and can be addressed in Step 5--for example, it may not be possible to achieve targets for both hazard elimination and preservation. Other types of conflicts (e.g., between project delivery speed improvement and project quality improvement) will require careful consideration to arrive at a reasonable balance. Step 3: Develop Long-Term Goals Setting short- to mid-term performance targets that are realistic based on funding availability is crit- ical for helping decision-makers to make performance-based resource allocation tradeoffs. This is a fundamental part of good asset management practice. However, many agencies want to establish long-term performance goals that stand on their own, independent of resource constraints. In some cases, such performance goals can be established based on considering the current baseline and the level of improvement that is feasible and desirable based on public input or benchmark values from peer agencies. However, there are also a host of techni- cal and economic considerations that should be explored that can serve as solid justifications for long-term goals. Targets based on these goals that are backed up by technical and economic analy- sis can be used as compelling arguments for budget requests. 30

OCR for page 29
Volume II: Guide for Performance Measure Identification and Target Setting Approaches to establishing long-term performance goals include the following: Measure what the current baseline level of performance is, and then set goals for improvement using benchmark information about what peer agencies have achieved through focused efforts to improve performance. Align agency goals with nationally based targets (e.g., the national highway safety target of 1.0 fatality per 100 million VMT). Use customer surveys and/or tracking of complaint information as a guide to establish "tolera- ble" threshold performance levels. Use available models that show how user costs (e.g., travel time, vehicle operating costs, and accident costs) change with different levels of condition or performance. Examining the slope of this curve can show the performance level at which user costs begin to increase rapidly. Use the law of diminishing marginal returns to identify the point at which additional investments begin to have a declining degree of impact on improvements in performance--in other words, where the slope of the investment performance curve begins to decline. Curves of changes in per- formance measure value per unit of investment (see Figures 4 and 5) can be used for this purpose. A corollary to the law of diminishing returns is that caution should be exercised about setting goals or targets that call for 100-percent achievement of a particular condition or service level, since the benefit/cost ratio associated with achieving the last 1 percent will typically be quite low. For example, setting a goal of no structurally deficient bridges could necessitate costly replacement of a bridge that provides a redundant link in the network and is not heavily used. Use a tool such as the Highway Economic Requirements System (HERS) or the Highway Design and Maintenance Standards Model (HDM-4) to identify the performance levels that could be achieved through implementing all improvements with benefits exceeding costs. Define a set of minimum standards for infrastructure characteristics and performance (such standards may already be in place), and establish a long-term goal of meeting these standards. Caution should be exercised in defining standards so that they are feasible and reasonable. Related to the standards-based approach, a methodology for defining either deficiencies or needs (i.e., the cost of addressing the deficiencies) can be applied. This then provides the basis for setting a long-term goal of eliminating all deficiencies or reducing the backlog to 0. Alterna- tively, a goal of keeping the backlog from growing might be established. For setting infrastructure condition goals, the following additional types of approaches can be used: Establish a threshold for "poor" pavement based on a level of roughness that is noticeable to road users (particularly trucks) and is associated with a marked increase in road user costs (due to vehicle wear and tear, increased fuel consumption, or reduced speed). Set a long-term goal of having minimal travel on poor roads (e.g., less than 5 percent). Use capabilities of pavement and bridge management systems to determine a long-term opti- mal network condition distribution, which minimizes life-cycle costs. Use this distribution to set goals for either average condition or percent of infrastructure (e.g., number of bridges, deck area, and miles) above a given threshold condition level. 31

OCR for page 29
Volume II: Guide for Performance Measure Identification and Target Setting Base goals on maintaining a steady-state condition distribution in order to avoid future peaks in preservation or replacement costs that would be difficult to address given a relatively con- stant level of funding. These goals would be expressed in terms of the percent of the network in different ranges of condition level (or remaining life categories). In conjunction with many of these approaches, separate goals can be established for different por- tions of the transportation network to reflect differing degrees of strategic importance (e.g., based on functional class, truck routes, and trunk line networks). Step 4: Consider Funding Availability It can be a challenge to gain consensus on performance targets when levels of performance that are desirable from an economic, engineering, or customer perspective are not feasible from a financial perspective. However, performance targets need to be achievable in order to be meaningful and to serve a useful purpose in an agency's decision-making process. Ideally all performance targets, particularly those that apply to a relatively short time horizon (e.g., fewer than 57 years), should be placed within the context of future funding availability. Projections of future funding can be made based on past trends and available information about future appropriation levels. Historical breakdowns of funding allocation and use to program areas and cost components can provide useful input to future projections. In many instances, a scenario approach can be used, in which targets are set based on continua- tion of current funding levels, 1020 percent lower, and 1020 percent higher. Additional sce- narios for allocation of available funds across program areas can be analyzed, as discussed below in Step 5. Step 5: Analyze Resource Allocation Scenarios and Tradeoffs The ability to meet performance targets depends not only on overall funding availability, but also on how funds are allocated across and within program areas. Performance targets can pro- vide the basis for determining how resources are to be allocated when there is available flexibil- ity to do so. The capability to predict performance and relate performance to cost is required for formulation of realistic targets based on anticipated funding availability. Many pavement and bridge management systems include this capability as a standard feature. Other tools, such as the FHWA Highway Eco- nomic Requirements System--State Version (HERS-ST), can predict a broader set of preservation, safety, and congestion-related performance measures for different resource allocation scenarios. Some agencies have developed in-house tools or informal spreadsheet-based or "back-of-the-envelope" calculation methods to provide this capability as well. Available analytical tools for relating asset performance to cost were identified in NCHRP Report 545 (2). Figures 4 and 5 illustrate examples of performance-cost relationships that can be produced by man- agement system tools and used as guides for target setting based on funding and resource alloca- tion scenarios. 32

OCR for page 29
Volume II: Guide for Performance Measure Identification and Target Setting Figure 4. Example of Budget Scenarios and Effects on Infrastructure Condition Bridges with Health Index >75% (in Percent) 100 High Expenditures, Increase Over Status Quo 80 60 Moderate Expenditures, Maintain Status Quo 40 20 No Additional Expenditures, Do Nothing Policy 0 0 1 2 3 4 5 6 7 8 9 10 Time (years) Figure 5. Relationship Between Infrastructure Condition and Needed Expenditure Relative Budget Dollars (in Millions/Year) 25 20 15 10 5 0 0 20 40 60 80 100 Percent Bridges with Health Index >75% in 10 years 33

OCR for page 29
Volume II: Guide for Performance Measure Identification and Target Setting Step 6: Consider Policy and Public Input For many agencies, target setting is a process that involves engagement with agency executives, transportation commissions or boards, elected officials, and customers. Approaches to obtaining input can vary, but ideally should involve a two-way communication process in which: The agency educates external stakeholders about current baseline performance and trends, fea- sible levels of performance given alternative levels of resources, and technical and economic fac- tors to be considered and External stakeholders provide feedback on desired performance levels and priorities across dif- ferent measures. Public opinion surveys can also be helpful in the target-setting process to understand the relation- ship between different transportation system performance levels and the level of inconvenience or discomfort perceived by users. Step 7: Establish Targets and Track Progress This final step involves selecting actual target values based on the work accomplished in prior steps, reporting these targets, and tracking progress toward meeting the targets (as discussed in the sec- tion on designing communication devices). Periodic adjustments to targets should be considered based on the degree of progress made, changes in policy or priorities, or emergence of information or factors not previously considered when the initial targets were established (e.g., dramatic rise in gasoline prices or passage of a new transportation finance package). Examples of targets established by state DOTs are provided in Appendix B. Figures 6 through 8 illustrate useful graphical formats for communicating performance measure values with respect to targets. 34

OCR for page 29
Volume II: Guide for Performance Measure Identification and Target Setting Figure 6. Minnesota DOT's Performance Target Levels Performance Level Future Performance Gap 20Year Performance Level rget ce Ta man erfor ed P y-Bas Polic ns Projectio ormance ased Perf Trend-B Current Performance Current Performance Level Level 1997 2001 6Year 10Year 20Year Target Target Target Baseline Performance Figure 7. Florida DOT Communication of Performance Targets Percent Pavement Which Meets Department Standards 100 90 Objective 80% 80 70 60 50 40 30 20 Actual Planned 10 0 1996 1997 1998 1999 2000 2001 2002 2003- 2004 2005 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 Fiscal Year 35

OCR for page 29
Volume II: Guide for Performance Measure Identification and Target Setting Figure 8. Montana DOT Communication of Performance Targets Average Congestion 100 Interstate System Primary System 80 NHS System 60 NHS and Primary Performance Goal Congestion Index 55 (>LOS D) 40 Interstate Performance Goal Congestion Index System 70 (>LOS C) 20 0 2003 2004 2009 2013 Year 36