Cover Image

Not for Sale

View/Hide Left Panel
Click for next page ( 21

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement

Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 20
10 Concern of union members that PBMC will undermine BENEFITCOST FRAMEWORK wages, benefits, work conditions, and job security that government provides It is natural to ask the question if in a particular instance or The need to secure substantial funds through the bud- setting whether PBMC is worth the cost. An economist might getary process for large, multiyear contracts ask, "Do the benefits exceed the costs?" A person experi- Concerns about the contractor's ability to effectively enced with PBMC in the international arena put the question handle reactive maintenance, such as snow and ice this way: "Is there value for money?" (Hardy 2001). control, repair of traffic control devices, and incident and emergency response Change in Levels of Service VALUE The challenges of reassuming the responsibility for FOR = maintenance if the contractor fails to perform, espe- cially if the contracting agency sells off its equipment MONEY Change in Costs and lays off all its maintenance staff except those who are necessary to administer the maintenance contracts Although PBMC has many advantages and disadvan- (Pakkala 2002; Ribreau 2004; Hill et al. 2007; Science tages, whether PBMC provides value for money to the con- Applications International Corporation 2007; survey tracting agency and its customers can be boiled down to one responses; and panel member input). of the following three simple criteria: The LOS remain the same while cost declines ADVANTAGES AND DISADVANTAGES OF The LOS improve for the same cost PERFORMANCE-BASED MAINTENANCE CONTRACTING The LOS improve and cost declines (Hardy 2001) Strictly speaking, a benefitcost framework would iden- Many close observers of PBMC have discussed the pros tify discounted future streams of benefits and costs in dol- and cons of this type of contracting. One observer who has lars, which lends itself to life-cycle cost analysis and an written extensively on PBMC lists these advantages and examination of avoidable user costs such as travel time. disadvantages: However, there are so many outcomes of a maintenance program that some type of multi-attribute benefit function, Advantages such as weighted LOS, makes more sense as a measure of Potential reduction in costs benefits. The multi-attribute benefit function can, in addition Improved level of service (could cost more) to levels of service, incorporate discounted streams of dif- The transfer of risk to the contractor ferent types of benefits, including avoidable life-cycle cost, More innovation accident, travel time, and pollution costs. More integrated services Enhanced asset management Levels of Service Ability to reap the benefits of partnering Building a new industry LOS pertain to all the dimensions of performance a con- Achieving economies of scale. tractor must address. For a single maintenance activity, Disadvantages such as line striping, there may be just a small number of A more costly procurement process performance criteria; for example, retroreflectivity, the A longer procurement process percent of the line that remains intact, and the timeliness A reduction in competition of response to repaint a line with poor reflectivity or a bro- Uncertainty associated with long-term contracting ken section. For contracts that involve numerous mainte- relationships nance activities, performance criteria could consist of a Challenges in mobilizing large number of performance measures and corresponding Loss of agency control and flexibility; for example, targets. Frequently, the values for different measures of to reallocate funds when there are large long-term performance are rolled up into an overall rating. The LOS commitments. that comprise the overall rating reflect many things, such as the condition of assets (e.g., pavements, bridges, guard- [Note: Some items have been tempered to be more gen- rails, and crash attenuators); the response of maintenance eral; for example, "potential reduction in costs," instead of services (incident management, snow and ice control); "reduction in costs" Pakkala (2002)]. and mobility, safety, environmental, and aesthetic issues

OCR for page 20
11 (Smith et al. 1997). Ratings are often grouped into useful Table 4 categories such as pavement, traffic, roadside, and overall COST SAVINGS OF PBMC RELATIVE TO CONVENTIONAL (Graff 2007). CONTRACTS IN SELECTED COUNTRIES Country Cost Savings Contracting agencies normally examine the LOS for individual or groups of assets or maintenance activities. Bar Norway About 20%40% charts such as those in Figure 1 are typical. If over time the Sweden About 30% LOS for most activities or assets goes up, then the conclusion is the overall level of service has improved. Figure 1 shows Finland About 30%35% that under performance-based contracts on both Interstate Holland About 30%40% 35 and Interstate 20 in Texas, there was a period in which the overall maintenance rating declined for some time and Estonia 20%40% then began rising. England 10% minimum Australia 10%40% Evidence on whether PBMC results in improved LOS is provided in the numerous case studies in the next chapter. New Zealand About 20%30% Sometimes, in aberrations such as those shown in Figure 1, United States 10%15% service levels decline for one or more years during the early Ontario, Canada About 10% part of the contract. In other instances, an agency success- fully achieves its goal of reducing costs while maintaining Alberta, Canada About 20% the LOS in-house staff have achieved in the past. Another Some, but might be on the British Columbia, Canada possibility is LOS quickly rise because the roads and main- order of 10% tenance appurtenances are in poor shape. This may require Source: P. Pakkala cited in World Bank Transport Note No. TN-27, rehabilitation of many roadway sections before contract Sep. 2005. maintenance begins on those sections. Under such circum- stances, there may be a significant investment and no cost reduction. Agencies frequently set performance standards at Published information indicates that the cost savings can levels consistent with maintaining assets in good or excel- be measured against engineers' bid estimates, the cost of in- lent condition and with providing good or excellent service house staff to perform the maintenance before the contract, to road users; for example, mowing grass at intersections to the cost of performing maintenance by a control group (usu- maintain good sight distance. ally in-house staff), or other baselines. Frequently, the litera- ture does not clearly state the basis for the cost comparison. Cost Savings No easily accessible documentation--reports, technical There is evidence from a number of sources concerning the papers, and Internet material--is available to support the cost savings of PBMC (Pakkala 2002; Segal et al. 2003; information in Table 4. In most cases, these numbers are Stankevich et al. 2005; Zietlow 2005a). Table 4 shows esti- almost certainly the product of expert judgment, because the mated cost savings regarding PBMC in many countries cost savings for each country refer to multiple performance- around the world. based contracts. Also, it is not clear over what years the cost savings were achieved. ! FIGURE 1 Example of rollup and time series for PBMC performance measures (Source: Graff 2007).

OCR for page 20
12 Nonetheless, it is known that a number of transportation after studies. Preparation of estimates of cost savings based agencies around the world have gone to substantial lengths on deterioration and life-cycle cost models appears to be the to compare the costs of PBMC with other forms of mainte- only way an agency can estimate cost savings over the long nance delivery. For example, the Road and Traffic Authority run or beyond relatively short contract terms. (RTA) of New South Wales conducted a 12-month pilot in Sydney, Australia, that allowed a comparison of maintenance Finally, when the executive or legislative branch directs efficiency of (1) a 100 km section of road maintained by a a transportation agency to downsize or caps its growth contractor under management of a private sector project man- despite increasing maintenance needs, the capacity of the ager; (2) a similar 100 km section of road maintained by RTA maintenance organization to perform all the needed work public employees under management of a private contractor; diminishes. The point is reached at which there may be no and (3) the balance of the publicly owned network maintained alternative but to outsource a significant part of the mainte- and managed by RTA employees (Smith et al. 1994). nance program. The agency is quite likely to respond to the outsourcing mandate without first collecting data to support The Finland Road Administration (Finnra) created a wholly rigorous and defensible before-and-after comparisons. distinct production unit, the Finnish Road Enterprise (FRE), which competes with maintenance contractors throughout the Through late 2004, the performance-based maintenance country. The FRE, composed largely of former public employ- contracts in Australia, New Zealand, and the United States ees, competed successfully for a large number of contracts. were reported to have achieved significant reductions in The reported cost savings of FRE is likely to be defensible costs. However, for Latin American contracts, no equivalent information on cost savings is available. Some contracts were because the FRE cost structure was well known to Finnra awarded for sums below the expected price, which points to when it was a public agency. The key questions are (1) by how possible cost savings. The DNER [Departamento Nacional de much did FRE's costs change once it was created? and (2) was Estradas de Rodagem] in Brazil canceled a procurement for it feasible to compare FRE's costs to Finnra's historical costs Performance Contracts because the contractors' bids were of performing the same or similar maintenance work? in excess of available funds. The bidders factored in the risk that the government might fail to honor its commitments. The upshot is that performance-based contracts might not yield Many maintenance managers regard the cost savings of cost savings right away (Zietlow 2005a). performance-based contracting as unproven or difficult to substantiate. It is difficult to establish the difference in the There is to date insufficient substantive evidence to state cost of government agency forces and private contractors categorically that the initial "savings" are a true reflection of performing the same types of maintenance work (Ribreau increased value for money. It is, however, similarly impos- n.d.). Another challenge is that some of these savings may sible to state that this is not the case (Hardy 2001). reflect the difference in the government's initial cost esti- mates to contract out versus the award amount. Frequently, a more sound approach is to base the evaluation on actual In 2007, a Swedish analyst published an online paper that costs before and after PBMC. attempted to apply regression analysis to determine whether cost savings resulted from PBMC in Sweden, southern Cana- Another problem is establishing defensible direct and indi- dian provinces, and the state of Washington (Stenbeck 2007). rect costs for each type of maintenance activity in both the This appears to be the first effort to rigorously apply statisti- public and private sectors. The private firms will not publicize cal or econometric techniques to determine whether PBMC their cost structure, because it would undermine their ability leads to cost savings. Indeed, the author states that quanti- to compete successfully. State DOTs are generally unable to fication and comparison of contracting outcomes based on accurately distinguish between direct and indirect costs. Two ex post data and new methods appears to be a research need state DOTs tried to estimate their direct and indirect costs by in both Sweden and North America. Although the analysis attempting to adopt Activity-Based Costing, efforts that were suggested that cost savings were achieved contrary to the never completed (S. Wilcox, personal communication, Nov. author's expectations, the estimated savings was not defensi- 2005; R. Arnebeck, personal communication, Sep. 2006). ble for many reasons, including unclear explanations, choice of explanatory variables, and a small data set. Even so, the It is also difficult to determine whether maintenance effort was laudable; however, more observations involving work over a contract increases or decreases life-cycle costs. higher-quality and more credible data are required along If the contract term is relatively short, the contractor usually with a reexamination of the methodology and the variables has no incentive to expend funds on projects with service included in the regression equation. lives longer than the contract term. Generally, lengthening the term of the contract helps align the owneragency and In summary, frequently unsubstantiated, inconsistent, the contractor's interests. If a performance-based contract and incomplete information exists regarding cost savings includes road and bridge work with service lives in the 20-to that have been achieved for specific performance-based 50-year range or more, it is difficult to perform before-and- maintenance contracts. Appendix C provides information