Cover Image

Not for Sale



View/Hide Left Panel
Click for next page ( 53


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 52
42 2. There is a strong emphasis on market research with Desired outcomes the cooperation of potential bidders. This must occur Required service without violating procurement integrity. Performance standard AQL 3. Federal agencies speak of preparing a PWS. This is Monitoring method an outgrowth of the statement of work and leads to Incentives and disincentives the Performance Requirements Summary, discussed later, which becomes the basis, if not the centerpiece, Note that the contractor is expected to have its own QC of a performance-based contract. There are a number process, whereas the Performance Requirements Summary of different recommendations regarding the contents addresses the QA process through performance standards, the of a PWS. The DOD has recommended the following AQLs, and some type of contractor monitoring procedure. typical sections of a PWS (DOD 2001): 5. Many federal agencies have issued Statements of Introduction Objectives (SOO) in lieu of an RFP with a PWS or ele- Scope of work ments of other traditional approaches for performance- Requirements based contracting. The SOO is a brief solicitation that Data requirements shifts most of the responsibility to the contractors for Appendix material preparing the scope of work, the PWS, and the con- tractor evaluation plan. The SOO has other benefits, The online guide, "Seven Steps to Performance-Based too. It promotes innovation and maximum latitude to Services Acquisition," recommends the following contents potential contractors to find the best way to meet the of a PWS: procurement objectives. The SOO is predicated on the belief that the private sector often has more knowledge Introduction and expertise than the government regarding how to Background information provide the services being sought. Scope Applicable documents To prepare an SOO, the government writes a brief solicita- Performance documents tion (usually two to 10 pages), states the agency's objectives, Deliverables and asks each offeror to submit a proposal that describes how it will satisfy the objectives. The SOO calls for a perfor- 4. Those developing performance-based contracts are mance-based contract. Prospective bidders and the contract- taught to not only establish performance targets but ing agency, in the spirit of developing a strong partnership as also Acceptable Quality Levels (AQLs). In theory, early as possible, typically will have discussed how to meet AQLs address the statistical variability of a measure, the agency's objectives well in advance of the release of the similar in concept to Percent within Limits used in SOO. The SOO does not become part of the binding terms of highway QA (Weed 2005). Therefore, not only is it the contract. Rather the scope of work, PWS, incentives and desirable to establish a target level of performance penalties, and the evaluation plan of the successful bidder are but also a minimum and/or maximum level of perfor- incorporated into the contract (Acquisition Central 2005). mance that reflects the known or predicted variation in the performance measure. As a practical mat- ter, many agencies simply set a minimum AQL that BASIC STEPS OF PERFORMANCE-BASED MAINTENANCE CONTRACTING INFERRED FROM THE addresses both the desired performance target and FEDERAL LITERATURE the minimum acceptable level of performance. Once the performance measures and standards have been developed (both targets and AQLs), then the acquisi- This section draws from an extensive federal literature to tion team is urged to prepare a Performance Require- elaborate on the basic steps of PBMC. This literature includes ments Summary matrix. The Department of Treasury a large number of guidance documents from federal agencies (Rogin 2002) has prepared an example Performance and a widely used online resource concerning performance- Requirements Summary with the following six col- based service acquisition. Based on federal resources, the umns (see Table 13, which has been slightly modified steps of PBMC can be described in many ways (Figure 7 for this report; (Rogin 2002)): illustrates one such way).

OCR for page 52
43 Table 13 SAMPLE PERFORMANCE REQUIREMENTS SUMMARY MATRIX Incentives/Dis- incentives for Performance Meeting or Not Standard Meeting Perfor- (what should the mance Standard Desired Outcomes standards be for (what carrot or stick (what do we want to Required Service completeness reli- Acceptable Qual- Monitoring Method will best reward accomplish as the (what task must be ability accuracy, ity Level (how (how will we deter- good performance end result of this accomplished to give timeliness quality, much error will we mine that success has or punish poor contract?) us the desired result?) and/or cost?) accept?) been achieved?) performance?) Customers calling The help desk shall be the help desk shall adequately staffed, 99% of calls are 99% of calls are Survey customers be able to contact a with a sufficient num- answered on the answered on the and evaluate feed- 0.5 of total support staff mem- ber of incoming lines customer's first customer's first back. Inspect call monthly price ber from 8:00 AM to handle potential attempt. attempt. logs (trend analysis). to 5:00 PM, MF. trouble calls. Random sampling of Calls are answered The help desk shall be Calls are answered call activity logs, within 30 seconds adequately staffed, within 20 seconds showing time of Calls are answered or a voice mail can with a sufficient num- or a voice mail can receipt of call and 0.5 of total promptly by help be left; call shall be ber of incoming lines be left; calls shall be call return time. Ran- monthly price desk personnel. returned within 2 to handle potential returned within one dom surveillance of hours for all classes trouble calls. hour of receipt. actual operations of customers. (trend analysis). Time to resolve cus- Time to resolve prob- Random sampling of tomer problem or lem/answer questions call activity logs, 96% of calls 96% of calls answer question is is within the time showing time of received are received are 0.5 of total as short as possible; frames set forth in the receipt of call and resolved within one resolved within one monthly price the need to dispatch SOW or in the Ser- closeout of trouble business day. business day. personnel is vice Level Agreement ticket (Trend minimized. (SLA). analysis). Sample/test calls will Personnel answering No more than two No more than two be made to the help telephones shall be complaints are complaints are Help desk personnel desk; customer sur- courteous and shall made per month made per month 0.5 of total are courteous and veys; complaints will accurately and effi- regarding courtesy regarding courtesy monthly price efficient. be investigated and ciently log in all and/or lost/late and/or lost/late resolved within 1 incoming calls. messages. messages. week of filing. Step 1. Form and Train an Acquisition Team tightly integrated working group that draws on different disciplines. Its size and composition would depend on the The literature on PBMC varies considerably regarding the complexity and scale of the contract. For small, straight- degree that the acquisition team is given attention. It is forward contracts, it is usually sufficient for the team to be hardly mentioned in conjunction with the highway main- composed of one or two knowledgeable technical people, tenance case studies in the literature. The Guide to Per- including the contract manager and an experienced con- formance-Based Operations Contracting (Hyman 2003), tract officer familiar with performance-based contracting. which covers certain types of maintenance such as snow If staff are not knowledgeable about PBMC, it is impera- and ice control operations, devotes considerable attention tive that the all members of the acquisition team educate to the composition and role of the acquisition team. This themselves or receive training regarding performance- guide emphasizes that the acquisition team needs to be a based contracting.

OCR for page 52
44 The online guide, "Seven Steps to Performance-Based Service Acquisition," says teamwork is crucial and that the acquisition team needs to establish ground rules for working together. Drawing on staff with experience in the acquisition process and providing training will help the team to work effectively. The team is likely to go through the well-known steps of becoming acquainted, confronting conflicts and different points of view, developing procedures or rules for working together, getting the job done, and breaking up. The job of the acquisition team is to ensure that the procure- ment meets the program objectives and that any contract the agency enters into is successful. Success will be determined by the cost and the degree that performance standards are achieved. More specifically, during its existence, the acquisi- tion team will need to devote significant time and effort to the following, among other things (DOD 2000; Department of Energy 2000; Hyman 2003; Acquisition Central 2005): Identify specific objectives associated with the prob- lem or each issue Determine the customers/users that will be served Establish the desired outcomes and outputs Prepare a performance-based work statement Establish performance measures Establish the performance standards for each measure; include response times as appropriate Establish AQLs Identify factors outside the control of the agency and contractor and determine the likelihood that those fac- tors will interfere with the contractor achieving the performance targets Decide how to best allocate different types of risk among the agency and the contractor. FIGURE 7 Steps for performance-based maintenance contracting. Step 2. Establish How to Meet Program Objectives The literature emphasizes a number of important points con- The guide suggests that for large, highly complex projects cerning how to achieve program objectives. The thoughts the acquisition team could include most or all of the follow- here amplify themes in the federal literature on performance- ing types of people: based contracting that address issues and practices that high- way maintenance organizations normally deal with. Program manager Project manager First, it is important to define the program objectives. Technical specialist for each important discipline Program objectives may stem from mandates of the execu- Contracting officer tive or legislative branch to increase the outsourcing of Market analyst maintenance, increase the amount of work the private sector Cost/price analyst performs, foster innovation, reduce the size of government, Small and disadvantaged business utilization specialist or improve the efficiency and effectiveness of maintenance. Finance/budget officer Attorney (needed to establish that the agency has statu- Program objectives may be spelled out in policy, regula- tory authority for PBMC and to review draft and final tions, plans, programs, and other written documents. procurement documents) Other stakeholders such as a sponsoring firm, a lending Regardless of how and where the program objectives company, a public safety agency (Hyman 2003). are expressed, it is necessary to clearly understand how

OCR for page 52
45 performance-based contracting will serve the customers Will the contract be a multiphase contract such as of the agency--that is, the road users, people who pay for design-build-finance-operate and maintenance? the roads, and owners of land adjacent to the rights-of-way. What should the term of the contract be? Should it be Typical customer-oriented program objectives include long enough that the contractor can internalize and improving accessibility and mobility, preserving invest- minimize life-cycle costs as well as amortize equip- ment, reducing accidents, reducing energy consumption, ment and facilities? and improving the environment. More narrow objectives What will be the contractor selection criteria? Best might focus mainly on reducing road user costs (accident, value, low bid? Part low bid and part technical? What travel time, and vehicle operating costs), life-cycle costs, should be the percent weighting of cost versus techni- and adverse environmental side effects (Booz Allen Ham- cal? (Department of Energy 2000; DOD 2000, Pakkala ilton 2002). 2002; Stankevich et al. 2006). A corollary to defining the program objectives is decid- Finally, what other institutional and technical issues ing what to contract. For example, the agency may wish to need to be addressed to achieve the contract objectives? An contract only maintenance of signs, striping, and markers, agency may need statutory authority to undertake PBMC. which has important implications for the safety of road An agency may need legal authority to consider unsolicited users. An agency could contract for snow and ice control or proposals for public-private partnerships--for example, for various bundles of maintenance activities. Another option is PBMCs that private firms offer or for design-build projects to contract for all types of maintenance within the right-of- that extend into the operations and maintenance phases. way of limited-access highways. Step 3. Establish a Partnering Strategy and Identify Another set of objectives concerns dealing with agency Qualified Contractors staff in a responsible and sensitive manner and ensuring ade- quate contractor capacity. Open communication and nego- The literature on PBMC is virtually unanimous about the tiation with labor unions is part of the answer. As in the case importance of partnering between the agency and the con- of Finnra, TransitNZ, and other maintenance organizations, tractors. It is critical to eliminate an adversarial relationship it is possible to largely privatize a transportation agency and so that both parties can focus on the program objectives and create something like the FRE, which competes with tra- achievement of performance standards. The literature is vir- ditional contractors. The privatization process needs to be tually unanimous regarding the need to identify qualified handled delicately; otherwise, government employees will contractors early in the process. A qualified contractor is not embrace the change and significant damage can occur to much more likely to be viewed as a trusted partner. TxDOT the contracting industry (Pakkala 2002). found that the professionalism of potential bidders was a key factor in developing acceptance within the agency concern- An issue related to the objective of ensuring adequate ing total maintenance contracts on the busiest portions of contractor capacity is the feasibility of using microcontrac- TxDOT's Interstate system. Experience and training of the tors. This was first tried in South America and has become a agency and the contractor staff both play an important role common practice (Zietlow 2005). In the United States, major in fostering an effective partnership (see citations in Acqui- maintenance contractors make good use of small contractors sition Central 2005; see also, Graff 2001; Science Applica- to perform various maintenance activities. For example, in tions International Corporation 2007). Virginia, the interstate maintenance contractor defined three levels of work and engaged corresponding types and sizes of According to the literature, there are many ways to identify subcontractors (Lande and Dennis 1999). and engage qualified contractors, including the following: The agency needs to ask what type of contract is best able Issue a request for prequalifications or use a postbid to achieve the objectives. The literature suggests the follow- qualifications process ing issues need to be resolved: Conduct market research to identify qualified contractors Will a lump-sum contract be used that includes dis- Solicit comments from contractors, contractor associa- incentives for failing to meet performance objectives; tions, and other stakeholders will there be both disincentives and incentives? Hold a prequalification or prebid meeting Will the contract be a hybrid and include lump-sum Solicit specific suggestions of what to include in the con- and unit prices? tract--for example, the duration and size of the contract Will the contract be a hybrid in other respects and and the contractor evaluation procedure (Acquisition include outcomes and outputs, maintenance and reha- Central 2005 and references cited therein; Pakkala 2002; bilitation, performance and method specifications? Science Applications International Corporation 2007).

OCR for page 52
46 Step 4. Gather Baseline Inventory and Condition Data, Objective--measured score based on a quantitative Estimate Contract Cost, and Secure Funding scale. Point specific--below a threshold or baseline expressed Transportation agencies about to embark on a performance- as a single measurement (e.g., 601 millirem of radia- based maintenance contract need to compile an inventory of tion exposure at a point in time). the assets they want maintained and assess the current LOS Range specific--performance is graded based on the being provided for different types of maintenance. Some LOS contractor falling within different ranges (e.g., from will use measures of condition for physical assets and other below 300 is unacceptable; 300500 is minimally LOS will pertain to measures corresponding to maintenance acceptable; 501700 is acceptable; and above 700 services or operations. This information needs to be shared exceeds baseline). with prospective bidders who would be given an opportunity Subjective--uses qualitative categories of performance to perform an independent assessment. The public agency such as unacceptable (e.g., inadequate management and bidders could exchange information on inventory, asset oversight); acceptable (e.g., satisfactory management condition, and LOS, so there is a mutual understanding of oversight); superior (management demonstrates excep- baseline conditions. The transportation agency will need to tional oversight). prepare an estimate of the cost to perform the work and each organization submitting a proposal will need to do likewise. A key issue is whether many or relatively few mea- sures will be used. The federal and other literature on The transportation agency will need to secure funding for performance-based services contracting acknowledges the performance-based contract. The contract period may be that in some circumstances relatively few performance short (1 to 3 years) with one or more renewal periods, or it measures are sufficient and in other circumstances many may be long (7 to 15 years). In either case, the agency would are required. make a firm commitment to satisfy its contract obligations. If the agency has a large enough maintenance program, it may In practice, within the highway area, most agencies use at be able to meet the commitment through normal expected least one measure for every type of physical asset and main- maintenance appropriations. However, the agency may need tenance service in the contract, such as mowing or snow to make a formal budget request of the governor (the premier and ice control. Different measures for each type of pave- in Canada) and the legislature. ment distress as well as other pavement characteristics are included in many performance-based contracts. Step 5. Prepare the Solicitation (including contract type, selection criteria, scope of work, performance measures, Similarly, there may be multiple condition measures for performance targets, duration of the contract, incentives bridges that, at the minimum, address the deck, superstruc- and disincentives, and performance evaluation criteria) ture, and substructure--for example, the National Bridge Inventory System (NBIS) condition ratings. However, for Most of the issues related to this step were addressed in chap- states that input inspection data into a Pontis Bridge Man- ter two. Here, considerations in establishing performance agement System database, it is more likely there will be 100 measures are addressed and there is further discussion con- condition ratings for each of the CoRe elements. Some or cerning incentives and disincentives. many of these ratings for specific bridge elements could be included in a maintenance contract (AASHTO 1997). A basic task for the acquisition team is to identify the performance measures that will be used. The DOE guidance If response times are coupled with each measure for a on performance-based contracting says that performance typical highway section, the total number of performance measures can be objective or subjective, or contain a combi- measures can double. In some cases, performance measures nation of both. As contracts move in the direction of becom- may number in the range of 150 to 250 in a performance- ing more outcome-based, performance measures could be based maintenance contract. With this many performance more objective, but completely abandoning subjective mea- measures, the contractor must have a rigorous and effective sures may not be a good idea. Objective measures are based QC plan, and the public agency must be equally fastidious on metrics and can be validated. Subjective measures are whether it elects to evaluate the contractor's performance or frequently categorical but are hard to measure. Subjective decides to have a third party monitor the contractor's perfor- measures often are appropriate when it is likely changes will mance in its behalf. occur outside the contractor's control. The DOE (2000) iden- tified the following metrics with respect to a baseline used to Table 14 presents examples of some performance mea- determine incentives or disincentives: sures found in various contracts.

OCR for page 52
47 Table 14 EXAMPLE PERFORMANCE MEASURES, TARGETS, AND RESPONSE TIMES Performance Measures Performance Targets and standards Response Times Potholes No potholes allowed [World Bank (WB), Argentina, Within 3 days after their detection (WB) Uruguay, Chile] Within 2 days after detection; if safety threat No potholes > 3 in. x 4 in. x 1 in. (Virginia, D.C.) act immediately (Virginia, D.C.) 100 mm on any continuous 5 km of major roads lane (New Zealand) Within 4 days (New Zealand) Patching Should be square or rectangular, level, similar materials, no cracks wider than 3 mm (WB) Non-complying patches must be repaired within 3 days after their detection Should be even and <0.5 in. higher or lower than surround- ing pavement (Virginia, DC) No cracks >3 mm wide For any 50 m section of the pavement, the cracked area can- not be >10% of the pavement surface (WB) No unsealed crack > in. on 95% interstate (Virginia, DC) Cracks more than 3 mm wide must be sealed Cracks within 7 days after their detection All cracks should be sealed (Uruguay, Chile) Cracks should be <30% for all sections and <20% for reha- bilitated sections. All cracks should be sealed (Argentina) Dirt, debris and obstacles must be removed: Within 1 h of detecting the condition if Cleanliness of the pave- The road surface must always be clean and free of soil, they pose a danger to traffic safety ment surface and debris, trash and other objects Within 36 h of detecting the condition if shoulders they do not pose any danger to traffic safety Remove all snow to no more than 1 cm Respond to snow conditions within 2 h Snow removal Achieve a measure of skid resistance of <0.3 Achieve snow removal and skid resistance targets within prescribed amount of time after (Finland) it stops snowing IRI < 2.8 m/km (Uruguay) Pavement roughness IRI < 3.3 m/km (Argentina) No tolerance allowed IRI<181 in/mi (Virginia, D.C.) No ruts >15 mm. Rutting <10 mm should not be present in >%5 of road (WB) Rutting above threshold value must be elimi- < 2 mm (Argentina) nated within 15 days (WB) <10 mm (Uruguay) Rutting <10 mm (Chile) <0.5 in. Virginia, D.C.) 12 months (New Zealand) 30 mm depth (New Zealand) Raveled areas must not exist (WB) Raveled area should be <50 ft sq. in 0.1 mile section Raveled areas must be sealed within 30 days Raveling (Virginia, D.C.) after their detection No raveling allowed (Argentina)

OCR for page 52
48 Performance Measures Performance Targets and standards Response Times Bleeding All bleeding surfaces treated (New Zealand) 2 h (New Zealand) Loose pavement edges There shall not be loose pavement edges, or pieces of pave- ment breaking off at the edges (WB) 7 days after the detection of the defect (WB) No >5 m of edge break per km where width of seal loss 1 month where sealed shoulder is >0.5 m from nominal sealed shoulder edge exceeds 100 mm (New wide, 3 months otherwise (New Zealand) Zealand) Height of shoulders vs. <15 mm difference in height (WB) Repairs must be completed within 7 days height of pavement <30 mm difference in height (Argentina) after the detection of the defect Cracks sealed, without deformations and erosions and free of potholes and erosions (WB) Repairs must be completed within 7 days Paved shoulders Cracks sealed, free of potholes and vertical alignment with after the detection of the defect pavement should be <1 cm (Chile, Uruguay) 95% of guardrail/barrier free of structural defects per 100 ft section Repair within 5 days of detection Guardrails/Cable Rail All guardrail posts, offset blocks, panels and connection (not actual requirement) hardware in good condition and in place Cables taut and properly secured according to standard (D.C.) Trees within right-of-way Trees within right-of-way must be protected as necessary None Height of vegetation (except trees) must <20 cm on slopes Right-of-way (outside towards the road and <1.0 m otherwise, must not disturb Vegetation exceeding the threshold height pavement and shoulders) drainage (WB) must be cut back within 7 days after detection <15 cm height (Argentina, Uruguay) With nearly 50 countries engaged in performance-based Turning to incentives, some considerations, drawn from contracting, it is a daunting task to tabulate and compare federal agencies, may be useful to those developing per- all the performance measures in use. (See Appendix B for formance-based contracts for highway maintenance (DOE links to sample procurement and contract documents that 2000; DOD 2000): contain performance measures from various transportation agencies.) Other sources of performance measures include The performance required to earn positive incentives the following: should be realistic and achievable. Incentives could add more value in terms of improved The World Bank (Stankevich et al. 2006) performance (outcomes) than the incentives cost. A Zietlow's website on performance-based manage- corollary is the contracting agency needs to ask whether ment and maintenance of roads from around the world it is willing to pay for the contractor to produce LOS in (Zietlow 2005b) excess of performance standards. Proceedings of the National Workshop on Commonly The contractor is not going to spend more than the Recognized Measures for Maintenance (Booz Allen incentive is worth. It is important that incentives be Hamilton 2002) consistent with the contract value and the effort that The appendix material in the NCHRP Guide for will be required to achieve the desired outcomes. Customer-Driven Benchmarking of Maintenance A cost incentive could be included when there are per- Activities (Hyman 2006) formance incentives to achieve outcomes and outputs. Otherwise, the contractor may not give sufficient atten- Measurements frequently involve taking observations tion to controlling costs. (Note: a lump-sum contract over a section of road. The length of the section needs to be is a natural way to provide a cost incentive because established. For example, should the section length be 0.1 controlling costs helps a contractor maximize profit.) mile, 50 ft, or 1 mile? If there will be incentives to control costs, the contrac- tor needs a good cost accounting system. Sampling plans are an integral part of measurement pro- When a baseline is a foundation for an incentive, objec- cedures. The agency needs to determine the desired accuracy tive performance measures are likely to be more appro- and confidence interval associated with sampling. Also, the priate. This will be the case if incentives are established agency needs to establish whether simple random sampling is with respect to cost, outputs, outcomes, or even sched- sufficient or whether stratified random sampling is needed. ule milestones.

OCR for page 52
49 Achievement of performance targets more important Step 7. Monitor and Inspect Contractor Performance to the success of the contract could receive greater and Pay the Contractor in Accordance with the Contract incentives. Terms The contracting agency should recognize that the incremental increases in performance reflected in Once a contractor is selected and begins work, it will be nec- one or more measures may require a disproportion- essary to monitor the contractor's performance. Generally ate increase in contractor costs. If costs are likely to speaking, an evaluation process, not an inspection process, is increase in this way, the contracting agency could required. Both the agency and contractor need to train their provide the contractor increasingly graduated incen- staffs regarding what are effective QA/QC processes for per- tives as performance improves. formance-based maintenance contracting. Generally the QA/ Avoid offering incentives under circumstances when QC process needs to focus on outcomes. There will be excep- the performance sought is beyond the control of the tions--for example, hybrid contracts that include both method contractor. and performance specifications or those with payments based Incentives need to be carefully tailored to yield the on both lump-sum and unit prices. By and large, a different desired effect and avoid unintended consequences. mind-set is required to evaluate a contractor doing PBMC in The contracting agency may wish to design an incen- comparison to traditional maintenance outsourcing. tive structure that is consistent with performance, changing on a continuous scale or in discrete incre- The Guidebook for Performance-based Services Acquisi- ments. For example, most rehabilitation may occur in tion identifies five assessment methods (DOD 2000): the first year of a highway contract, followed by nearly all maintenance in the concluding years. In this case, 1. Random sampling--Works well for establishing an incentive structure that reflects the sharp change in whether the contractor meets a target a certain percent the type of highway work would be appropriate. of the time. Random sampling is appropriate for large Performance along a certain dimension may cease to populations. Samples may be taken at any time. The be important after a period of time, and so it would sample size may be adjusted if the contractor consis- make sense for any incentive for that performance tently demonstrates good or superior performance. measure to end. For example, an incentive may per- tain to an activity on a critical path. When the end 2. Periodic sampling--Similar to random sampling, but of the path has been reached, further incentive is not occurs at fixed intervals. needed. If some dimensions of performance have incentives 3. Trend analysis--Important for assessing the contrac- and others not, the contractor may neglect perfor- tor's performance over time. mance in those areas that do not have incentives. One way to guard against this type of distorted behavior is 4. Customer feedback--There are a variety of methods to condition payment on the contractor's overall per- to obtain customer input regarding the contractor's formance--that is, require the contractor to achieve performance. In the case of maintenance customer an overall level of performance that subsumes any surveys and a customer service desk that can be contract goals without specific incentives. reached by telephone and e-mail are two good ways. Step 6. Issue the Solicitation and Pick a Contractor 5. Third-party audits--independent audits. Based on the Response Practice at the state and provincial level frequently reflects The transportation agency will have to follow the procedures these basic assessment methods. of its office of procurement and contracts in issuing a solici- tation and selecting a contractor. The contractor monitoring process is closely related to the nature of the partnering between the agency and the The acquisition team will receive proposals, pick a con- contractor. The evaluation process should be constructive tractor based on the selection criteria, and conclude any nec- and help the parties achieve the contract's goals and perfor- essary negotiations. The acquisition team must document mance objectives. This does not mean one can dispense with the reasons it selected a specific contractor. If one or more objectivity. The agency and contractor, through effective entities not selected requests a debriefing, the agency must communication, with a positive attitude, and by maintaining be prepared to provide it. The winning proposal often will an appropriate distance, can have an objective and mutually become part of the binding terms of the contract (Federal acceptable evaluation process. An independent evaluator Acquisition Regulation 2007); also see various state con- can ensure objectivity; however, as indicated earlier, many tracting regulations). agencies believe they lose control by ceding the evaluation responsibility to another entity.