Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 88
78 APPENDIX A QUESTIONNAIRES This appendix contains blank questionnaires that were administered to U.S. states and Canadian provinces, and private contractors that do PBMC. STATE/PROVINCE QUESTIONNAIRE NCHRP TOPIC 37-09 PERFORMANCE-BASED CONTRACTING FOR MAINTENANCE Background and Purpose State departments of transportation (DOTs) are confronted with a combination of growing needs and resource limitations for maintaining the highway system. This has intensified their interest in contracting for maintenance services. The transportation agencies have developed various performance-based contracting methods, including the means to measure and report on performance. NCHRP has engaged Applied Research Associates, Inc. (ARA) to conduct a synthesis of practice to obtain information on performance-based maintenance contracting. The synthesis will explore the following topics: x Performance standards and relevant measures commonly used to address performance-based contract delivery for different types of maintenance activities x How performance levels are established, measured, and evaluated in maintenance contracting x Identified best practices in monitoring and reporting performance-based contract maintenance delivery x Reported costs, benefits, risks, and possible shortcomings of adopting performance-based maintenance contracts x Basic elements necessary to initiate performance-based maintenance delivery, including contractor acquisition strategy, evaluation criteria for contractor selection, and prequalification process x Contractual provisions, such as payment methods, including incentives and disincentives x Examples of successful and failed applications in performance-based maintenance contracting One of the main sources of information for this synthesis is a survey of state DOTs and key maintenance contractors. Please complete the survey on the following pages and return it by e-mail to the principal investigator, William Hyman, at bhyman@ara.com. All the states share in the costs of this research. Your agency's response will maximize the value of the resulting Synthesis report to every transportation department that sponsored this work. Thank you very much. Part I. Respondent Information Head of Maintenance Name: Title: Agency: Address: City: State: Zip: Phone: Fax: E-mail:

OCR for page 88
79 Please return the completed questionnaire by April 24, 2006, to: William Hyman Principal Investigator Applied Research Associates, Inc. 7184 Troy Hill Drive, Suite N Elkridge, MD 21075 E-mail: bhyman@ara.com Phone: 410-540-9949 Cell: 301-593-7842 Fax: 410-540-9288 Part II. Involvement in Performance-Based Contracting (PBC) for Maintenance 1. PBC is a contracting method that provides incentives and/or disincentives to the contractor to achieve desired targets for measurable outputs and outcomes. Does your agency use performance-based contracting for any of the following types of maintenance contracts (check all that apply)? a. Areawide contracts covering a subunit of the state and involving a single activity or a related group of activities such as rest area maintenance b. Areawide contracts covering more than one activity or related group of activities within a state c. Areawide contracts covering all of the state and all or most maintenance activities d. For selected activities in a corridor e. For fence-to-fence maintenance covering all activities in a corridor f. No. Our agency has never done PBC. 2. Please estimate how many PBCs of different types your agency currently has? a. Areawide PBC b. Areawide PBC c. Areawide PBC d. PBC for e. PBC for fence- covering a subunit for more than one covering all or selected activities to-fence of the state for one activity or related most activities within a corridor maintenance activity or related group of activities within a state covering all group of activities activities in a corridor 3. If you do not do PBC, check each reason that applies. If you do PBC, skip to question 4. (Note: If you do not feel qualified to give a particular reason, leave the box blank.) a. We have no experience with PBC and have found our current contracting methods satisfactory. b. There is resistance or discomfort with PBC on the part of staff and our maintenance leadership. They fear that in the future not only will the performance of contractors be measured but so will the performance of agency staff. c. Our staff and maintenance leadership fears that PBC could jeopardize many civil servant jobs. d. We do not have the maintenance management systems and quality assurance procedures in place to make PBC work. e. There is a general mistrust of the private sector. f. Agency maintenance personnel are unionized and the union strongly believes PBC is not in their interest.

OCR for page 88
80 g. Our agency has the resources and the expertise to do most maintenance and therefore we contract out only what is essential and see no need to do PBC. h. We have tried PBC in the past and did not find it to be successful. If you check this box, please explain: i. Other. Please explain directly below: 4. Does your agency or maintenance organization have a published plan (strategic, policy, business, system, or highway plan) or program (maintenance program, highway program, transportation improvement program) that calls for PBC? Yes No 5. Do any of your agency's published plans or programs identify specific performance measures that relate to important categories of maintenance? Yes No 6. Do any of your agency's published plans or programs establish maintenance performance targets or standards corresponding to the performance measures? Yes No IF YOUR AGENCY HAS NEVER USED PERFORMANCE-BASED CONTRACTING (YOU ANSWERED "NO" TO QUESTION "1f"), THEN YOU ARE DONE ANSWERING QUESTIONS. PLEASE RETURN THE COMPLETED PORTION OF THIS SURVEY TO BILL HYMAN BY E-MAIL OR MAIL AT THE ADDRESS ABOVE. Part III. Motivation for Performance-Based Contracting Rate on a scale from 1 (not relevant) to 5 (highly relevant) each possible reason listed below regarding why your agency decided to use performance-based contracting: 7. Over time the number of staff in our agency has been reduced and we do not have sufficient labor resources to conduct all the maintenance required. Management views PBC as an effective response to downsizing. Rating: 8. There has been political pressure to rely increasingly on the private sector and PBC because they are driven by profit and other performance measures that affect their bottom line. Rating: 9. We decided to try PBC based on the reported success of this approach in other states and/or countries. Rating: 10. Reported cost savings of other states and countries prompted us to try PBC. Rating: 11. We felt we could shift most of the risk of contracting from our agency to the contractor by using PBC. Rating: 12. Our PBCs use lump-sum contracts that provide predictable financial obligations and promote stable expenditures for contract work. Rating: 13. Our agency applies the philosophy of "management by results." PBC fits well with this management approach. Rating: 14. The management approach of our agency has become outcome and customer oriented. PBC is consistent with this approach. Rating:

OCR for page 88
81 15. Our agency contracts out a lot of maintenance and the use of PBC is a logical extension of our past contracting practices. Rating 16. Our agency's commitment to continuous quality improvement was an important reason for adopting PBC because of its focus on measurable outcomes and outputs of contractors. Rating: 17. Management had a conviction that the efficiency and effectiveness of maintenance could be improved if contractors were given incentives (or disincentives) to achieve measurable maintenance outcomes. Rating: 18. Other reason for using PBC: Rating: Part IV. Market Research, Procurement, and Partnering 19. During the procurement phase for a PBC, does your agency identify potential bidders and their capabilities? Yes No Sometimes 20. Do you interview industry leaders to obtain their input before drafting an RFP for a PBC procurement? Yes No Sometimes 21. Do you issue Requests for Information to learn more about what a PBC entails from the private sector point of view? Yes No Sometimes 22. Do you issue a Request for Qualifications (RFQ) or Letter of Interest (LOI) with a request for supporting qualifications before issuing an RFP for PBC? Yes No Sometimes 23. Do you issue an announcement of forthcoming procurements regarding RFPs and allow questions and answers prior to the issuance of an RFP? Yes No Sometimes 24. Do you hold one or more preproposal conferences and respond to both oral and written questions? (a) Yes No Sometimes. If you have a preproposal conference(s), is it/are they mandatory? (b) Yes No Sometimes 25. Do you allow potential contractors to have individual discussions with a PBC technical program manager in advance of issuing an RFP? Yes No Sometimes 26. Do you provide baseline inventory and condition data to potential bidders either in advance of issuing the procurement or as a part of the RFP? Yes No Sometimes 27. Do you allow potential bidders to inspect the facilities they may be responsible for maintaining to determine the accuracy of the baseline inventory and condition data and estimate the cost of doing different types of maintenance work on the roads that will be covered by the PBC? Yes No Sometimes 28. Do you seek agreement with bidders that the performance measures that will be used are practical to apply and will yield measurements consistent with the project objectives the agency is trying to achieve? Yes No Sometimes If no, why not? 29. Do you seek agreement with bidders that the incentives and disincentives incorporated in the PBC are reasonable? Yes No Sometimes If no, why not?

OCR for page 88
82 30. Do you have prospective bidders review and comment on the draft RFP? Yes No Sometimes 31. Is developing a strong partnership between your agency and the contractor an essential part of your approach to PBC and is the importance of partnering reflected in all your procurement announcements and the RFP? Yes No Sometimes Part V. Use of the Maintenance Quality Assurance Process and Benchmarking A previous study, NCHRP 14-12, Maintenance Quality Assurance, defined a performance measurement process that involves taking a random sample of highway sections, and periodically measuring the Levels of Service (LOS) that result from different maintenance activities. LOS refers to either the condition of an asset or the service level that is achieved, for example, roughness of pavement or grass height achieved by mowing. Typically, LOS for each type of asset and service is scored on a scale of 0 (worst possible) to 100 (best possible) and the results weighted to achieve a composite LOS score, also on a scale of 0 to 100. 32. Does your agency use a Maintenance Quality Assurance process to measure outputs and outcomes of in-house maintenance efforts? Yes No 33. Does your agency use a Maintenance Quality Assurance process to measure outputs and outcomes of contract maintenance? Yes No 34. Does your agency use PBC consisting of a Maintenance Quality Assurance process coupled with incentives/disincentives? Yes No NCHRP Project 14-13 resulted in a guide on customer-driven benchmarking of maintenance activities. Benchmarking is defined as using outcome, output, and input measures along with factors outside the agency's control to identify best performers and their corresponding business processes and work methods. These are best practices. Customer-driven benchmarking focuses on customer-driven outcomes. 35. Do you use benchmarking to compare the maintenance performance of organizational units within your agency? Yes No 36. Do you use benchmarking to compare the maintenance performance of organizational units in your agency with the performance of maintenance contractors? Yes No Part VI. An Example of a Performance-Based Contract In this section, you are requested to provide information on a performance-based contract that you think would be of interest and provide valuable lessons to other states. 37. What is the name of the PBC? 38. Please describe the scope of work (activity specific, corridor, areawide, etc), the general nature of the activities involved, and whether incentives and/or disincentives apply to contractual provisions and/or the achievement of targets for measurable outcomes. 39. What was the type of contract? 40. What was the contractor selection criteria?

OCR for page 88
83 41. What was the term of the PBC? Number of years for the base period? Total number of years the contract could be extended? Maximum possible duration of the contract including renewals (extensions)? 42. Maximum amount of contract award for the base period? Total possible contract award covering the base period and all extensions? 43. For the example performance-based contract, please list the key measures, give a brief description of the measurement procedure, and identify the desired performance target: Target Measure Measurement Procedure (& unit of measure) a. b. c. d. e. f. g. h. i. j. k. l. m. n. o. p. q. r. s. 44. Please explain how the performance targets or standards were set for the example contract. Check each that applies: a. We use a Maintenance Quality Assurance process and have followed the generally accepted practice of establishing a desired Level of Service of 80 for each measure b. Key maintenance managers from throughout the department recommend to top managers what the performance targets or standards should be c. Top management establish what the performance targets or standards should be based primarily on strategic goals, business objectives, and political considerations d. Other:

OCR for page 88
84 45. In regards to the example contract, please rate your agreement or disagreement with the following statements on a 1 to 5 scale (1 = do not agree; 5 = fully agree): a. The measurement procedures are repeatable: Rating: b. We insist that measurement instruments are calibrated where applicable and they are in good working order. Rating: c. Where industry measurement protocols exist, we rely on them for specific types of measurements. Rating: d. Our measurement procedures have a relatively high statistical accuracy and confidence intervals. Rating: e. We use measurement procedures that are commonly recognized within the industry even though no formal set of protocols exist. Rating: f. Other: Rating: 46. For the example contract, which of the following monitors the contractor's performance? Check each that applies: a. Agency staff b. Contractor monitors its own performance and provides periodic reports to our agency c. An independent third party d. Other: 47. What incentives and disincentives are included in the performance-based contract? Check each that applies: a. This PBC only has disincentives for failure to comply with specific contractual provisions and failure to meet performance targets or standards. b. This PBC uses only incentives--positive financial rewards--for achieving or exceeding performance targets or standards. c This PBC uses a combination of positive and negative financial incentives to ensure that the contractor is conforming to contract provisions and/or exceeding performance targets or standards. d. The incentives/disincentives are based entirely on measurable outcomes and outputs. e. The incentives/disincentives relate to contractual provisions but not to measurable outcomes and outputs. f. Our performance measures are all expressed in units of time, such as response time or the period of time in which work must be done. g. The results of the performance measures are used to affect the contractor's ability to secure future work with the agency. h. Other:

OCR for page 88
85 48. OPTIONAL. For the performance-based contract you selected, please list the main steps of the business process your agency used to procure a contractor, award a contract, monitor contract performance, and pay a contractor. If you have a flow chart of your performance-based maintenance contracting business process, you can e-mail that instead to Bill Hyman or send it by regular mail to him at the address at the beginning of this questionnaire. Step Description of Action (Indicate if Decision Is Involved) Who Does It? 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35

OCR for page 88
86 Part VII. Lessons Learned from PBC Please list the three most important lessons your agency has learned regarding PBC that might be of interest to other states. Lesson 1: Lesson 2: Lesson 3: Part VIII. Contractual Provisions Please describe three contractual provisions in your performance-based contracts that have influenced the success, strengths, shortcomings, or failures of your PBC efforts: Contract Provision 1: This provision contributed to (check one): Our success Strengths Shortcomings Failure Contract Provision 2: This provision contributed to (check one): Our success Strengths Shortcomings Failure Contract Provision 3: This provision contributed to (check one): Our success Strengths Shortcomings Failure

OCR for page 88
87 Part IX. Agency Perception of Contractor and Feedback with Regards to Service and Price 49. Have you found any ways in which by better serving the contractors' needs, your agency has received better levels of service or a reduction in the cost of providing service? Yes No If the answer is yes, please explain: Part X. Additional Issues Please answer "yes" or "no" regarding these additional questions about PBC: 50. Do you include methods-based and technical specifications along with your desired performance standards/targets in your PBC? Yes No 51. Have you found that PBC fosters creativity and innovation on the part of the contractor(s) because they are generally free to achieve the performance targets or standards in any manner they choose? Yes No 52. Does your agency try to shift to the contractor most or all the responsibility for repairing damage caused by motor vehicle accidents and recovering repair costs? Yes No 53. Does your agency expect the contractor to assume most or all the risks for material quantity fluctuations (e.g., sand and salt)? Yes No 54. Does your agency expect the contractor to assume most or all of the risks associated with price increases for high- priced items? Yes No 55. Have you found that PBC has allowed you to more easily internalize different phases of a job under one contract (e.g., build, operate, and maintain)? Yes No 56. Has PBC simplified contract administration? Yes No 57. Has PBC resulted in documented cost savings? Yes No If yes, what is the average percentage cost savings per contract? 58. Do you require your PBC contractors to have a quality assurance plan? Yes No 59. Did you have to obtain special legal authority to do PBC? Yes No 60. Do you have a legislative provision for public­private partnerships that allows a private entity to propose a performance-based maintenance contract on part or all of the state highway network? Yes No

OCR for page 88
88 Part XI. Samples of Performance-Based Contracts When you return the completed survey, please e-mail to Bill Hyman sample Performance-Based Contract(s) that are typical of the different types you have used in your state. If you do not have copies in electronic form, please mail them to Bill Hyman at his address shown at the beginning of this questionnaire. THANK YOU FOR YOUR HELP AND COOPERATION Please return the completed questionnaire by April 24, 2006, to William Hyman at bhyman@ara.com

OCR for page 88
89 CONTRACTOR QUESTIONNAIRE NCHRP TOPIC 37-09 PERFORMANCE-BASED CONTRACTING FOR MAINTENANCE Background and Purpose State and provincial departments of transportation (DOTs) are confronted with a combination of growing needs and resource limitations for maintaining the highway system. This has intensified their interest in contracting for maintenance services. The transportation agencies have developed various performance-based contracting methods, including the means to measure and report on performance. NCHRP has engaged Applied Research Associates, Inc. (ARA) to conduct a synthesis of practice to obtain information on performance-based maintenance contracting. The synthesis will explore the following topics: x Performance standards and relevant measures commonly used to address performance-based contract delivery for different types of maintenance activities x How performance levels are established, measured, and evaluated in maintenance contracting x Identified best practices in monitoring and reporting performance-based contract maintenance delivery x Reported costs, benefits, risks, and possible shortcomings of adopting performance-based maintenance contracts x Basic elements necessary to initiate performance-based maintenance delivery, including contractor acquisition strategy, evaluation criteria for selection, and prequalification process x Contractual provisions, such as payment methods, including incentives/disincentives x Examples of successful and failed applications in performance-based maintenance contracting A key source of information for this synthesis is a survey of maintenance contractors that have considerable experience in performance-based contracting. Please complete survey on the following pages and return it by e-mail to the principal investigator, William Hyman, at bhyman@ara.com. Thank you very much for your time and effort in completing this questionnaire! NOTE: ALL ANSWERS TO THIS QUESTIONNAIRE WILL BE TREATED AS PROPRIETARY AND WILL BE LUMPED WITH OTHER RESPONSES AND NOT IDENTIFIED WITH A FIRM. Part I. Respondent Information Position in Maintenance Contracting Firm Name: Title: Organization: Address: City: State: Zip: Phone: Fax: E-mail:

OCR for page 88
90 Please return the completed questionnaire by April 23, 2006, to: William Hyman Principal Investigator Applied Research Associates, Inc. 7184 Troy Hill Drive, Suite N Elkridge, MD 21075 E-mail: bhyman@ara.com Phone: 410-540-9949 Cell: 301-593-7842 Fax: 410-540-9288 Part II. Acquisition Phase and Partnering In responding to the following questions, please indicate the percent of time each of the following occurs in regards to procurements for performance-based contracts for all maintenance organizations your firm deals with: 1. During the procurement phase for a PBC, what percent of time does the procurement agency identify potential bidders and their capabilities? Percent of time: 2. What percent of time do procurement agencies interview industry leaders in the private sector to obtain their input regarding a forthcoming PBC procurement? Percent of time: 3. What percent of the time do maintenance organizations issue Requests for Information to learn more about what a PBC entails from the private sector point of view? Percent of time: 4. What percent of time do maintenance organizations issue a Request for Qualifications or Letter of Interest with supporting qualifications before issuing an RFP? Percent of time: 5. What percent of time do maintenance organizations issue an announcement of forthcoming procurements regarding RFPs and allow questions and answers prior to the issuance of an RFP? Percent of time: 6. What percent of time do maintenance organizations hold one or more preproposal conferences and respond to both oral and written questions? Percent of time: 7. If most of the time maintenance organizations hold a preproposal conference, are they mandatory? Yes No Sometimes 8. What percent of time do maintenance organizations allow potential contractors to have individual discussions with a PBC technical program manager in advance of an RFP being issued? Percent of time: 9. What percent of time do maintenance organizations provide baseline inventory and condition data to potential bidders either in advance of issuing the procurement or as a part of the RFP? Percent of time: 10. What percent of time do maintenance organizations allow potential bidders to inspect the facilities they may be responsible for maintaining to determine the accuracy of the baseline inventory and condition data and estimate the cost of doing different types of maintenance work on the roads that will be covered by the PBC? Percent of time: 11. What percent of time do maintenance organizations seek agreement with bidders that the performance measures that will be used are practical to apply and will yield measurements consistent with the project objectives the agency is trying to achieve? Percent of time: 12. What percent of the time do maintenance organizations seek agreement with potential bidders that the incentives and penalties incorporated in the PBC are reasonable? Percent of time:

OCR for page 88
91 13. What percent of time do maintenance organizations have prospective bidders review and comment on the draft RFP? Percent of time: 14. What percent of the time do maintenance organizations seek to develop a strong partnership with private contractors and view this partnership as an essential part of the approach to PBC? Percent of time: 15. What percent of the time do maintenance organizations believe they have little need for input from potential bidders during the acquisition process and the preparation of an RFP? Percent of time: Part III. Business Issues Provide a rating from 1 (disagree) to 5 (agree) on the following questions: 16. The rewards exceed the risks on most performance-based contracts? Rating: 17. It is hard to live within the cost constraints of lump-sum contracts. Rating: 18. Lump-sum contracts provide the opportunity to earn larger profits than normal. Rating: 19. Too much of the risk is borne by the contractor. Rating: Comment: 20. Incentives in performance-based contracts are all negative. Rating: Comment: 21. The prospect of being able to renew a performance-based contract is a major incentive. Rating: 22. Do you have the means to control the following costs adequately? We realize you may be unwilling to explain your answers, because your reasons are considered proprietary. a. Subcontractor costs? Yes No If you are willing, explain: b. Administrative costs? Yes No If you are willing, explain : c. Equipment costs? Yes No If you are willing, explain: d. Material costs? Yes No If you are willing, explain: e. Your firm's labor costs? Yes No If you are willing, explain: f. Routine maintenance costs? Yes No If you are willing, explain: g. Preventive maintenance costs? Yes No If you are willing, explain: h. Demand responsive maintenance costs? Yes No If you are willing, explain:

OCR for page 88
92 i. Rehabilitation costs? Yes No If you are willing, explain: j. Life-cycle costs? Yes No If you are willing, explain: k. Traffic control costs? Yes No If you are willing, explain: l. Emergency operations costs? Yes No If you are willing, explain: m. Customer interaction costs? Yes No If you are willing, explain: n. Client interaction costs? Yes No If you are willing, explain: 23. Do you have a maintenance management system? Yes No 24. A performance measurement system? Yes No 25. Do you have insurance or the equivalent to address unusual costs caused by weather? Yes No 26. Are you typically required to indemnify the public agency against all damage or costs caused by you as a contractor? Yes No If yes, do you see indemnification as part of the cost of doing business and an acceptable risk? Yes No 27. How many years is the shortest period of performance of your performance-based contracts, including all option renewals that do not require competition? Years 28. How many years is the average period of performance of your performance-based contracts, including all option renewals that do not require competition? Years 29. How many years is the longest period of performance of your performance-based contracts, including all option renewals that do not require competition? Years 30. How long do you think the period of performance for large-scale performance-based contracts should be? Years Explain: Part IV. Use of the Maintenance Quality Assurance Process and Benchmarking A previous study, NCHRP 14-15, Maintenance Quality Assurance, defined a performance measurement process that involves taking a random sample of highway sections, and periodically measuring the Levels of Service that result from different maintenance activities. Levels of Service refers to either the condition of an asset or the service level that is achieved, for example, roughness of pavement or grass height achieved by mowing. Typically, LOS for each type of asset and service is scored on a scale of 0 (worst possible) to 100 (best possible) and the results weighted to achieve a composite LOS score, also on a scale of 0 to 100. 31. What percent of the time do maintenance organizations use a Maintenance Quality Assurance process to measure outputs and outcomes of contract maintenance? Percent of time: 32. What percent of time do maintenance organizations use PBC consisting of a Maintenance Quality Assurance process coupled with incentives/penalties? Percent of time:

OCR for page 88
93 NCHRP Project 14-13 resulted in a guide on customer-driven benchmarking of maintenance activities. Benchmarking is defined as using outcome, output, and input measures along with factors outside the agency's control to identify best performers and their corresponding business processes and work methods. These are best practices. Customer-driven benchmarking focuses on customer-driven outcomes. 33. Do you use benchmarking to compare the maintenance performance of your firm with other private firms? Yes No 34. Do you use benchmarking to compare the performance of your firm with the maintenance performance of state or provincial DOTs? Yes No Part V. An Example of a Performance-Based Contract In this section, you are requested to provide information on a performance-based contract that you think would be interesting and provide valuable lessons to states, Canadian provinces, or other contractors. Skip this section if you have no example to provide. 35. What is the name and location of the PBC? 36. Please describe the period of performance, scope of work, the general nature of the activities involved, and whether incentives apply to contractual provisions and/or the achievement of targets for measurable outputs and/or outcomes. 37. Please describe what you consider the best feature(s) of this performance-based contract: 38. Please describe what you consider the worst feature(s) of this performance-based contract:

OCR for page 88
94 39. Please list the key measures in the example performance-based contract, give a brief description of the measurement procedure, and identify the desired performance target, including the unit of measure. You can forward a copy of the maintenance contract instead to Bill Hyman by e-mail, bhyman@ara.com, or to the address at the beginning of the questionnaire: Target Measures Measurement Procedures (and units of measure) a. b. c. d. e. f. g. h. i. j. k. l. m. n. o. 40. Which of the following monitors the contractor's performance in the example performance-based contract? Check each that applies: a. Agency staff b. Contractor monitors its own performance and provides periodic reports to the contracting agency c. An independent third party e. Other: 41. Please rate your agreement or disagreement with the following statements on a scale of 1 to 5 (1 = do not agree; 5 = fully agree) in regards to the example contract: a. The measurement procedures are repeatable. Rating: b. The measurement instruments are calibrated where applicable and are in good working order. Rating: c. We rely on industry measurement protocols for specific types of measurements. Rating: d. Our measurement procedures have a relatively high statistical accuracy and confidence intervals. Rating:

OCR for page 88
95 e. We use measurement procedures that are commonly recognized within the industry even though no formal set of protocols exist. Rating: f. Other: Rating: 42. What incentives and penalties are included in the example performance-based contract? Check each that applies: a. This PBC has penalties only for failure to comply with specific contractual provisions and failure to meet performance targets or standards. b. This PBC uses only positive financial rewards for achieving or exceeding performance targets or standards. c. This PBC uses a combination of positive and negative financial incentives to ensure that the contractor is conforming to contract provisions and/or exceeding performance targets or standards. d. The incentives/penalties are based entirely on measurable outcomes and outputs. e. The incentives/penalties relate to contractual provisions but not to measurable outcomes and outputs. f. All performance measures are expressed in units of time, such as response time or the period of time in which work must be done. g. Other: Part VI. Lessons Learned from PBC Please list the three most important lessons your firm has learned regarding PBC that might be of interest to states, Canadian provinces, and other contractors. Lesson 1: Lesson 2: Lesson 3: Part VII. Contractual Provisions Please describe three contractual provisions in your performance-based contracts that have influenced the success, strengths, shortcomings, or failures of your PBC efforts: Contract Provision 1: This provision contributed to (check one): Our success Strengths Shortcomings Failure. Please describe the provision:

OCR for page 88
96 Contract Provision 2: This provision contributed to (check one): Our success Strengths Shortcomings Failure. Please describe the provision: Contract Provision 3: This provision contributed to (check one): Our success Strengths Shortcomings Failure. Please describe the provision: Part VIII. Additional Issues Please answer "yes" or "no" to the following issues concerning PBC: 43. Do you observe that methods-based and technical specifications are included along with performance specifications and targets in your PBC? Yes No 44. Have you found that PBC fosters creativity and innovation on the part of your firm because your firm is free to achieve the performance targets or standards in any manner it chooses? Yes No 45. Have you found that PBC has allowed your firm to more easily internalize different phases of a job under one contract (e.g., build, operate, and maintain)? Yes No 46. Do you observe that PBC simplifies contract administration for the contracting agency? Yes No 47. Do you observe that PBC has simplified contract administration for your firm? Yes No 48. Has BPC resulted in documented cost savings to your clients? Yes No If yes, what is the average percentage client cost savings per contract? Please return documentation of cost savings by e-mail or mail to Bill Hyman at the points of contact at the beginning of this questionnaire. 49. Have contracting agencies required you to have a quality assurance plan? Yes No 50. Have you ever taken advantage of a legislative provision for public­private partnerships that allows a private entity to propose a performance-based maintenance contract on part or all of a state or provincial highway network? Yes No Part IX. Samples of Performance-Based Contracts When you return the completed survey, please enclose sample performance-based contract(s) that are typical of the different types your firm has been engaged in. If you do not have copies in electronic form, please mail them to Bill Hyman at his address shown at the beginning of this questionnaire. THANK YOU FOR YOUR HELP AND COOPERATION Please return the completed questionnaire by April 23, 2006, to William Hyman at bhyman@ara.com.