National Academies Press: OpenBook

Performance Metrics for Public–Private Partnerships (2021)

Chapter: Chapter 1 - Introduction

« Previous: Summary
Page 3
Suggested Citation:"Chapter 1 - Introduction." National Academies of Sciences, Engineering, and Medicine. 2021. Performance Metrics for Public–Private Partnerships. Washington, DC: The National Academies Press. doi: 10.17226/26171.
×
Page 3
Page 4
Suggested Citation:"Chapter 1 - Introduction." National Academies of Sciences, Engineering, and Medicine. 2021. Performance Metrics for Public–Private Partnerships. Washington, DC: The National Academies Press. doi: 10.17226/26171.
×
Page 4
Page 5
Suggested Citation:"Chapter 1 - Introduction." National Academies of Sciences, Engineering, and Medicine. 2021. Performance Metrics for Public–Private Partnerships. Washington, DC: The National Academies Press. doi: 10.17226/26171.
×
Page 5
Page 6
Suggested Citation:"Chapter 1 - Introduction." National Academies of Sciences, Engineering, and Medicine. 2021. Performance Metrics for Public–Private Partnerships. Washington, DC: The National Academies Press. doi: 10.17226/26171.
×
Page 6
Page 7
Suggested Citation:"Chapter 1 - Introduction." National Academies of Sciences, Engineering, and Medicine. 2021. Performance Metrics for Public–Private Partnerships. Washington, DC: The National Academies Press. doi: 10.17226/26171.
×
Page 7
Page 8
Suggested Citation:"Chapter 1 - Introduction." National Academies of Sciences, Engineering, and Medicine. 2021. Performance Metrics for Public–Private Partnerships. Washington, DC: The National Academies Press. doi: 10.17226/26171.
×
Page 8
Page 9
Suggested Citation:"Chapter 1 - Introduction." National Academies of Sciences, Engineering, and Medicine. 2021. Performance Metrics for Public–Private Partnerships. Washington, DC: The National Academies Press. doi: 10.17226/26171.
×
Page 9

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

3   MAP-21, the Moving Ahead for Progress in the 21st Century Act (P.L. 112-141) of 2012, contained a mandate to develop performance measures for most transportation project types. This eventually evolved into a set of “Final Rules” for National Performance Management Measures established by the Federal Highway Administration (FHWA) that govern the pro- gram to monitor the health of the nation’s highway networks. These rules are focused on the operations and maintenance (O&M) phase of a transportation asset’s life cycle and provide a basis from which state departments of transportation (DOTs) develop specific performance measures for the post-construction phase of public–private partnership (P3) contracts. As such, the focus of this synthesis is the state-of-the-practice P3 performance metrics that are applied after construction is complete, and the asset is being employed for its intended function. Much has been written about the procurement, design, and construction phases of P3 project delivery (DeCorla-Souza 2018; VanMeter 2018; Ramsey and El Asmar 2020); however, due to the fact that P3 concessions (also called development agreements) are often decades long and P3 delivery is relatively new to the United States, almost no research has been conducted on the performance of P3 projects throughout their complete life cycle. A study by Cui and Ham (2017) reviewed P3 highway projects, finding only three that had completed their concession periods. Unfortunately, those were international projects. Hence, published knowledge regarding the effectiveness of U.S. P3 project performance metrics is lacking. The one feature that distinguishes most P3 contracts from other forms of transportation asset procurement is the true life-cycle nature of the projects. In traditional Design-Bid-Build (DBB) and alternative project delivery methods, the DOT assumes control over the asset once the con- struction is accepted and bears the O&M risk for the asset’s service life. Hence, if the constructed level of quality is less than intended or expected, the agency will bear the consequences of the asset failing to achieve its expected design life. The project’s quality is defined during the design and construction processes, with a general focus on minimizing the capital construction cost, and the project is generally awarded to the lowest bidder. Hence, there is little incentive to design or build a product that exceeds the minimum acceptable level of quality in traditional project delivery. Alternative delivery methods generally provide for best value awards. Still, research has shown that in most cases the lowest bidder wins because the agency often weights the award process in a manner that makes it difficult for competitors to chance submitting a price proposal that has not been minimized (Gransberg and Barton 2007; Molenaar et al. 2015; Gad et al. 2015). With a P3, however, the O&M risk can be transferred to the Concessionaire that designed and built the project, which shifts the “minimize construction cost paradigm” to one that seeks to optimize capital costs with life-cycle costs (Barutha 2016). This shift affects the decisions made during design from merely the least-cost solution to the solution that provides enhanced life-cycle cost certainty. An excellent example of how this risk transfer works is found in the Pennsylvania DOT’s (PennDOT) P3 Rapid Bridge Replacement Program (further details are C H A P T E R 1 Introduction

4 Performance Metrics for Public–Private Partnerships in Chapter 4). The Concessionaire chose to increase the bridge design criteria above the min- imum mandated by the agency, citing long-term cost certainty as the primary motivation for the design decision (Barutha 2016). Measuring potential benefits like this one is essentially the focus of this synthesis. It is important not only for agencies to be compliant with the law regarding federal aid highway funding (if federal funding is utilized) but, more specifically, to make a well- documented business case for the future use of P3 project delivery to legislatures, the traveling public, and agency upper management. Background There is some debate in the industry as to the definition of a P3. Contract and statutory struc- tures, including Design-Build-Finance (DBF) and Design-Build-Finance-Operate-Maintain (DBFOM), as well as other variations, are often cited within these varying definitions. Regardless, for the purposes of this discussion, a P3 is defined as a contract structure where private entities, commonly referred to as either “Concessionaires” or “Developers,” contract with public owners to design, build, finance, and, if applicable, operate and maintain projects. The classic DBFOM structure typically includes a Design-Build (DB) contract, a long-term O&M contract, financ- ing elements, and other project-specific aspects. Figure 1 shows the contractual relationships between the public project sponsor, the Concessionaire, and the key project financial elements. The financing elements and resultant long-term obligations of a P3 concession bring unique challenges and opportunities to project delivery. P3 contracting brings into the process very different financial, legal, and commercial considerations. Risk Transfer These projects tend to be large, complex, and schedule driven. They also provide a means for significant risk transfer to the private sector—both financial and technical. The long-term OWNER DESIGNER-OF- RECORD GENERAL CONTRACTOR Design Subconsultants Trade Subcontractors DESIGN- BUILDER OPERATIONS & MAINTENANCE CONTRACTOR Developer/ ConcessionaireBanks/Lenders Project Sponsors/ Equity Providers $ Return on Equity $ Debt Service (Only If DBFOM) Figure 1. The P3 model.

Introduction 5   obligations on the part of the Concessionaire are a relatively dramatic departure for public agencies from the traditional public contracting models. For example, using alternative tech- nical concepts (ATC) in conjunction with long-term performance requirements and perfor- mance penalty systems are key elements of the P3 process and very different from other public owner procurement approaches. The risk transfer that is a key part of these contracts places many non-traditional risks on the private sector that typically and historically have been retained by public owners. This risk transfer brings both challenges and opportunities for both parties. Ultimately, the Concessionaire/ Developer is at risk for final capital costs, long-term maintenance and operational costs, and schedule performance. The schedule performance includes ensuring completion of the Design- Build (DB) portion of the contract while also ensuring that upon opening the project provides the required “availability” for the public sector, with potentially severe financial repercussions for failure. Key Definitions Before continuing, it is important that several key terms be defined to ensure the reader’s understanding of the remainder of the document. A glossary containing the full set of defini- tions for terms used in the synthesis is provided at the end of the report. The following defini- tions come from FHWA’s P3 Procurement Guide (Smith et al. 2019). • Concession: a contractual right granted by an agency to a developer to design, build, finance, operate, and maintain a particular asset owned by the agency, as documented and governed by the terms of a concession agreement • Concession period: the duration of a concession, commencing upon either execution of the agreement or completion of the project and ending upon handback of the facility to the public agency • Concessionaire: the private sector party to a P3 agreement, also called a developer or the “private partner” • Equity investors: the entities that directly or indirectly own the Developer and invest equity into the project, also known as “sponsors” or “shareholders” • Handback criteria: Project condition criteria used at the conclusion of the P3 agreement to transfer the facility back to the agency • Performance metrics: metrics used to measure the behavior, activities, and performance of a project; this could be in the form of data that measure required data within a range, allow- ing a basis to be formed supporting the achievement of overall project performance goals/ objectives. These fundamental definitions need to be understood in terms of the procurement/ compensation approach from any concession agreement: • Availability payments: a contract approach where the Concessionaire/Developer will be reimbursed for project costs by the public owner, with periodic payments over the con- tract term • Revenue risk: the Project Agreement assigns the risk for actual revenues generated through tolling or other revenue streams on the Concessionaire/Developer • Hybrid P3: any P3 financing structure with both an availability payment and a revenue risk component that is payable over the term of the contract Variations There are several common variations available to agencies interested in P3 delivery. Table 1 contains a list of the most common variations with a definition for each. Chapter 2 will further

6 Performance Metrics for Public–Private Partnerships refine the discussion of P3 types with regard to classifications by risk transfer, payment method, and project scope (DeCorla-Souza 2013). Thus, the potential variations on the P3 theme can be numerous and provide the agency with a great deal of flexibility in using this project delivery method. P3 Performance Metrics According to the FHWA’s P3 Procurement Guide (Smith et al. 2019): the key advantage of a P3 procurement structure is that it enables the agency to focus on desired performance outcomes of a facility to maximize public benefit while letting the private entity determine the most efficient means to achieve those outcomes. Contractually, this is made possible through per- formance requirements. Performance requirements detail the expectations for facility performance and define what is needed to accomplish the project’s objectives. By establishing requirements based on the performance of the facility or the project, the agency transfers risks to the Concessionaire while providing measurable metrics to gauge performance. A typical concession agreement will have a large number of perfor- mance metrics. Each of those descends from a critical analysis of the performance requirements developed by the agency during procure- ment. Figure 2 is a model for how those are derived. In this figure, the column listed as performance criteria contains several typical per- formance metrics that can be used to measure the post-construction performance of the highway’s pavement. Those metrics drive the Concessionaire to make appropriate design decisions during the DB  phase of the project to ensure that life-cycle performance of the constructed product will comply. The metrics are, in turn, used by the agency to monitor actual performance, and the metric measurement process becomes critical because these metrics can impact a Concessionaire, as they are normally tied to the periodic payment process on availability payment projects and can result in penalties of various kinds on revenue risk Variation Characteristics Example Project Design-Build-Finance (DBF) A design-build contract combined with short-term private sector financing of the design and construction. The public owner retains responsibility for long-term O&M. If desired, the financing can be structured so that the debt providers do not assume construction risk. Northwest Corridor Express Lanes, Georgia DOT Design-Build-Finance- Operate (DBFO) Design, construction, financing, and operations responsibilities are transferred to a single private sector partner. The public owner retains responsibility for long- term maintenance. Foley Beach Express, City of Foley, Alabama Design-Build-Finance- Maintain (DBFM) Design, construction, financing, and maintenance responsibilities are transferred to a single private sector partner. The public owner retains responsibility for long- term operations. Gordie Howe International Bridge, Windsor-Detroit Bridge Authority Design-Build-Finance- Operate-Maintain (DBFOM) Design, construction, financing, operations, and maintenance responsibilities are transferred to a single private sector partner. I-77 Express Lanes, North Carolina DOT Design-Build-Operate- Maintain (DBOM) Similar to DBFOM but with no private financing. DBOM is included, as it requires the same post-construction project performance metrics. Tren Urbano, Puerto Rico Departamento de Transportacion y Obras Publicas (DTOP) Leasing arrangements (also termed asset recycling or asset monetization) A public asset is leased to a private sector partner, which then operates, maintains, and collects revenue during the concession period. Chicago Skyway – City of Chicago, Illinois Table 1. Common P3 variations. Impact of P3 Performance Metrics on Design Decisions The Pennsylvania DOT’s Rapid Bridge Replacement Program replaced 558 low- volume bridges, and its solicitation required the concessionaire to use 50-year bridge design criteria. However, the use of the National Bridge Inspection System criteria to measure post- construction performance drove the Concessionaire to choose to design to 100-year criteria as a means of enhancing life-cycle cost certainty during the O&M period. —Barutha 2016

Introduction 7   projects. As a result, P3 performance metrics must be completely clear and unambiguous to be effective. Many DOTs have turned to a point-based scoring system for routine monitoring of the Concessionaire’s performance during O&M. The system involves the agency inspecting the facility on a regular basis, noting the deficiencies, and notifying the Concessionaire. The Con- cessionaire then has a specified cure period to correct the deficiencies. If it fails to complete the necessary corrective action, the agency then assesses a predetermined number of points based on the classification of the violation. On availability contracts, for each pay period, there is a reconciliation of the deficiencies reported and corrected. The resulting point total then serves as input to a formula that calculates what percentage of the scheduled amount for that month will be paid. Table 2 is an excerpt from the Florida DOT’s (FDOT) I-595 Corridor Improve- ments P3 concession agreement and is provided as a typical example of this concept. Synthesis Objectives The objective of this synthesis was to document key performance metrics used in various long-term DOT P3 contracts for the delivery of highway projects and services. The synthesis will focus on the post-construction stage of P3 contracts. The study is intended to showcase success- ful applications, lessons learned, and a qualitative assessment of the effectiveness of P3 perfor- mance metrics based on survey and interview data. Synthesis Scope and Approach The synthesis scope is limited to documenting the state of the practice in DOT application of P3 performance metrics. The scope includes, but is not limited to, the following: • Documenting key performance metrics • Focusing on the post-construction stage of the P3 project Reduce Whole Life Cost Functionally and Structurally Adequate Pavement Adequate Pavement Structure Smooth and Safe Riding Surface Fatigue Cracking <10% Total Rutting < 0.5 Inches IRI < 120 inches/mile Pavement Thickness HMA Compaction HMA Composition Quality of HMA Mix Asphalt Binder Grade Base Thickness Quality of Base Materials Subgrade Strength WorkmanshipHMA Rutting < 0.25 Inches User Requirement Essential Function Performance Requirements Performance Criteria Design Decisions What is needed? How to achieve? Figure 2. Model for developing performance metrics from performance requirements (adapted from Smith et al. 2019).

8 Performance Metrics for Public–Private Partnerships • Synthesizing agency P3 policies and procedures for P3 performance measurement • Presenting measurement and payment procedures for P3 post-construction services • Presenting enforcement measures • Presenting incentive/disincentive procedures The approach for collecting the necessary information to cover the scope included a review of the literature, a content analysis of agency P3 documentation, a survey of DOTs known to have P3 experience, and interviews of DOT and Concessionaires on six “typical” P3 projects. A simple, multiple choice, web-based questionnaire was developed and deployed to obtain information related to agency P3 procedures and the metrics used to measure P3 project per- formance. The proposed questionnaire was first submitted to NCHRP for review and comment. Once approved by NCHRP, a web-based survey tool was used to develop the questionnaire, and the link was distributed directly to the participating agencies. A list of P3 points of contact was developed by the consultants based on information gained in a recently completed FHWA proj- ect involving P3s (FHWA 2019a). The questionnaire was emailed directly to the targeted DOT’s P3 points of contact. The FHWA project showed that 26 DOTs have the authority to use P3s in some form. Responses were received from 20 DOTs, yielding a response rate of 77%. Appen- dix B contains the details of the survey along with Figure 19, which is a map of survey responses. Agencies that indicated a willingness to share their P3 experience and that use P3 metrics were contacted for follow-up interviews to gain a greater depth of understanding of the informa- tion provided in their web-based survey response. One of the survey questions asked respon- dents to submit agency information related to P3 practices, specifications, reports, etc. These data were used to conduct a P3 performance metric content analysis that is detailed in Chapter 3. The information extracted from the documents includes • Warranty provisions. • Performance indicators, thresholds, and remedial action. • Maintenance requirements. • Quality control, inspection, and acceptance. • Method of measurement. • Basis of payment (if applicable). • Conflict resolution. Element Category Required Task Minimum Performance Requirements O&M Violation Classification Cure Period Category 1 Pavement (post-3 years after substantial completion) Maintain flexible pavement at acceptable level of safety for traveling public. Meet the performance requirements set forth in Division II, Section 6 of the Technical Requirements for the following: - - Rutting to be maintained less than a depth of 0.375 in. B 180 Days Ride to be maintained at ride number (RN) greater than 3.5. B 180 Days Settlement/depression a maximum depth of 0.5 in. C 7 Days Cracking for a pavement condition rating above 7.0. B 180 Days Raveling and/or delamination of the Friction Course as defined and determined by the Department in accordance with the examples displayed at: www.dot.state.fl.us/specificationsoffice/pavement.htm or its successor. B 180 Days Pot holes and slippage area(s) cannot be greater than 0.5 square feet in area and 1.5 in. deep. C 24 Hours Source: FDOT 2009. Table 2. Example I-595 Corridor Improvement P3 O&M performance requirements.

Introduction 9   The results of the questionnaire and follow-up questions are summarized in graphs and tables in Chapter 3 and provide a summary of the state of the practice for P3 performance measure- ment. Responses from the questionnaire, follow-up questions, and the literature search were used to identify six case examples to demonstrate the diversity of procedures, practices, and specifications for P3 performance metric approaches. The case example projects are detailed in Chapter 4. Organization of the Report The contents of the synthesis report are organized as follows. Chapters 1 through 5 are fol- lowed by references. There is a glossary and then five appendices. Appendix A is a literature review, Appendix B presents the survey questionnaire, Appendix C is a concession agreement performance metrics content analysis, Appendix D presents a content analysis of handback criteria, and Appendix E offers proposed research needs statements.

Next: Chapter 2 - Literature Review »
Performance Metrics for Public–Private Partnerships Get This Book
×
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Public–private partnerships (P3) allow public transportation agencies to attract private financing to deliver needed highway infrastructure and not have to wait until the required funding is fully in place via traditional state and federal sources.

The TRB National Cooperative Highway Research Program'sNCHRP Synthesis 563: Performance Metrics for Public–Private Partnerships documents key performance metrics used in various long-term P3 contracts for the delivery of highway projects, including services by Departments of Transportation (DOTs).

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!