Value, Benefits, and Limitations of Qualifications-Based Selection for Airport Project Delivery (2019)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

12 This chapter benchmarks the state of the practice and reviews findings as they relate to policies, approaches, and guidelines for qualifications-based selection (QBS) awards for airport projects. The chapter combines information collected through a comprehensive literature review of policies, procedures, guidelines, and decision processes of primary delivery methods and variations on those ACMs. This chapter establishes background and context for the descriptions of current QBS practices projects in Chapters 3 and 4. The details of the commonly available sources of airport project funding are provided in Appendix B. Each source imposes conditions and constraints on the expenditure of these funds. To explain these constraints, a brief review of relevant procurement laws will be helpful. The 1972 Brooks Act (Public Law 92-582, October 27, 1972) mandated the QBS procurement process for the selection of architect/engineers (A/Es) for design contracts with federal agencies. Essentially, the spirit of the law requires the government to select the most qualified design professionals for federally funded projects and prohibits the use of price as a selection factor. While Public Law 92-582 does not directly apply to federal assistance programs, the enabling legislation for various federal assistance programs such as the Airport Improvement Program (AIP), has extended QBS to recipient procurements as a condition of eligibility. According to Public Law 92-582, the agencies procure “A/E services” by first announcing the requirements for such services and then negotiating contracts for architectural and design services based on the qualifications of the consultant and “reasonable prices.” The agency will typically select at least three firms deemed to be qualified under the criteria published in the solicitation and negotiate a contract with the highest qualified firm at compensation rates deemed to be fair and reasonable by the agency head. This process follows the RFQ/RFP process where the agency reviews contending proposals by qualified firms. If the agency cannot come to an agreement on compensation with the most qualified firm, they commence negotiations with the next qualified contender. Procurement Methods in Federally Funded Airport Projects The AIP provides a major source of federal funding for airport projects. Requirements, restrictions, and details of projects eligible for AIP funding are documented in the AIP Hand- book, published under Order 5100.38. The handbook discusses two types of procurement for construction: (1) sealed bid and (2) competitive proposals. In the sealed bid procurement, the contractor is selected based on the lowest price. In the competitive proposal procurement C H A P T E R 2 Overview of Qualifications-Based Selection The lowest contemporaneous price cannot be guaranteed to yield the overall lowest project cost after execution. Lingard et al. 1998

Overview of Qualifications-Based Selection 13 approach, the award is based on price and/or other factors, such as qualifications, contract duration, and method of construction (2 CFR § 200.320). While competitive procurements may be conducted using any project delivery method, QBS with a negotiated price can only be used in procuring professional services. According to 49 U.S.C. § 47107(a)(17), AIP-funded projects must use the QBS method under the Brooks Act for “program management, construction management, planning studies, feasibility studies, architectural services, preliminary engineering, design engineering, surveying, mapping, and related services.” The competitors do not submit any pricing information before the airport agency determines the most qualified competitor. DB is established under 49 U.S.C. § 47142 as an approved project delivery method under AIP and requires that “three or more bids must be submitted.” The AIP Handbook also allows the use of CMR, but the construction services still need to be procured competitively, where price needs to be a factor. “The CM-at-risk conducts document reviews, constructability reviews, cost estimating and scheduling. The CM-at-risk then competitively procures construction component of the project and is responsible for ensuring the project is completed within budget and schedule” (FAA 2014). The handbook further requires that two or more competitive proposals be received for all delivery methods except DB where the minimum number of proposals is three. The handbook goes on to state that Grantees and subgrantees may use competitive proposal procedures for qualifications-based pro- curement of architectural/engineering (A/E) professional services whereby competitors’ qualifications are evaluated, and the most qualified competitor is selected, subject to negotiation of fair and reasonable compensation. The method, where price is not used as a selection factor, can only be used in procurement of A/E professional services. It cannot be used to purchase other types of services though A/E firms are a potential source to perform the proposed effort. (FAA 2014, p. U-16) Implementing Alternative Contracting Methods for Airport Projects CMR, DB, and PDB contracts are concluded when construction is complete. Post-construction operation and maintenance scope is sometimes added to DB projects and called design–build– operate–maintain (DBOM). When private funding is brought to the project, a public–private partnership (PPP) contract is formed and the prime contractor, known as the concessionaire or developer, provides design, construction, and financing, as a minimum. PPP projects often have post-construction variations where the operations and/or maintenance of the constructed facility is included in the PPP contract. IDIQ contracts provide a mechanism to deliver more than one project (often termed a job or task order) using a single procurement transaction to advertise, evaluate, and award a contract. Once awarded, an IDIQ contract provides the capacity to deliver an indefinite quantity of services or products over the period of the contract. IDIQ procurement furnishes the airport with the flexibility to utilize the IDIQ contractor as much or as little as the require- ments of the agency and the availability of funds dictate. In the end, the airport is typically only required to give the IDIQ contractor enough work to satisfy the guaranteed minimum amount specified in the contract. Thus, if the contractor’s performance is not satisfactory, the airport is not obligated to issue any further task orders, which creates an operating incentive to provide timely delivery of quality construction (Farris 2002). Rueda (2013) found that IDIQ contracts have been structured to include design, construction, CMR, and/or DB delivery of the task orders that formalize each transaction. Therefore, IDIQ provides a framework in which other ACMs can be employed to deliver multiple projects with a single procurement transaction.

14 Value, Benefits, and Limitations of Qualifications-Based Selection for Airport Project Delivery Procurement Structural Frameworks Before getting into the details of airport project delivery, it is important to develop working definitions that differentiate between two common terms: project delivery method and procurement method. The following definitions are consistent with ACRP Report 21 (Touran et al. 2009a). Project Delivery Method Project delivery method refers to all contractual relations, roles, and responsibilities of the entities involved in a project. The Asso- ciated General Contractors of America (AGC) defines project delivery method as “the comprehensive process of assigning the contractual responsibilities for designing and constructing a project. A delivery method identifies the primary parties taking contractual responsibility for the performance of the work” (Kenig 2011). Thus, the different project delivery methods are distinguished by the way the contracts between the owner, the designer, and the builder are formed and the technical relationships that evolve between each party inside those contracts. The AGC holds that the four main types of project delivery methods are DBB, DB, CMR, and IPD (Kenig 2011). As previously stated, this report adds PPP and IDIQ to the four cited in the AGC document. Both PPP and IPD include all QBS-related elements of a typical DB contract. Additionally, both PPP and IPD are not widely used at present. PPP projects are often reserved for large, highly technical projects involving sophisticated systems like people movers, and IPD has primarily been used in private projects. Thus, the report will not provide specifics on these two methods. Procurement Method Procurement method is the process used by the owner to select the contractor and/or A/E. One TRB report discusses three types of procurement processes (Scott et al. 2006): • Low bid (LB): Contract is awarded on the basis of price alone. No other factors are considered. • Best value (BV): Contract is awarded on a combination of price and other key factors such as qualifications, schedule, technical approach, etc. • Qualifications-based selection (QBS): Contract is awarded on the basis of qualifications alone. Price is not considered. The current AGC guide to project delivery systems (Kenig 2011) considers four types of procurement methods by considering two types of BV selection processes in addition to low bid and QBS as described in the previous definition of BV. The contracting process used to get to a final award is usually one of the following: • One-step: Competitors are asked to submit all of the required information at one time. Those submissions are evaluated, and an award is made in accordance with the selected procurement methodology. • Two-step: Competitors are asked to submit qualifications in the first step which are eval- uated to form a shortlist of qualified competitors. The second step comprises the submission and evaluation of all other required information such as technical proposal and project team makeup. Again, the award is made in accordance with the selected procurement methodology. • Multiphase: The project is divided into phases, and the winning competitor is selected using the qualifications-based procurement method. Upon selection, the required information is submitted and evaluated on a phase-by-phase basis until the entire project is awarded. This Airports have considerable freedom in their choice of project delivery and procurement compared with other transportation sectors, mainly due to airports’ funding sources being more varied compared with highway and transit agencies where most of the funding comes from federal and state sources.

Overview of Qualifications-Based Selection 15 process has witnessed increasing use by the owners in the progressive design–build (PDB) delivery method. The details of the common project delivery methods are discussed in Appendix A. The main source of these descriptions is ACRP Report 21 (Touran et al. 2009a). Qualifications-Based Selection Factors Before getting into the details of the factors that are used to make a QBS award, it is important to distinguish QBS from the typical public agency contractor prequalification system. A simple working definition for the difference between QBS and prequalification was developed and is presented as follows: • Contractor prequalification: The contractor’s experience, capacity, financial information, and other submittals required by a public agency to permit a given contractor to submit a bid or a proposal on a public project. This administrative system does not vary appreciably from project to project. • QBS: A project-specific set of factors used to define the most highly qualified contractor, which will then be awarded the contract. Put another way, in most cases a contractor will have to meet the prequalification requirements to be able to submit its statement of qualifications in response to a specific project’s RFQ/RFP. Contractor prequalification is normally specific to the given agency, flows from the statutory constraints that govern its procurement practices, and comprises the specific administrative requirements that define con- tractor responsibility. Common factors range from registration with an appropriate state agency, such as the U.S. Department of Commerce, to the submission of proof of financial capacity to the certification that the contractor is not a felon. Since contractor prequalification applies to all types of project delivery and procurement methods, the synthesis will instead focus on the project-specific elements of qualifications, past performance, etc. that are common to QBS contract award. Motivations for Developing and Implementing Qualifications-Based Selection Awards A study of contractor qualification issues by Minchin and Smith (2001) found that the motivations for implementing a contract award process based on contractor performance and qualifications sprang from two primary concerns. The authors identified the first deals with “frustrations” felt by both owners and construction contractors. These are described in the following list: • Public owners generally treat low-quality construction work no differently than high-quality construction work. • Public owners indirectly reward poor workmanship by not penalizing poor workmanship, thus giving a bidding edge to those contractors who consistently perform poorly. • Administrative prequalification merely establishes a benchmark for financial capacity, not technical capability. • Reliance on performance bonding does not protect the public owner from marginally competent contractors who have a strong financial foundation. Gransberg and Riemer (2009) These frustrations are founded in typical public agency statutory mandates to ensure free and open competition and to avoid potential project delays due to bid protests. Project-specific qualification inherently reduces the field of eligible competitors. Therefore, these programs The first step in a sound construction risk management program is the selection of a qualified contractor. Prichard 2000

16 Value, Benefits, and Limitations of Qualifications-Based Selection for Airport Project Delivery must be well designed and avoid the appearance of being arbitrary. The Delaware Code (2001) for public contracting furnishes that state’s agencies with authority to qualify construction contractors (29 Del. C. § 6962), citing 10 specific reasons why a contractor can be found to not be qualified to bid. Two of these, “inadequate experience to undertake the project” and “documented failure to perform on prior public or private construction contracts,” would fall into the QBS area. The report by Minchin and Smith (2001) goes on to detail the second motivation for imple- menting performance-based contractor selection. The trend of increasing usage of ACMs neces- sarily places a greater reliance on contractor quality control as described below: Changes in regulations regarding use of contractor quality testing in quality assurance decisions and continuing reduction in [agency] personnel will increase the need for “quality driven” contractors in public transportation construction projects. This change, coupled with more departments adopting performance-based and performance-related specifications, places more need on contractors to know and use quality management in their field operations management. With more contractors providing the quality control function, the agencies’ role would change to a quality assurance role. As one part of the quality assurance process, there is a need for comprehensive methods to evaluate a contractor’s eligibility to engage in work from a quality perspective. (Minchin and Smith 2001, p. 5) This concern is echoed by another author expressed in a paper about the efficacy of contractor-led quality control and described as follows: As state highway agencies move further in this direction, it is incumbent on them to first plan carefully during the procurement phase to ensure that they choose qualified teams. They must then draft contracts and specifications that put sufficient checks and balances in place so that these project delivery methods return quality equal to or better than that obtained by the traditional methods. (Ernzen and Feeney 2002, p. 259) Thus, the motivation for shifting from a low-bid award to qualifications and performance-based contractor selection is twofold. First, it provides an incentive by rewarding good performance through enhanced competitiveness on future projects, and second, it reduces performance risk by allowing the agency to select the best-qualified contractor with a record of good performance and entrust it with a greater deal of autonomy in the quality management process during the delivery of ACM projects. In both cases, the agency can discharge its fiduciary responsibility to the public to deliver value for money. To achieve the purpose, the QBS process needs include all required components for collecting contractor performance data, evaluating those data in a fair and transparent fashion, and making QBS awards in a consistent manner over the long run. Contractor Qualifications-Based Selection Factors According to Russell and Jaselskis (1992), the potential for contractor failure increases as the effort spent by the owner evaluating a contractor’s ability to perform the work before contract award decreases. Increases in the final project cost and schedule duration are the impact of contractor failure. Prichard (2000, p. 1) describes the same notion this way: “The first step in a sound construction risk management program is the selection of a qualified contractor.” A more recent study found that a contractor qualification system adds value to the project by reducing performance risk (Gransberg and Riemer 2009). The Australian State of New South Wales (NSW) uses a QBS process (called the “scheme”) and describes the following benefits: • Allows the NSW Government as a major buyer of construction related services to more effectively implement continuous improvement initiatives in the construction industry to achieve better project outcomes; and • Results in significantly reduced tender assessment times and simplified contract administration because prequalified tenderers [bidders] have already demonstrated an understanding of and compliance with

Overview of Qualifications-Based Selection 17 NSW Government construction industry benchmarks, with management procedures and systems re- quirements; and • In line with the NSW Government’s direction to do business with the best of the private sector, the scheme provides for incentives for good performance and also for the application of restrictions or sanctions in the event of poor performance as measured against the respective scheme requirements. (New South Wales Department of Commerce 2007, p. 3, emphasis added) Thus, given the benefits previously identified with selecting a contractor on the basis of quali- fications and past performance rather than low bid, determining which qualification factors are included in the evaluation becomes important. Gransberg and Riemer (2009) did a comprehen- sive review of factors commonly used in performance-based prequalification of construction contractors. It can be argued that QBS award is a competitive prequalification process because those competitors that do not make the shortlist are eliminated from the competition, which constitutes a form of prequalification. Therefore, the list of performance-based qualification factors found in that document was used as the foundation for conducting a content analysis on QBS-related solicitation documents. Table 3 is the list found in the synthesis. The population of factors was further categorized into the four areas shown in the table. Solicitation Document Content Analysis The content analysis reviewed 81 procurement documents from federal, state, and local agen- cies, including documents from 31 airport projects in 18 states. The documents came from four modes of transportation: airport, seaport/civil, highway, and rail/transit. Content analysis is commonly used to develop “valid inferences from a message, written or visual, using a set of procedures” (Neuendorf 2002, p. 37). The first step is to develop a set of standard categories in which key terms that are observed in the text of the solicitation document are recorded. Next, the frequency of their appearance is a quantitative method to infer the content of the solicitation document (Weber 1985). Using the factors shown in Table 3 as the initial coding structure, the number of times each factor was found in each document was recorded. The results were then consolidated to provide a simplified dataset that was broken down as follows: • Financial: Includes all information commonly requested to determine if a given contractor is financially capable of completing the given project. • Bonding: Includes the ability to provide performance, payment, bid, and other required bonds from a qualified surety. Performance-Based Qualification Factors Administrative Factors Personnel and Experience Financial capability Resumes for key personnel Calculated capacity factor from financials Professional licensing for key personnel Bonding capacity Key personnel past project experience Surety statements Past project experience Detailed financial analysis Timely completion of past projects Bank statements Performance evaluations Insurances Technical ability Availability Equipment and plant Legal Factors Management Plans Past illegal behavior Quality management plans Traffic control plans Claims history Safety plans Subcontracting plans Previous debarment Workmen’s compensation modifier Disadvantaged business enterprise plan Environmental plans Source: Gransberg and Riemer 2009. Table 3. Performance-based qualification factors.

18 Value, Benefits, and Limitations of Qualifications-Based Selection for Airport Project Delivery • Current workload: Assesses the contractor’s ability to provide the necessary human and equipment resources to the project within the context of its other ongoing workload. • Key personnel experience: Includes all information commonly requested to assess the credentials and past experience of key contractor personnel. • Past performance: Includes all information commonly requested to determine if the contrac- tor’s past projects have been satisfactorily completed. • Management plans: Includes the commonly requested plans that are necessary to determine if the contractor understands the complete scope of work as well as the constraints imposed by external sources such as statutes, environmental mandate, owner policies/preferences, secu- rity restrictions, and impact on operations. • Quality plans: Typically include the contractor’s approach to design and construction quality management. • Subcontracting plan: Includes all commonly requested information to determine the amount of subcontracting, the key trades that will be provided by subcontractors, and the disadvantaged business enterprise (DBE) utilization. • Safety record: Includes all the information commonly requested to assess the contractor’s safety performance such as its Workmen’s compensation modifier rate and lost-time accident rate on past projects. The content output was analyzed first by mode to determine if differences existed between the airport output and that from the other three transportation modes. Figure 2 shows the results sorted by most to least frequent for the airport results. One can see that airports perceive that contractor past performance, key personnel, financial capacity, and bonding are the most important factors. This is generally the case for the other three modes. Additionally, airport solicitation documents also asked for management and subcontracting plans in more than 50% of the cases. In fact, past performance, key personnel, financial capacity, bonding, and manage- ment plans were the top five factors for all four modes. Figure 3 shows the comparison of the same set of qualification factors when sorted by project delivery method. Once again, contractor past performance, key personnel, financial capacity, and bonding are the top four factors for all five delivery methods. Management plans were the fifth factor for all methods except IDIQ. This leads one to conclude that the top five qualification factors are important regardless of mode or project delivery method. CMR and DB projects made up 66 of the 81 documents reviewed. Of the airport sample, there were 8 CMR projects and 14 DB projects. Thus, it is instructive to look at only those two delivery methods to see if the trend remains the same. Figure 4 shows only the CMR and DB sample and confirms the above conclusion regarding the top five factors. It also shows that subcontracting, management, and quality plans were found in more than 50% of the DB projects. Quality management plans also were found in more than 50% of the CMR documents. Therefore, one can infer that the QBS evaluation and award plan for a PDB project would be more involved than one for a QBS CMR award. Overview Summary This chapter defined the basic terms and criteria that frame the QBS discussion. As part of this review, definitions of project delivery method, procurement methods, and basic characteristics of various ACM, especially as used and practiced by airports, were provided. The chapter reviewed the issues that must be addressed for airports to implement QBS award of ACM contracts. It discussed the applicability of the Brooks Act and its variants to the process and concludes that the Brooks Act applies only to professional services contracts. As a result, airports are not con- strained by this federal law when services other than professional services are being procured.

Overview of Qualifications-Based Selection 19 0% 20% 40% 60% 80% 100% Current Workload Quality Plans Safety Record Subcontracting Plan Management Plans Bonding Key Personnel Financial Past Performance Airport Seaport/Civil Highway Rail/Transit Figure 2. Content analysis results sorted by mode.

20 Value, Benefits, and Limitations of Qualifications-Based Selection for Airport Project Delivery 0% 20% 40% 60% 80% 100% Current Workload Quality Plans Safety Record Subcontracting Plan Management Plans Bonding Key Personnel Financial Past Performance DBB-BV CMR DB IDIQ P3 Figure 3. Evaluated qualification factors sorted by project delivery method.

Overview of Qualifications-Based Selection 21 This extends to DB projects as a result of the comptroller general’s determination that DB projects are construction projects and not governed by the Brooks Act. The major constraints faced by airports regarding the implementation of QBS of contractors are associated with federal AIP grants and state funding that mandate that price be included in the award decision. Once the airport has satisfied itself that it can use QBS and has selected a project delivery method, the content analysis shows that the qualifications evaluation criteria should contain pertinent factors. These factors include, at a minimum, contractors’ past project performance, qualifications and experience of key contractor personnel, the company’s financial capacity to complete the work, and its ability to furnish the requisite bonds. 0% 20% 40% 60% 80% 100% Current Workload Quality Plans Safety Record Subcontracting Plan Management Plans Bonding Key Personnel Financial Past Performance CMR DB Figure 4. Evaluated qualification factors for CMR and DB projects.