Value, Benefits, and Limitations of Qualifications-Based Selection for Airport Project Delivery (2019)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

58 A P P E N D I X C Survey Questionnaire and Results The synthesis used two surveys. The first survey was a screening survey to members of the Airport Consultants Council, whose purpose was to identify airports that utilize qualifications- based selection (QBS) awards. The second survey was a survey to airports and sought to document current QBS practices at the responding airport. Short Screening Survey to Identify Airports That Use QBS for Contractor Selection This survey has been developed for identifying airports that have used Qualifications-Based Selection (QBS) for construction services procurement. This effort is part of the ACRP Synthesis Project 11-03/Topic S01-18 to explore Qualifications-Based Selection (QBS) for construction phase. Compiling this information provides additional resources to assist in decision making when choosing the most advantageous procurement approach. The intended audience for this survey includes airport administrators and procurement decision makers. 1. Do you have knowledge about the construction services procurement/decision making process at the airports where you furnish professional services? [Yes/No] SKIP LOGIC—If “no” to (1), then message: “Please forward to someone in your organization who has knowledge of airport construction procurement. Thank you.” If “yes”: Respondent Information First, Last Name: Organization: Email Address/Phone Number: 2. At which airports do you have knowledge about the construction procurement process? Please provide the following information for each airport Airport Code Point of contact’s name/position Email

Survey Questionnaire and Results 59 selection are qualifications and demonstrated competence. Price and cost are not selection criteria, but they may be considered during contract negotiation (AGC 2011). a. Yes b. No c. Unsure 4. Which of the alternative contracting methods are used by the airports identified in Question 2 above? Airport Code CM-at- Risk (CMR)1 Design– Build (DB) Progressive DB (PDB)2 Public–Private Partnerships (PPP) Indefinite Delivery/ Indefinite Quantity (IDIQ)3 Integrated Project Delivery (IPD) 1 Also called: CMAR, CM@Risk, CM/GC, GC/CM 2 Also called: DB QBS 3 Also called: Job order contract, task order contract, on-call contract, master contract, etc. 5. Which of the following answers describes your personal perception of using QBS to procure construction services: (Check all that apply) a. QBS should not be used on publicly funded projects for any services. b. QBS should only be used for professional services procurement (i.e., architectural, engineering services). c. Replacing a low bid selection with QBS will provide airports with better qualified contractors. d. Replacing a low bid selection with QBS will provide airports with ability to avoid awarding contracts to contractors with poor records of past performance. e. Replacing a low bid selection with QBS will increase the cost of airport projects. f. Replacing a low bid selection with QBS will increase the quality of airport projects. g. Replacing a low bid selection with QBS will decrease disputes on airport projects. h. Replacing a low bid selection with QBS will enhance the timely completion of airport project. i. No opinion. 3. Do the airports where you work have enabling legislation that allows Qualifications Based Selection (QBS) for selection of the construction contractor for airport public project delivery? Qualifications Based Selection (QBS): A procurement method in which the final criteria for

60 Value, Benefits, and Limitations of Qualifications-Based Selection for Airport Project Delivery Survey #2 Introduction This survey has been developed for ACRP Synthesis Project 11-03/Topic S01-18 to explore Qualifications-Based Selection (QBS) for construction services procurement. Compiling this information provides additional resources to assist in decision making when choosing the most advantageous procurement approach. The intended audience for this survey includes airport administrators and procurement decision makers. 1. Are you involved in airport administration and/or construction services procurement/decision making? [Yes/No] SKIP LOGIC—If “no” to (1), then message: “Please forward to someone in your agency who is involved in construction procurement. Thank you.” If “yes”: Respondent Information First, Last Name: Organization: Email Address/Phone Number: General Questions 2. Does your state have enabling legislation that allows Qualifications-Based Selection (QBS) for selection of the construction contractor for airport public project delivery? Qualifications Based Selection (QBS): A procurement method in which the final criteria for selection are qualifications and demonstrated competence. Price and cost are not selection criteria, but they may be considered during contract negotiation (AGC of America 2009). a. Yes b. No c. Unsure 3. Select any of the following that apply to your organization related to use of Qualifications Based Selection (QBS): [if b. selected SKIP LOGIC to matrices—hypothetical] d. Only professional services procurement (i.e., architectural, engineering services) e. Professional and construction services procurement (i.e., designer, construction contractor) f. My organization has not used QBS to procure services g. My organization is interested in/considering using QBS to procure services h. Please briefly describe your role related to project delivery and procurement. i. Would you be willing to participate in a 30-minute telephone interview regarding the details of one of your QBS projects? a. Yes b. No

Survey Questionnaire and Results 61 j. Please provide the status of the following project delivery methods in your organization: Design–Bid–Build (DBB): a contracting method in which the construction contract is awarded to the lowest responsive bidder. Design–Build (DB): a contracting method in which a single entity provides design services and constructs the project all under one contract. Progressive DB (PDB): a variation of DB where the design–builder is selected using QBS and the construction price is negotiated during preconstruction. Construction Manager-at-Risk (CMR): a contracting method that allows an owner to engage a contractor during the design process to provide constructability input with the intention of letting the contractor to later construct the project [also termed construction manager/general contractor (CMGC or GCCM); construction manager as constructor (CMc), early contractor involvement (ECI), etc.]. NOTE: CM-Agency (CMA) is not included in this category. CMA is merely a separate professional services contract to act as the owner’s representative during design and construction. Public–Private Partnerships (PPP): a contracting method between a public agency and a private-sector entity that allow for greater private-sector participation in the delivery and financing of capital projects. Integrated Project Delivery (IPD): a contracting method that joins the primary disciplines to the owner by a multiparty agreement where the team shares risk as well as the performance outcomes. Indefinite Delivery/Indefinite Quantity (IDIQ): a contracting method that is a multiproject, multiyear mechanism to permit an agency to hire a contractor on an as-needed basis to provide a specified set of construction services [also termed job order contract (JOC), task order contract (TOC), on-call contract, stand-by contract, master contract, etc.]. Alternative Contracting Methods Authorized Authorized, Not Used Not Authorized, Interested in Using Not Authorized Design–Build (DB) Progressive Design–Build (PDB) Construction Manager-at-Risk (CMR) Public–Private Partnership (PPP) Integrated Project Delivery (IPD) Indefinite Delivery/Indefinite Quantity (IDIQ) When Do You Use QBS for Construction Services Procurement? k. Please indicate the number of projects delivered by the following project delivery methods in which QBS is used to procure construction services in your organization: Contracting Method Using QBS to Procure Construction Services Number of Projects Using QBS for Construction Services Never 1–2 3–5 6–10 More than 10 Design–Bid–Build (DBB) Design–Build (DB) Progressive Design Build (PDB) Construction Manager-at-Risk (CMR) Public–Private Partnership (PPP) Integrated Project Delivery (IPD) Indefinite Delivery/Indefinite Quantity (IDIQ)

62 Value, Benefits, and Limitations of Qualifications-Based Selection for Airport Project Delivery l. In your organization, which funding sources are used in QBS for construction services procurement? [Select all that apply.] a. Federal Funding (PDB, CMR) b. State Funding c. Passenger Facility Charges d. Airport Cash Flows for Project Finance e. Debt Financing/Airport Revenue Bonds f. Other Public Funding ________________________ g. Other Private Funding ________________________ m. Which types of projects would/has your organization use/used QBS to procure construction services? Project Type Using QBS for Construction Procurement N/A DBB DB PDB CMR P P P IPD IDIQ New Construction/Expansion: Airside New Construction/Expansion: Landside New Construction/Expansion: Terminals Reconstruction/Rehabilitation (Civil) Renovation (Buildings) Maintenance Safety Projects (e.g., Runway Incursion) Systems (e.g., baggage handling, people mover, MEP) Small projects (<$5M) Larger projects (≥$5M) Other Why Do You Use QBS for Construction Services Procurement? n. Indicate the factors considered when considering deliver a given project using QBS for construction. Scale: 1 = Always consider, 2 = Usually consider, 3 = Considered, 4 = Rarely considered, 5 = Not considered. Selection Criteria N/A 1 2 3 4 5 Project complexity Number of public third-party stakeholders (environmental, permitting, security, etc.) Number of private third-party stakeholders (airlines, concessions, utility owners, etc.) Need for early contractor involvement to achieve constructability Number of utility conflicts New technology included in design Impact on operations during construction Schedule considerations Complicated sequence of work Need for phased construction Physical security issues Personnel security issues Need for enhanced cost certainty Transparency in pricing process (open books accounting) Contingency management/control Other

Survey Questionnaire and Results 63 o. What benefits do you believe you would realize or have realized by using QBS in construction procurement (compared to “low-bid” procurement)? Scale: 1 = Significant benefit, 2 = Some benefit, 3 = No benefit, 4 = Negative impact, 5 = Significant negative impact. QBS in Construction Procurement N/A 1 2 3 4 5 Best-qualified team selected (flexibility) Early involvement of the key members of project teams Enhanced cooperation, collaboration, and communication among project teams Lower design costs Lower construction costs Higher quality construction Greater project value More accurate project sequencing/scheduling Lower life-cycle costs Shorter project timelines Better project control predictability (e.g., price, schedule, quality) More efficiency throughout the project process Greater continuity throughout project process Better long-term relationships with construction service providers Prequalification of subcontractors Better promotion of technical innovation Greater utilization of technology such as building information modeling Transparency in decision making and documentation among project team Enhanced ability to overcome challenges and problems Reduced change orders Reduced risk in complex projects Reduced litigation Fewer warranty issues Other p. Which challenges has your organization experienced when using QBS in construction procurement? Scale: 1 = Not challenging, 2 = Minor challenge, 3 = Routine challenge, 4 = Major challenge, 5 = Show stopper. QBS in Construction Procurement N/A 1 2 3 4 5 Project funding source Working with existing procurement codes Dealing with passage of enabling statutes Use of new procurement methods/contracts Determining which projects will benefit from QBS Establishing the selection committee Getting buy-in for contractor QBS from stakeholders Getting buy-in from project team members to collaborate Having adequate skill among staff to execute QBS projects Educating and training project team Resourcing the required owner staffing Determining appropriate selection criteria and weighting scheme Establishing accountability and common goals of project team members Reduced cost certainty for construction at time of contractor selection Executing negotiations with contractors Obtaining required leadership from project teams Developing the appropriate dispute resolution process Determining appropriate project performance indicators

64 Value, Benefits, and Limitations of Qualifications-Based Selection for Airport Project Delivery q. Rate the frequency of the following selection criteria on your decision to deliver a given project using QBS for construction. Scale: 1 = Every decision, 2 = Most decisions, 3 = Some decisions, 4 = Rarely used in decision, 5 = Not considered. Selection Criteria N/A 1 2 3 4 5 Experience of the company Past performance of the company Experience (skill sets) of assigned individuals Past performance of assigned individuals Experience and past performance with desired delivery system (method) Capacity of contractor to perform the work Availability to perform the work Financial strength and bonding capability Safety plan and safety record Quality assurance plan Management plan and subcontractor relationships Other r. Rate the impact of the following selection criteria on your decision to deliver a given project using QBS for construction. Scale: 1 = Greatest impact, 2 = High impact, 3 = Some impact, 4 = Minimal impact, 5 = No impact/not considered. Selection Criteria N/A 1 2 3 4 5 Experience of the company Past performance of the company Experience (skill sets) of assigned individuals Past performance of assigned individuals Experience and past performance with desired delivery system (method) Capacity of contractor to perform the work Availability to perform the work Financial strength and bonding capability Safety plan and safety record Quality assurance plan Management plan and subcontractor relationships Other Feedback About Use of QBS for Construction Services Procurement s. Is there a particular project type that immediately causes you to consider QBS contractor selection? t. What would you consider to be effective practices for developing/executing the process of the use of QBS in construction procurement (e.g. staffing needs, education, training, selection process, selection criteria, interviewing, negotiating)? Please specify the related project delivery method as applicable. u. What are some lessons learned from the use of QBS in construction for your organization that you believe would be helpful to other airports? Please specify the related project delivery method as applicable. v. What else would you like to add regarding key success factors for QBS in construction procurement that you believe would be helpful to other airports (e.g. more information, documents)?

Survey Questionnaire and Results 65 Summary of Survey Results 4. Which of the following answers describes your personal perception of using QBS to procure construction services: (Check all that apply) Perception Response Rate Replacing a low-bid selection with QBS will provide airports with ability to avoid awarding contracts to contractors with poor records of past performance. 92.3% Replacing a low-bid selection with QBS will increase the quality of airport projects. 88.5% Replacing a low-bid selection with QBS will provide airports with better qualified contractors. 76.9% Replacing a low-bid selection with QBS will decrease disputes on airport projects. 61.5% Replacing a low-bid selection with QBS will enhance the timely completion of airport projects. 46.2% Replacing a low-bid selection with QBS will make it difficult to determine reasonableness of costs. 42.3% Replacing a low-bid selection with QBS will increase the cost of airport projects. 15.4% QBS should only be used for professional services procurement (i.e., architectural, engineering services). 3.8% QBS should not be used on publicly funded projects for any services. 0.0% 5. Please provide the status of the following project delivery methods in your organization: 6. Please indicate the number of projects delivered by the following project delivery methods in which QBS is used to procure construction services in your organization: ACM Authorization and Experience Authorization (# responses of 26 total) Number of Projects Completed Using Each ACM ACM Authorized Not Authorized, But Interested Don’t know Never 1-2 3-5 6-10 More Than 10 CMR 24 1 2 2 9 4 1 8 DB 24 0 2 5 11 3 1 3 PDB 12 1 7 12 3 1 2 1 PPP 12 2 7 12 6 1 0 0 IDIQ 24 1 1 4 5 4 5 7 IPD 1 1 6 20 0 0 0 0 7. In your organization, which funding sources are used in QBS for construction services procurement? [Select all that apply.] Funding Sources for QBS-ACM Projects (# responses of 26) Federal Funding State Funding Passenger Facility Charges Airport Cash Flows for Project Finance Debt Financing/Airport Revenue Bonds Other Public Funding 19 9 19 24 22 7

66 Value, Benefits, and Limitations of Qualifications-Based Selection for Airport Project Delivery 8. Which types of projects would/has your organization use/used QBS to procure construction services? QBS-ACM Project Types, Sizes, and Sector (# responses of 26) Project Type Using QBS for Construction Procurement DBB CMR DB PDB PPP ACM IDIQ New construction/expansion: airside 19 11 9 2 2 24 0 New construction/expansion: landside 14 14 10 4 6 34 1 New construction/expansion: terminals 9 16 9 5 0 30 1 Small projects (<$5M) 15 7 2 0 1 10 12 Larger projects (≥$5M) 10 13 10 5 3 31 0 Reconstruction/rehabilitation (civil) 17 6 6 0 2 14 5 Renovation (buildings) 13 9 7 3 1 20 11 Maintenance 11 0 1 0 0 1 13 Safety projects (e.g., runway incursion) 13 6 7 3 0 16 6 Systems (e.g., baggage handling, people mover, MEP) 6 9 9 4 2 24 5 9. Indicate the factors considered when considering deliver a given project using QBS for construction. Scale: 1 = Always consider, 2 = Usually consider, 3 = Considered, 4 = Rarely considered, 5 = Not considered. Factors Considered in Decision to Use QBS Average Project complexity 4.70 Complicated sequence of work 4.55 Need for early contractor involvement 4.30 Need for phased construction 4.25 Contingency management/control 4.25 Schedule considerations 4.20 Impact on operations during construction 4.15 Transparency in pricing process 3.85 Number of utility conflicts 3.70 Physical security issues 3.60 Need for enhanced cost certainty 3.50 New technology included in design 3.45 Personnel security issues 3.45 Number of public third-party stakeholders 3.25 Number of private third-party stakeholders 3.10

Survey Questionnaire and Results 67 10. What benefits do you believe you would realize or have realized by using QBS in construction procurement (compared to “low-bid” procurement)? Scale: 1 = Significant benefit, 2 = Some benefit, 3 = No benefit, 4 = Negative impact, 5 = Significant negative impact. Anticipated Benefits of QBS Delivery Average Best-qualified team selected (flexibility) 4.65 Prequalification of subcontractors 4.65 Early involvement of the key members 4.60 More accurate project sequencing/scheduling 4.40 More efficiency throughout the project process 4.35 Greater continuity throughout project process 4.35 Better project control predictability 4.30 Better long-term relationships 4.30 Greater project value 4.25 Enhanced ability to overcome challenges 4.20 Reduced litigation 4.20 Reduced risk in complex projects 4.10 Enhanced collaboration/communication 4.00 Higher quality construction 4.00 Reduced change orders 4.00 Transparency in decision making 3.95 Greater utilization of technology such as building information modeling 3.90 Fewer warranty issues 3.90 Shorter project timelines 3.80 Better promotion of technical innovation 3.55 Lower construction costs 3.40 Lower design costs 3.35 Lower life-cycle costs 3.20

68 Value, Benefits, and Limitations of Qualifications-Based Selection for Airport Project Delivery 11. Which challenges has your organization experienced when using QBS in construction procurement? Scale: 1 = Not challenging, 2 = Minor challenge, 3 = Routine challenge, 4 = Major challenge, 5 = Show stopper. Challenges Faced with QBS Use Average Determining which projects will benefit from QBS 3.25 Project funding source 3.00 Establishing accountability and common goals 2.80 Reduced cost certainty at contractor selection 2.75 Executing negotiations with contractors 2.75 Determining project performance indicators 2.65 Enhanced ability to overcome challenges 2.55 Working with existing procurement codes 2.35 Getting buy-in from project team 2.35 Use of new procurement methods/contracts 2.25 Resourcing the required owner staffing 2.25 Educating and training project team 2.20 Obtaining required leadership from project teams 2.20 Staff experience with QBS projects 2.05 Determining selection criteria 2.05 Dealing with passage of enabling statutes 1.90 Developing dispute resolution process 1.80 Establishing the selection committee 1.75 Getting buy-in from stakeholders 1.70 12. Rate the frequency of the following selection criteria on your decision to deliver a given project using QBS for construction. Scale: 1 = Every decision, 2 = Most decisions, 3 = Some decisions, 4 = Rarely used in decision, 5 = Not considered. Frequency Average Experience of the company 4.9 Past performance of the company 4.9 Availability to perform the work 4.9 Capacity of contractor to perform the work 4.9 Safety plan and safety record 4.8 Past performance of contractor’s proposed project personnel 4.7 Experience (skill sets) of contractor’s proposed project personnel 4.7 Experience and past performance with desired delivery system (method) 4.7 Quality assurance plan 4.5 Financial strength and bonding capability 4.3

Survey Questionnaire and Results 69 13. Rate the impact of the following selection criteria on your decision to deliver a given project using QBS for construction. Scale: 1 = Greatest impact, 2 = High impact, 3 = Some impact, 4 = Minimal impact, 5 = No impact/not considered. Value Ave Capacity of contractor to perform the work 4.9 Experience of the company 4.8 Past performance of the company 4.8 Availability to perform the work 4.8 Experience (skill sets) of contractor’s proposed project personnel 4.6 Past performance of contractor’s proposed project personnel 4.6 Safety plan and safety record 4.6 Experience and past performance with desired delivery system (method) 4.4 Quality assurance plan 4.4 Financial strength and bonding capability 4.1