National Academies Press: OpenBook

Construction Manager-at-Risk Project Delivery for Highway Programs (2010)

Chapter: Chapter Four - Construction Manager-At-Risk Selection Methods

« Previous: Chapter Three - Procedures
Page 33
Suggested Citation:"Chapter Four - Construction Manager-At-Risk Selection Methods." National Academies of Sciences, Engineering, and Medicine. 2010. Construction Manager-at-Risk Project Delivery for Highway Programs. Washington, DC: The National Academies Press. doi: 10.17226/14350.
×
Page 33
Page 34
Suggested Citation:"Chapter Four - Construction Manager-At-Risk Selection Methods." National Academies of Sciences, Engineering, and Medicine. 2010. Construction Manager-at-Risk Project Delivery for Highway Programs. Washington, DC: The National Academies Press. doi: 10.17226/14350.
×
Page 34
Page 35
Suggested Citation:"Chapter Four - Construction Manager-At-Risk Selection Methods." National Academies of Sciences, Engineering, and Medicine. 2010. Construction Manager-at-Risk Project Delivery for Highway Programs. Washington, DC: The National Academies Press. doi: 10.17226/14350.
×
Page 35
Page 36
Suggested Citation:"Chapter Four - Construction Manager-At-Risk Selection Methods." National Academies of Sciences, Engineering, and Medicine. 2010. Construction Manager-at-Risk Project Delivery for Highway Programs. Washington, DC: The National Academies Press. doi: 10.17226/14350.
×
Page 36
Page 37
Suggested Citation:"Chapter Four - Construction Manager-At-Risk Selection Methods." National Academies of Sciences, Engineering, and Medicine. 2010. Construction Manager-at-Risk Project Delivery for Highway Programs. Washington, DC: The National Academies Press. doi: 10.17226/14350.
×
Page 37
Page 38
Suggested Citation:"Chapter Four - Construction Manager-At-Risk Selection Methods." National Academies of Sciences, Engineering, and Medicine. 2010. Construction Manager-at-Risk Project Delivery for Highway Programs. Washington, DC: The National Academies Press. doi: 10.17226/14350.
×
Page 38
Page 39
Suggested Citation:"Chapter Four - Construction Manager-At-Risk Selection Methods." National Academies of Sciences, Engineering, and Medicine. 2010. Construction Manager-at-Risk Project Delivery for Highway Programs. Washington, DC: The National Academies Press. doi: 10.17226/14350.
×
Page 39
Page 40
Suggested Citation:"Chapter Four - Construction Manager-At-Risk Selection Methods." National Academies of Sciences, Engineering, and Medicine. 2010. Construction Manager-at-Risk Project Delivery for Highway Programs. Washington, DC: The National Academies Press. doi: 10.17226/14350.
×
Page 40
Page 41
Suggested Citation:"Chapter Four - Construction Manager-At-Risk Selection Methods." National Academies of Sciences, Engineering, and Medicine. 2010. Construction Manager-at-Risk Project Delivery for Highway Programs. Washington, DC: The National Academies Press. doi: 10.17226/14350.
×
Page 41
Page 42
Suggested Citation:"Chapter Four - Construction Manager-At-Risk Selection Methods." National Academies of Sciences, Engineering, and Medicine. 2010. Construction Manager-at-Risk Project Delivery for Highway Programs. Washington, DC: The National Academies Press. doi: 10.17226/14350.
×
Page 42
Page 43
Suggested Citation:"Chapter Four - Construction Manager-At-Risk Selection Methods." National Academies of Sciences, Engineering, and Medicine. 2010. Construction Manager-at-Risk Project Delivery for Highway Programs. Washington, DC: The National Academies Press. doi: 10.17226/14350.
×
Page 43
Page 44
Suggested Citation:"Chapter Four - Construction Manager-At-Risk Selection Methods." National Academies of Sciences, Engineering, and Medicine. 2010. Construction Manager-at-Risk Project Delivery for Highway Programs. Washington, DC: The National Academies Press. doi: 10.17226/14350.
×
Page 44
Page 45
Suggested Citation:"Chapter Four - Construction Manager-At-Risk Selection Methods." National Academies of Sciences, Engineering, and Medicine. 2010. Construction Manager-at-Risk Project Delivery for Highway Programs. Washington, DC: The National Academies Press. doi: 10.17226/14350.
×
Page 45
Page 46
Suggested Citation:"Chapter Four - Construction Manager-At-Risk Selection Methods." National Academies of Sciences, Engineering, and Medicine. 2010. Construction Manager-at-Risk Project Delivery for Highway Programs. Washington, DC: The National Academies Press. doi: 10.17226/14350.
×
Page 46
Page 47
Suggested Citation:"Chapter Four - Construction Manager-At-Risk Selection Methods." National Academies of Sciences, Engineering, and Medicine. 2010. Construction Manager-at-Risk Project Delivery for Highway Programs. Washington, DC: The National Academies Press. doi: 10.17226/14350.
×
Page 47
Page 48
Suggested Citation:"Chapter Four - Construction Manager-At-Risk Selection Methods." National Academies of Sciences, Engineering, and Medicine. 2010. Construction Manager-at-Risk Project Delivery for Highway Programs. Washington, DC: The National Academies Press. doi: 10.17226/14350.
×
Page 48
Page 49
Suggested Citation:"Chapter Four - Construction Manager-At-Risk Selection Methods." National Academies of Sciences, Engineering, and Medicine. 2010. Construction Manager-at-Risk Project Delivery for Highway Programs. Washington, DC: The National Academies Press. doi: 10.17226/14350.
×
Page 49

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

34 INTRODUCTION One of the advantages of CMR as cited in Table 1 was the ability to select the contractor on the basis of its qualifications, past performance, and record of success. NCHRP Synthesis 390 concluded that “the ‘soft’ factors related to managerial com- petence and past performance are more important to the prequalification process than the ‘hard’ aspects related to bonding and financial status” (Gransberg and Riemer 2009). Hence the procurement method used for CMR project delivery follows a “best-value” process. The following is its definition from NCHRP Report 561: Best-Value—a procurement process where price and other key factors are considered in the evaluation and selection process to enhance the long-term performance and value of construction (Scott et al. 2006). NCHRP Report 561 goes on to describe the fundamental theory of best-value procurement. That description is worth repeating in this synthesis because it essentially forms a foun- dation around which CMR selection methods can be designed and evaluated. Best-value procurement methods allow various elements to be considered in selecting a contractor on the basis of performance. Objective elements include contractor experience with similar projects, completion within schedule, compliance with material and workmanship requirements, timeliness and accuracy of submittals, and record of safety. Subjective elements include effective management of subcontractors, proactive measures to mitigate impacts to adjacent properties and businesses, training and employee development programs, corporate commitment to achieving customer satisfaction, and client relations. These elements not only affect the ultimate performance and overall cost of the completed facility, but also contribute to the efficient execution of the work. Efficiency is very important to contracting authorities that are interested in a high level of public acceptance. It is also recognized that, because of constrained staffing and budgets, it is not possible for state agencies to “inspect” quality into the work. Therefore, a procurement process is needed that considers value-related elements in awarding contracts (Scott et al. 2006; italics added). The layman’s definition of best-value selection systems is simply selecting a contractor on a basis of something other than price alone. The selection methods found in the case study of CMR projects ranged from pure qualifications- based selection (QBS) to combining qualifications, proposed schedule, proposed fees, and unit prices for selected critical pay items. None awarded to the lowest bidder. The design industry is represented by the American Council of Consulting Engineers (ACEC), which is the primary advocate of QBS for awarding design contracts. CONSTRUCTION MANAGER-AT-RISK SELECTION MODELS CMR selection processes and procedures were found in four study instruments. The combination of the case studies, the content analysis, the contractor interviews and the literature review found that there are three standard models for select- ing a CMR. Table 17 shows the frequency of each of the four possible options found in the study. They are differentiated by whether or not the competitors are asked to include proposed fees and other costs and whether the selection decision is made in one step or two steps. The LOI and the RFQ only solicitations are essentially the same process. The difference is only in the level of detail requested in the submittal. Therefore, these two can be com- bined and that leaves three models from which an agency can pick to develop its CMR selection process. The three models are: 1. One-step QBS selection—One-step response to an RFQ only. 2. One-step Best-value selection—One-step response to an RFP only. 3. Two-step Best-value selection—Two-step response to RFQ and RFP. As a result of this synthesis, three figures were derived from the analysis, one to represent each of these selection models. Each includes information regarding the actions for the CMR or its staff and actions by the agency. Also contained in each figure is an example of the information that is included in the response to solicitation documents. One-Step Qualifications-Based Selection Construction Manager-at-Risk Selection Model The first selection method is the QBS One-step model. This model is the simplest and probably the fastest CMR selec- tion method. Four of the ten case study projects have used this method: the Alaska Department of Transportation and Public Facilities (DOT&PF), MDOT, city of Glendale, and Pinal County. It used an even simpler form. FDOT issues a request CHAPTER FOUR CONSTRUCTION MANAGER-AT-RISK SELECTION METHODS

35 for a LOI along with a preliminary set of plans and specifica- tions that only seek a limited amount of qualifications and past performance data. The competitors that are found to be minimally qualified are then interviewed from a pre-published set of questions. The selection decision is then made based on the results of interviews. To keep the numbers straight, FDOT is included in the QBS One-step group for analysis. Addi- tionally, 18% of the transportation solicitation documents reviewed in the content analysis used one-step QBS and 17% of the non-transportation documents fell into this category. The RFQ contained a description of the scope of work in all five cases. In two cases preliminary plans and specifications are also provided (FDOT and Pinal). Finally, Pinal County also included a description of the agency’s quality management roles and responsibilities and expects the competitors to organize to satisfy those requirements. The SOQ submittal requirements for the competing CMRs are shown in Table 18 for those items found in at least two cases. The top three are the qualifications of the project manager, the company’s prior experience executing CMR projects, and the company’s experience on other projects of a similar type. Figure 13 shows how the various components in this model relate to one another. Figure 13 illustrates the flow of documents and decisions through this model. Once a CMR is selected, the remainder of the contract is formed through negotiation. Nothing from the SOQ is carried forward to the GMP. Two of the agencies (Pinal County and MDOT) asked the CMR to declare the work packages that it intended to self-perform. Both allow the CMR to prequalify and select its subcontractors without constraint. MDOT limits the CMR to self-performing 35% of the contract and Pinal County’s limit is 45%. Of the five, Pinal County asks for the most information in the SOQ. One interesting requirement is a critical analysis of the project’s published budget. They also ask for a preliminary schedule and a public relations plan with the qualifications of the public information manager. Hence, this agency has some quantitative information that it can refer to in the GMP nego- tiations. Asking for the budget analysis also allows the CMR to express its budget concerns, if any, before award. So if the CMR’s final GMP is higher than the county hoped, it is not a surprise. The preliminary schedule serves much the same purpose to justify time growth. Figure 13 shows that the agency can ask for anything that it believes will assist it on selecting the best-qualified CMR. However, when approaching QBS, the agency restricts itself to items that assist it in discriminating between highly qualified competitors rather than accumulating submittals that essen- tially say the same thing. Therefore, if the CMR’s safety plan is cogent to the selection process, the agency has justification to ask for those parts of the overall plan that will help it identify the best plan. Every contractor will require its personnel to obey Occupational Safety and Health Administration constraints. Therefore, having the entire safety plan before award has limited utility. However, asking each competitor to detail its plan for protecting the public during construction in an area with a large volume of pedestrian traffic (city of Glendale case study) will draw different approaches from different contractors. This then allows the agency to effectively rate both the CMR’s understanding of the hazards and the realism of its proposed solution. In Figure 13, once a CMR is selected the process proceeds to the provision of preconstruction services and a negotiation of the GMP as the design is developed. The important point Procurement Type Research Instrument LOI RFQ Only RFP Only RFQ/RFP Content Analysis: Transportation 0 10 15 0 Content Analysis: Non-Transportation 0 9 12 8 Highway Agency Case Study 1 4 1 1 Non-Highway Agency Case Study 0 0 2 1 Contractor Interviews 1 4 1 1 Literature Review 0 2 5 4 TABLE 17 PROCUREMENT TYPE FINDINGS SOQ Requirement No. Agencies Requiring Qualifications of the CMRís project manager 5 Past CMR project experience 4 Past related project experience (non-CMR) 4 References from past projects 3 Organizational structure/chart 3 Qualifications of the CMR’s preconstruction services manager 2 Construction quality management plan 2 Declaration of self-performed work 2 Preliminary project schedule 2 TABLE 18 ONE-STEP QBS STATEMENT OF QUALIFICATIONS (SOQ) SUBMITTAL CONTENTS

in this model is the need for agency documentation of the discussions held during the interviews. There is also a need to ensure that the SOQ commitments for specific individuals has been achieved; these could potentially influence the GMP itself. For instance, if a given subcontractor named in the SOQ later becomes unavailable to work on the project, the GMP line for that trade may change. A similar issue arises in the quality assurance (QA) program. The cost of the QA/QC (quality control) tasks that were assigned to the CMR will be reflected in the GMP and the agency can evaluate the thoroughness of the program by looking at how much money has been assigned to cover its costs. To summarize, the one-step QBS CMR selection model is the simplest procedure of the three. Therefore, it can be fast and uncomplicated. 36 The trade-off is the lack of quantitative cost information that can be carried forward to GMP negotiations. One-Step Best-Value Construction Manager-at-Risk Selection Model The one-step best-value model looks similar to a one-step DB procurement used by the federal government before 1996 (Gransberg et al. 2006). The major difference is that the pro- posed costs are not a lump sum price for the project. In this model, the competing CMR firms are asked to furnish their qualifications, proposed fees, and other technical information such as a conceptual schedule or manufactures’ catalog cuts for specific alternative technical concepts. The selection panel then evaluates the proposal and selects the winner using a best-value award algorithm that was published in the RFP. The mechanics of this model are shown in Figure 14. This model was used by the ODOT, the UTA, and Texas Tech University case study projects. The content analysis found that 28% of transportation projects and 22% of non- transportation projects used this model for CMR selection. Table 19 lists the submittal requirements for items that were required by two or more agencies. The majority of these cases had roughly the same requirements for the quali- fications portion as the QBS model. They appeared to request additional management plans. In the cost portion, all asked the competitors to propose their preconstruction services fee. ODOT and Texas Tech also asked for the construction fee and the contractor’s general conditions. The content analysis revealed that some RFPs also ask for certain construction costs that are cogent to the selection process. For example, in the Boston Logan International Airport CMR RFP, the CMR is asked to furnish the cost of preparing and submitting the application for Leadership in Energy and Environmental Design (LEED®) certification. Others ask for mobilization costs and unit prices for a limited number of pay items. Figure 14 shows the major advantage of this model in that the selection process creates quantitative cost information that can be carried forward into the GMP negotiations. Several of the case study interviewees believed that it created some competitive pricing for those items. That notion was confirmed by the contractor interviews where concern about competitive- ness drove the actual numbers that they had to provide in the proposal. The UTA case study project CMR stated that they cut their preconstruction services fee and made a conscious business decision to self-finance the preconstruction phase because they were ready to do this project and believed that they could recover the losses booked in preconstruction by influencing the final design in a manner that aligned with their strengths as well as their existing plant and equipment. To summarize this model, its main feature is the com- bination of qualifications and price. The cost information CMR Request for Qualifications CMR’s Statement of Qualifications Evaluation of Qualifications CMR Selection Award CMR Contract Preconstruction Services Develop & Negotiate GMP GMP Reasonable? Award Construction Contract Yes Construction Organization Chart Corporate Project Experience Key Personnel Experience Required Management Plans Other Submittal Requirements Contents of CMR SOQ Contents of CMR Statement of Qualification Contains Only What Was Requested in RFQ. This Graphic Shows Possible Submittal Requirements Advertise DBB Unable to Agree on GMP Yes No FIGURE 13 QBS one-step CMR selection model. (Shaded areas are actions by the CMR or its staff; unshaded areas are agency actions.)

37 CMR Request for Proposals CMR’s Proposal Evaluation of Qualifications and Fees CMR Selection Award CMR Contract Preconstruction Services Develop & Negotiate GMP GMP Reasonable? Award Construction Contract Yes Organization Chart Corporate Project Experience Key Personnel Experience Required Management Plans Proposed Precon Fee Contents of CMR Proposal Proposed Construction Fee Proposed General Conditions Proposed Construction Costs as req’d Contents of CMR Proposal Contains Only What Was Requested in RFP. This Graphic Shows Possible Submittal Requirements Proposed Construction Schedule as req’d Construction Advertise DBB Unable to Agree on GMP Yes No No Propo sal Com ponents U sed in G M P FIGURE 14 One-step best-value CMR selection model. RFP Requirement No. Agencies Requiring Qualifications of the CMR’s project manager 3 Past CMR project experience 3 References from past projects 3 Proposed preconstruction fee 3 Qualifications of key personnel 2 Construction quality management plan 2 Other key plans: Public relations 2 Preliminary project schedule 2 Proposed construction fee 2 Proposed general conditions 2 TABLE 19 ONE-STEP BEST-VALUE PROPOSAL SUBMITTAL CONTENTS

contained in the proposal act as an owner’s risk mitigation tool to control total cost in GMP negotiations by locking down the indirect costs and fees. Some agencies also lock down a few items of direct construction cost if it appears appropriate for their project. There are two major disadvantages of this model. First, agencies need to keep in mind that best-value is not low bid. However, the temptation to allow an unexpectedly low proposed cost to override the remainder of the evaluated criteria will be strong. Second, furnishing proposed costs demands that the project be developed to the point where reasonable numbers can be generated. This pushes CMR selection to appoint after design has begun, and reduces the opportunity for the CMR to add value through its input. Two-Step Best-Value Construction Manager-at-Risk Selection Model This model is based on the dominant form of DB procurement in transportation (“Design-Build Effectiveness Study” 2006), whereby the agency issues an RFQ and evaluates the qualifi- cations of the respondents. It then develops a short list of the most qualified firms and invites the short listed competitors to submit a proposal in response to an RFP. The proposal will include specified cost information as well as other items of interest to the agency’s selection panel. The cost-related por- tion of the proposal generally asks for the CMR’s proposed fees for preconstruction, construction, and sometimes its general conditions. This allows the owner to establish those costs on a competitively negotiated basis. It also allows those to carry forward to the GMP negotiations. The cost submittal may also contain elements of construction direct costs. This model requires a significant amount of front-end effort by the agency to administer the selection process. As with the One-step model, the competing contractors will need design information to generate proposed costs. Additionally, this model allows the agency to ask for possible design alternates during the selection process and use that information to iden- tify the best-value proposal. Figure 15 shows this model’s sequence and flow of information. Two case study projects used this model: UDOT and Memphis International Airport. The content analysis also found that 15% of the non-transportation documents were based on this model. Table 20 describes the contents of the solicitation documents that were common to both case studies, plus the cost-related items. Figure 15 illustrates the flow of this model. The two cases in this category are an interesting and infor- mative contrast in the way two agencies approach the cost features required in the selection process. Memphis asked for all the fees and general conditions of the CMR to be included in the proposal. Therefore, they locked these in and carried them forward into GMP negotiations. Memphis did not ask for any direct construction cost numbers at this point, but it 38 required the CMR to advertise and accept low bids from its subcontractors. Thus, this agency fixed the indirect costs and trusted in the competitive bidding process to furnish direct costs and competitive pricing. On the other hand, UDOT only asked for the preconstruc- tion fee and required the competing contractors to furnish unit prices for several high-value pay items. It expected the contractors to include their construction general conditions and profit in the unit prices it would furnish in the GMP. The unit prices contained in the proposal were carried forward into the GMP. UDOT’s rationale is as follows. Price is important in the selection process because: 1. Price motivates the contractor to think about the task and focus on the detail of what has to be done. Until the contractor has put a price to the task it is only an intel- lectual exercise. 2. Price brings cost competition and the innovation to deliver an affordable project. 3. Price documents costs and approach to cost that we can use in negotiations. 4. Contractors who honor their cost proposals show accountability and we can use their commitment to a reasonable price in the selection for future projects. 5. Price demonstrates to the public our stewardship over public funds (Alder 2007). Although UDOT has the most experience implementing CMR, it has still not finalized its selection process, as evi- denced by the following from the same report: We need to develop the price component of the selection process to create a stronger link between the items asked for in the selection process and costs in the negotiation process. The price tool also needs to be developed in a way that allows us to select the contractor earlier in the process even as soon as the design con- sultant. This will enable contractor input at the beginning of the design process before too many design decisions are made (Alder 2007). In summary, this model seeks to reduce the cost to industry of preparing a full proposal by splitting the procurement into its two steps. This method also furnishes competitive pricing of CMR fees and potentially major bid items. A word of caution is needed about this particular feature. UDOT very consciously keeps the pay items selected to a minimum and picks only those that it believes are potential budget busters. It would be easy and not perceptive to ask for more and more direct cost pricing and erase the benefit of not forcing the contractor to price undesigned features of work, the benefit of a dramatically reduced contingency from those measured in DB projects (Armstrong and Wallace 2001). The content analysis was nearly silent on this issue with only 6 transporta- tion and 11 non-transportation documents requesting proposed indirect cost and fee information. None of the 54 documents reviewed asked for construction direct cost information.

39 CMR Request for Qualifications CMR’s Statement of Qualifications Evaluation of Qualifications Shortlist Selection Contents of CMR Proposal Contains Only What Was Requested in RFP. This Graphic Shows Possible Submittal Requirements Award CMR Contract Preconstruction Services Develop & Negotiate GMP GMP Reasonable? Award Construction Contract Yes CMR Request for Proposals CMR’s Proposal CMR Proposal Evaluation CMR Selection Proposed Precon Fee Proposed Construction Fee Proposed General Conditions Proposed Construction Costs as req’d Contents of CMR Proposal Proposed Construction Schedule if req’d Contents of CMR Statement of Qualifications Contains Only What Was Requested in RFQ. This Graphic Shows Possible Submittal Requirements Organization Chart Corporate Project Experience Key Personnel Experience Required Management Plans Contents of CMR SOQ Other Submittal Requirements Construction No Advertise DBB Unable to Agree on GMP Yes Propo sal Com ponents U sed in G M P FIGURE 15 Two-step best-value CMR selection model.

CONTENT ANALYSIS REQUEST FOR QUALIFICATIONS/REQUEST FOR PROPOSAL SUBMITTAL REQUIREMENTS The content analysis indicates that there are a number of requirements for the submittal of proposals (see Table 21). The most predominant requirements are similar between the transportation and non-transportation documents and include organizational charts, past experience, references, 40 qualifications of key people, and project plans. Submittal requirements that appear more predominantly important in non-transportation projects than transportation projects include the proposed fees. In addition to the qualifications of the project manager, preconstruction services manager, superintendent, estimator/ scheduler, and quality manager, qualifications were also RFP Requirement No. Agencies Requiring Organizational structure/chart 2 Qualifications of the CMR’s project manager 2 Past CMR project experience 2 Past project experience (non-CMR) 2 References from past projects 2 Qualifications of CMR’s general superintendent 2 Subcontracting plan 2 Construction traffic control plan 2 Preliminary project schedule 2 Proposed preconstruction fee 2 Proposed construction fee 1 (MEM) Proposed general conditions 1 (MEM) Rates for self-performed work 1 (UDOT) MEM = Memphis International Airport; UDOT = Utah DOT. TABLE 20 TWO-STEP BEST-VALUE PROPOSAL SUBMITTAL CONTENTS Requirement No. of Transportation Documents No. of Non-transportation Documents Organizational structure/chart 20 16 Past CMR project experience 23 23 Past related project experience (non-CMR) 18 22 References from past projects 18 23 Qualifications of the CMR project manager 10 10 Qualifications of the CMR’s preconstruction services manager 1 4 Qualifications of the CMR’s general superintendent 4 11 Qualifications of the CMR’s estimator/scheduler 2 7 Qualifications of the construction quality manager 1 3 Qualifications of other key personnel 22 19 Construction quality management plan 5 10 Safety management plan 3 7 Schedule control plan 4 9 Sustainability certification plan 1 5 Construction traffic control plan 0 1 Other key project plans 14 17 Preliminary project schedule 4 5 Declaration of self-performed work 2 7 Subcontracting plan 9 8 DBE plan 4 9 Proposed preconstruction services fee 4 11 Proposed construction fee 4 11 Proposed post-construction fee (profit) 1 0 Proposed general conditions fee 2 6 List of proposed subcontractors 0 1 Claim reduction and claim resolution plan and dispute resolution 1 2 Claim history and litigation 7 12 Contract or subcontract default history 7 10 Current workload 7 10 Adequate bonding capacity 14 12 DBE = Disabled Business Enterprise. TABLE 21 CONTENT ANALYSIS PROPOSAL SUBMISSIONS

41 requested for the security information manager, public involve- ment specialist, utility manager, traffic control manager, and safety manager. Often the RFQ was general and would simply ask for the qualifications of key personnel or the people who would be working on the project. Key plans that are not individually listed in Table 21 include environmental management, cost control, project management, individual phases of the project, project staffing, request for information approach, value engineering, coordination of design solutions with architect and other consultants, commu- nication, problem solving, design review, utility coordination, public involvement, phasing, maintenance of traffic (MOT), document control, dispute resolution, and apprenticeship/ training program. These are sometimes asked for in a narrative form, such as, “Describe systems used for planning, schedul- ing, estimating, and managing pre-construction and construc- tion phase services (AZ document).” Other types of submittals contain a wide range of items, including: • Professional and contractor licenses; • Identification of challenges that might be faced on the project and how to address these challenges; • Unique qualifications of the company to perform the work; • “Approach to Price—Bidders will describe risk associ- ated with each unit price. Bidders will provide reasons and justifications on what would cause prices to either increase or decrease. These reasons and justifications will be the basis for future price negotiations. Identify how raw material cost variations would affect your unit price; in your response consider addressing how the following will affect unit prices—schedule, maintenance of traffic, risk, innovation, substantial changes in quan- tities, etc.” (UDOT). • “In conjunction with your team’s approach to the proj- ect, your team may have some innovative ideas that may or may not meet the technical information pro- vided that could increase the likelihood for success with this project. The selection team will consider how well your innovative ideas help balance the goals of the project” (UDOT). • “Share 2 significant lessons learned on innovative deliv- ery projects and provide a narrative on how you would approach the issues differently, or apply the lessons learned in those experiences on this project” (ODOT). • “Knowledge of market volatility as it relates to con- struction commodities and long range estimating skills” (Parker, Colorado). • “Why is it in the City’s best interest to select (the sub- mitting firm) for the project” (Golden, Colorado)? • “Awards: List awards, recognition received, or any other achievements that demonstrate your commitment to construction excellence. Any unsolicited letters of appreciation from owners, professional organizations, or regulatory agencies for any outstanding action per- formed by the contractor over and above the contract requirements may be submitted” (Denver, Colorado). SELECTION PROCESS DETAILS The selection process can be broken down for analysis into four stages. The first is the timing of the CMR selection with respect to the selection of the designer. Next is the composition and duties of the CMR selection panel. The third stage is the evaluation itself, including interviews if appropriate and the final stage deals with the algorithm used to select the winning contractor. Information for this section was drawn from all the study instruments. Timing of Construction Manager-at-Risk Selection The UTA’s CMR policy states that: “Prior to or concurrently with selection of the final designer, the CM/GC is selected based upon criteria that allow the owner to evaluate past performance, quality, and price” (“CM/GC Peer Review Meeting” 2003). This represents one end of the selection timing and is one that was advocated by most of the inter- viewed contractors in this study. The rationale is this permits “the participation of the construction manager in design and phasing decisions so that “unbuildable” or costly design details or phasing plans may be avoided and design/drawing inconsis- tencies may be limited” (Rojas and Kell 2008). Early selection of the CMR also allows input “very early in the process when there is little information available but decisions must be made that affect the future of the project, what is the max- imum increase in project cost that could be justified by the value added by faster project delivery” (Reinschmidt and Trejo 2006). The United Kingdom Highways Agency uses a project delivery approach that is similar to but not the same as CMR. It is termed “early contractor involvement” (ECI) and it brings the constructor to the project during the planning and permitting process (Early Contractor Involvement . . . 2004). This system actually allows the constructor to assist the owner in the designer selection process. That is not unheard of in U.S. CMR projects. The content analysis found one non- transportation project where this was required. Many private commercial building owners advocate this type of process (Ketcham and Epps 2000). NCHRP Synthesis 379 draws the distinction as follows: “. . . early contractor involvement may be most influenced by project complexity, whereas construc- tion manager at risk is most influenced by project type” (Anderson and Damnjanovic 2008). The top four advantages cited in Table 1 all deal with getting the constructor involved at an early point in the project delivery process. Therefore, the wisdom that early selection provides viable benefits is supported by both the ECI model and the CMR process. UDOT’s opinion is that the selection process can be configured in a “way that allows us to select the contractor earlier in the

process even as soon as the design consultant . . . before too many design decisions are made” (Alder 2007). Three Table 2 citations listed “picks CMR early in the design process” as a disadvantage. These encapsulate a second school of thought that holds that the constructor needs some design detail to work with to be able to develop a reason- able and realistic proposal. This is particularly cogent when the contractor is being asked to furnish direct cost pricing in addition to fees and general conditions. Studies in the build- ing sector recommend that the CMR be selected at around 30% design (Carlisle 2006) to give the CMR enough infor- mation to be able to furnish realistic subcontracting plans, preliminary schedules, etc. However, building projects are often much more complex in terms of the number of sub- contractors, number of systems to coordinate, and the level of architectural detail. Therefore, it makes sense to wait until preliminary designs are complete in that sector. The three Table 2 disadvantages cited all related to the accuracy with which the CMR can furnish cost information. One of those was the UDOT report (Alder 2007) and because UDOT asks for unit prices in the proposal, lack of design detail makes accuracy at that stage more difficult. Therefore, by bringing all of that above information together, the conclusion on selection timing becomes one of determining how much cost information the agency wants to receive and what the demands for accuracy are at the time of selection. The overwhelming sense found by all study instru- ments is that the best practice is to select the CMR as soon as practical given the requirements of the project. This leads one to question the reason the agency needs cost information before selection is made. There are at least five possible reasons: 1. The agency’s statutory constraints require pricing to be included in the selection decision. 2. The agency prefers to eliminate the reasonableness of construction fees and general conditions as a point negotiated in the GMP. 3. The agency believes that receiving direct construction cost data in the proposal allows it to shed risk for escala- tion and other potential reasons for cost growth. 4. The agency wants to have competition among the contractors for the cost of preconstruction services. 5. The agency is seeking to demonstrate price competition in the selection process. To sum this up, the point at which the CMR is selected in the design development process does make a difference. Early selection gives the CMR the most time to have a bene- ficial impact on the final design (Carlisle 2006; Ladino et al. 2008). If cost and price factors are going to be included in the selection process, then selecting the CMR after enough design progress to furnish a good picture of the scope of work would be indicated. If direct construction cost information is required, then CMR selection can be timed to happen when the agency 42 has sufficient design detail to permit reasonable numbers to be submitted that do not contain excessive contingencies. Selection Panel Composition and Duties Empanelling a selection committee is very straightforward; there are two primary decisions to be made. First, that the panel has the technical expertise and experience to make an informed decision. This usually requires assigning agency members from both design and construction to cover those areas. The second decision is whether or not to involve members from outside the agency. Typical external members are representatives from the local contractor’s association whose charter is to ensure that no favoritism is allowed to influence the decision: Members from third-party stakeholders that have a vested interest in facilitating the process and will add value to the decision with their specialized expertise, members of the impacted community to give the public a voice at the table and, finally, the design consultant with whom the CMR collaborates. Often the composition of the panel is defined by the agency’s enabling legislation. For example, Arizona law requires the panel to have a minimum of three, but not more than seven members. They include a licensed contractor, an agency senior manager, and one licensed design professional (Ladino et al. 2008). The UTA Weber County project added a member of the railroad company with which they shared track and a representative from the state environmental agency. Including a specific external member such as the railroad in the UTA case study creates “buy-in” to the process that UTA was later able to trade on when it needed to negotiate a right- of-way swap to facilitate a value engineering change. To loosely paraphrase the interviewee, “the railroad was with us every step of the way and as a result understood the real value of the changes we asked for and why they were important. They gave nothing away, but were more amenable to dis- cussing issues with the CMR than if they had not been included in the selection process.” If the designer is selected before the CMR, then an oppor- tunity exists to include it on the selection panel. Eight of the case study projects opted for this method. The Florida and Oregon DOTs did not involve the designer. Florida’s selection system is the most uncomplicated of the case study projects; FDOT merely asks the competing contractors to answer a set of pre-published questions at the in-person interview. So with those constraints, the designer would add no value to the process. Oregon uses an RFP process with a formal inter- view and stated that although it did not involve the designer in the case study project, it will in the future. The project was their first CMR contract and their procedures are still evolving based on their experience from the Willamette River Bridge project. Table 22 shows the tasks that were assigned to the designer and their frequency of occurrence by selection model when it was asked to assist the selection panel. Seven agencies assigned the designer to the panel and of those two (MDOT

43 and Pinal County) gave the designer voting privileges. The remaining agency (Alaska) merely used the designer’s services on behalf of the panel. The content analysis only found one transportation project and one non-transportation project that specifically indicated that the designer would assist the owner in CMR selection and neither spelled out the nature of that assistance. The lack of information in the content analysis and Table 22 shows that the case study agencies do not ask much from their designer in the way of assistance during the selection process. Evalu- ating the CMRs’ qualifications is the only task that was reported by more than two agencies. Construction Manager-at-Risk Interview Process One of the authors of this report had the privilege of being a member of the selection panel of the first CMR contract awarded in Oklahoma. The agency issued a letter to the four contractors on the short list to appear at its office for an inter- view. No specific details were included regarding the infor- mation that the agency wanted to evaluate in the interview. The agency head believed that the constructors “knew” what to do, the same as the design consultants. The contents of the four interviews ranged from an exceptionally long formal presentation of the contractor’s qualifications including video- taped testimonials and animated fly-bys of past projects to one firm that walked in, took their places at the table, and asked what questions the panel needed answered. The lesson learned for subsequent selection panel experiences is that the agency may specify exactly what it wants the CMR to present and make that information available at the interview for further discussion. FDOT’s list of published questions confirms the wisdom of that lesson. The problem with ambigu- ous requirements for interview content is that it makes it extremely difficult to compare apples to apples. In the example just given, the panel had to differentiate between a competitor that gave too much information and a competitor that offered nothing, not to mention the other two that fell somewhere in the middle. As a result, the panel finally decided to discount the interview ratings and select the CMR based on the written material in their SOQs. The previous discussion leads to the idea that the agency determines exactly what it is that it wants to get from the interview. The CMR literature was silent in this regard. The most relevant writings were in the DB area, where the purpose of the interview is quite different. However, the structure of that mechanism can probably be replicated. A report on DB project delivery (Gransberg et al. 2006) recom- mends that the owner specify not only the agenda of the interview but also the people that the owner would like to have make the presentation. It also expands on the depth it would like to see in each agenda item, noting any specific information that is particularly important to the selection process. For example, three of the case study projects asked the contractors for a “critical analysis” of the project budget in their proposals. If there were questions or the need for more information in the submitted analysis, the agency could then indicate in the letter announcing the interview schedule that it expected the competitors to justify their budget analy- sis as part of the presentation and bring facts and figures that could be shown if necessary. The content of the CMR interviews was not consistent among case study projects. On one end, Pinal County (One- step QBS model) required a very comprehensive presenta- tion of the contents of the SOQ. The other end was FDOT with its list of questions and nothing else. ODOT focused its interview on presentation of project issues and the CMR’s proposed preconstruction service package, relying on the written proposals for the remainder of the evaluation. Table 23 shows the case study results for interview content alongside the results of the content analysis. This juxtaposition is impor- tant because the solicitation documents are the first form of project communications between owner and contractor. The number of instances where the interview was mentioned in the solicitation documents was high (24 transportation, 25 non-transportation). That it was included this frequently in the RFQs and RFPs shows that it does have a place at that point in the process. The formula for success in CMR project delivery is in developing a sense of trust through free-flowing, information- rich communications (Gambatese 2002; Doren et al. 2005; Storm 2007). Detailing the requirements for the interview in the initial solicitation document allows the competitors to not only prepare for it as early as possible but also to craft their written submission in a manner that compliments their oral presentation. Lastly, it creates a “no surprises” atmo- sphere at the outset of the process and reinforces an “open book” approach to the entire process, which “eliminates Designer Selection Panel Responsibility 1-Step QBS (out of 4) 1-Step BV (out of 2) 2-Step BV (out of 2) Voting member of panel 2 0 0 Non-voting member of panel 1 2 2 Developing short list 1 0 0 Evaluation of CMR qualifications 4 0 1 Checking CMR references 2 0 1 Evaluation of CMR interviews/presentations 2 0 1 Evaluation of CMR fees 1 0 0 TABLE 22 DESIGNER SELECTION PROCESS ASSISTANCE DUTIES

hidden agendas. . . . and creates a ‘win-win-win’ environment” (Ladino et al. 2008). Of the 49 solicitation documents that indicated there would be an interview only 27 went on to detail the content of that interview. An example taken from a CMR RFP for the Washington State Ferries’ (WSF) multi-modal terminal project (a two-step best-value model) illustrates how the content of the interview can be included in the solicitation. Should your firm be invited to an interview, questions will be directed solely to the proposed key project staff. At a minimum, the corporate executive dedicated to the project, the project man- ager(s), the project superintendent(s), project estimator(s), and the key individuals responsible for pre-construction services shall be in attendance. In addition to presenting their qualifica- tions and experience and the Project Team’s approach to the project, the interviewees will be expected to respond to questions from the panel regarding the firm’s proposal as well as addi- tional questions that might be posed in correspondence to the “most qualified” Proposers invited to an interview (“Request for Proposals . . .” 2005). The RFP for the UDOT case study project also indicated the content of the interview. This document was more concise than the WSF description. UDOT merely indicated that the formal presentation includes: “Understanding of the work, approach to the project, schedule control, [and] management of project.” A UDOT RFP for another CMR project asked for 44 a different content: “Constructability, MOT [maintenance of traffic], business impacts, utility relocations, [and] ROW [right-of-way].” The point to be taken is that UDOT tailors each interview to focus on the issues that are of most concern to selecting the best CMR for a given project. Selection Process Decision Procedures It is important that the process used to identify the winning CMR be transparent, logical, and defensible (Shane et al. 2006). Figure 16 provides a model illustrating the components of the selection system for QBS and best-value selection systems. It shows that the design of a means to identify and select a CMR using QBS or best-value begins with determining the param- eters that will be evaluated to make the CMR selection decision. Once established, the parameters guide the development of the evaluation criteria against which each SOQ or proposal will be measured, the method used to actually rate a given criterion, and the algorithm used to determine the winning competitor to whom the CMR contract will be awarded. Parameters are drawn from the benchmarks and constraints inherent in the given project and come from five general areas: 1. Cost parameters: includes any and all items that are expressed in financial terms, such as a preconstruction services fee. Case Studies Content Analysis Selection Process Topic 1-Step QBS (out of 5) 1-Step BV (out of 3) 2-Step BV (out of 2) Transport Documents (out of 25) Non- Transport Documents (out of 29) Formal presentation of corporate qualifications/past projects 4 2 2 1 2 Formal presentation of qualifications/past project experience for key CMR personnel 3 2 2 1 3 Formal presentation of project- specific issues 1 3 2 13 1 Formal presentation of preconstruction services components 2 3 2 0 0 Other: Answer pre-published list of questions 1 0 0 0 0 TABLE 23 CONTENT OF THE SELECTION PROCESS INTERVIEWS Best- Value Parameters Best-Value Evaluation Criteria Best-Value Evaluation Rating Systems Best-Value Award Algorithms QBS Parameters QBS Evaluation Criteria QBS Evaluation Rating Systems QBS Award Algorithms FIGURE 16 CMR selection framework (adapted from Scott et al. 2006).

45 2. Time parameters: covers all items that either define or affect the project’s schedule. A preliminary schedule or a list of long-lead time items would fall here. 3. Qualifications/performance parameters: includes both the personal qualifications and experiences of key project personnel and the corporate experience of the entities that will complete the construction. The subcontracting plan is an example of this type of parameter. 4. Quality parameters: covers all aspects of the project’s quality management system including mechanisms to ensure long-term quality such as warranties. Most are not biddable in that they cannot be converted to a dollar or time amount. 5. Design alternates parameters: covers any aspects of the project’s preliminary design (if there is one) that the con- struction contractor is allowed to either propose different solutions or alter the existing solution. Proposed resizing of structural members to accommodate the limitations of the contractor’s equipment fleet is an example. Once the parameters are established, evaluation criteria can be developed to provide the agency with a logical and defensible means to differentiate between competing SOQs and proposals. These are connected with a system to assign a rating to each criterion to ensure a uniform evaluation of all competitors across the selection panel. There are four types of rating systems in use in the United States for this application (Scott et al. 2006). • Satisficing (more commonly called “Go/No Go”): To use it, the evaluator establishes a minimum standard for each and every evaluation criterion against which the proposals can be measured. • Modified Satisficing: recognizes that there may be degrees of responsiveness to any given criteria. As a result, the range of possible ratings is expanded to allow an evaluator to rate a given category of a proposal across a variety of degrees. Thus, a proposal that is nearly responsive can be rated accordingly and not dropped from the competition as a result of a minor deficiency. Additionally, a proposal that exceeds the published criteria can be rewarded by a score that indicates that it exceeded the standard. • Adjectival Rating: These systems use a specific set of adjectives to describe the conformance of an evaluated area within a proposal to the project’s requirements in that area. • Direct Point Scoring: evaluation allows for more rating levels and thus may appear to give more precise distinc- tions of merit. However, it may lend an unjustified air of precision to evaluations. Evaluators assign points to evaluation criteria based on either some predetermined scale or the preference of the evaluator (Scott et al. 2006). Using the previous WSF example, this CMR project’s evaluation criteria and their direct point ratings that show each criterion’s weight are as follows: • Ability of professional personnel and qualifications of the firm (9 points). • Past performance on GC/CM projects (5 points). • Past performance of the firm in completing similar projects (6 points). • Ability of the firm to meet time and budget requirements (5 points). • Location (2 points). • Recent, current, and projected workload, and capability of the firm including financial stability (5 points). • Concept of the proposal (5 points). • Accident prevention program (5 points). • Preconstruction services (6 points). • QC (2 points). • Interview (35 points). • Final proposals [proposed costs] (15 points as presented in Table 24) (“Request for Proposals . . .” 2005). The final element is the award algorithm used to identify the winning candidate. These can range from algebraic for- mulae that combine direct points with weighted categories to the mere subjective evaluation of the ratings by the panel. For a detailed discussion on the rating, evaluation, and award process for alternative project delivery methods, the reader is referred to NCHRP Report 561: Best-Value Procurement Methods for Highway Construction Projects (Scott et al. 2006). Completing the WSF example, the best-value award algorithm is described in the RFP as shown in Table 24. This agency uses the “lowest conforming proposal” as the benchmark against which the others are measured. The lowest conforming pro- posal is the one that proposes the lowest combined cost of fees and general conditions and is rated as satisfactory in all the previous evaluation criteria. WSF uses direct point scoring, but it also includes a “fatal” deficiency for each evaluation criterion that ensures that the proposals are fully responsive. All but one of the case study projects (Texas Tech) used direct point scoring similar to the WSF example, and of those, only FDOT did not adjust the point scores by measures of individual importance or weights. Florida merely added up the total score for each competitor. Using the weighted criterion award algorithm (Scott et al. 2006) requires that the agency rank each evaluation criterion in order of importance. This can be accomplished two ways. The first is as shown in the WSF example where each criterion is assigned a specific number of possible points. Thus, the criterion with the most assigned points is the most important factor in the evaluation. The second way is to assign a fixed number of rating points to each criterion and then differentiate between them by assigning a weight. These weights are often expressed as a percentage. Then a standard arithmetic weighted average is computed by multiplying the score in each criterion by its weight and adding the total sum. Regardless of which method is used, the agency furnishes the relative importance of each criterion to the competitors so that the contractors can craft their proposal in a manner that makes then highly responsive to the project’s needs. The weighting scheme in the WSF

example shows that the most important factor in the evalua- tion is the interview rating, followed by the proposed costs and the qualifications of the CMR team. Therefore, to be as competitive as possible, a contractor will need to ensure that the five specific individuals proposed to be assigned to this contract are present at the interview. The WSF scheme also indicates that the proposed fees and other costs are more important than the qualifications of the team, leading to a potential trade-off between lower cost, less experienced individuals and/or perhaps an effort to minimize general conditions costs by keeping the project QC staff at a minimum to reduce the proposed fee. Therefore, it is extremely important that the agency make a deliberate effort to assign points and weights to each criterion in a manner that accurately reflects their contribution to project success. The Memphis, Oregon, and Utah case study projects all used cost as a selection criterion. Memphis weights cost at approximately 25%. Oregon gave cost a weight of 15%, whereas Utah assigned a higher weight that ranges from 26% to 50%, but usually averages 30%. The content analysis found that 12 transportation and 12 non-transportation documents showed that cost was included in the selection decision. Only two disclosed the assigned weight of cost factors and they correspond to the weights found in the case study projects. This lack of transparency is disturbing and can lead to protests based on an agency not awarding to the proposal with the lowest costs (Parvin 2000). SELECTION PROCESS PROTESTS When a public agency attempts to implement a procurement method that does not depend on awarding to the lowest bid, it will open itself up to criticism unless staff carefully devises a system where it can prove, if necessary, that favoritism did not enter into the contractor selection decision. One author, a lawyer serving the construction contracting industry, discussed 46 the legal view of the need for fair and open evaluation processes in the transportation industry. In it, he highlighted the follow- ing two points that are of interest in this synthesis: 1. Clearly state the evaluation criteria and weight given for each item and ensure that the evaluation team uses them. 2. Leave no doubt about the honesty and integrity of the public agency’s evaluation team (Parvin 2000; italics added). The commentary argued that without a transparent evalua- tion plan, the owner would find itself constantly defending award protests. A case study that involved a protest of a best- value selection system is presented next to provide context to the discussion of this critical issue. Minnesota Department of Transportation Protest Case The Minnesota DOT (MnDOT) successfully defended itself and its best-value selection process from a protest on its first project delivered using an award that was not based on low bid. The case presents a number of rules for an agency that wants to implement QBS or best-value selection of CMRs to abide by while developing the details of their proposed system. Although this case revolved around a DB project, its results are generally applicable to any type of nontraditional selec- tion process that uses contractor qualifications in its evaluation plan. In this case, MnDOT chose to use a two-step best-value procurement to select a DB contractor for a $110 million project (Shane et al. 2006). The first step of this method is to evaluate the qualifications of interested contractors and develop a short list comprised of the best qualified group. The RFQ stated that “The selection team shall evaluate the DB qualifi- cations of responding firms and shall compile a short list of no more than five most highly qualified firms in accordance with qualifications criteria described in the RFQs” (Shane et al. 2006). To accomplish this task, it established and pub- Rating System for Final Proposals The firms/teams that the Selection Committee believes to be most qualified based upon their evaluation of the submittals received in response to this Request for Proposals will be requested to submit Final Proposals for fee and specified general conditions that will be evaluated as follows. Low Conforming Proposal 15 points Proposals within 5% of Low Proposal* 12.5–14.5 points Proposals within 10% of Low Proposal* 7.5–11.5 points Proposals within 15% of Low Proposal* 0–6 points Others 0 points * Computed as follows: Proposal being evaluated – Low Conforming Proposal = % Low Conforming Proposal For each percent that the proposed sum of fee and general conditions work is higher than the low conforming proposal deduct ½ point up to 5%, one point up to 10%, and 1.5 points up to 15%. Final Award Algorithm The firm/team with the highest scoring proposal resulting from the Selection Committee’s evaluation of the submittal, the interview, and the final proposal will be selected for MACC [GMP] Negotiations. (Total Possible: 100 points). In the event of a tie in total points, the firm/team with the lowest conforming final proposal will be selected. TABLE 24 WASHINGTON STATE FERRIES (2005) CMR AWARD ALGORITHM

47 lished a set of evaluation criteria and a method for scoring each potential competitor as shown in Table 25. Within the “Organization and Experience” category, the evaluation criteria were further defined to include: • Effective project management authority and structure • Design and construction management structure • Effective utilization of personnel • Owner/client references • Experience on projects of similar scope and complexity • Experience with timely completion of comparable projects • Experience with on-budget completion of comparable projects • Experience with integrating design and construction activities • Experience of DB team members working together. The RFQ asked the contractors to both describe their specific DB experience by listing at least one completed DB project and to list other projects “with scope comparable” to the project on which they were competing. It went on to state that DB experience would be considered but was not required. Issues at Stake in the Protest Five contractors responded to the RFQ, submitting their qual- ifications in accordance with the RFQ, and were evaluated as described in that same document. Scores ranged from 69.4 to 85.7, with three contractors being rated above 80. Those three were then announced as comprising the short list of “most highly qualified firms” and deemed to be qualified to continue in the competition. The fourth ranked firm with a score of 71.9 filed a protest citing the following reasons: • MnDOT violated the state DB statute by requiring that the evaluation rely on and emphasize previous DB experience, which would restrict competition (as this was MnDOT’s first DB project and therefore no local firms had DB experience); • MnDOT engaged in unpublicized rule making; and • That the judgment regarding who was short listed was arbitrary and capricious fact finding and the conclusions were not substantiated by the evidence (Shane et al. 2006). The court overturned the protest and upheld the validity MnDOT’s evaluation system for this specific project. First, it found that considering DB experience for MnDOT’s first DB project was entirely reasonable and, as it was considered but not required, it did not restrict competition. Next, it found that as MnDOT published the details of its qualification eval- uation plan and as those rules applied only to a single project, the process did not constitute “unpublicized rule making” as alleged and, finally, it found that MnDOT had followed its procedures exactly as they were published and had a ratio- nal basis for justifying their short listing decision. Therefore, the process was not “arbitrary and capricious.” Lessons Learned for Implementing CMR Selection Programs This court test yielded some excellent information for agen- cies that plan to use CMR project delivery on a general scale. MnDOT won this case for three major reasons: • The prequalification evaluation criteria were transparent to all offerors, • The owner followed its prequalification plan as published, and • The owner could defend its decision logically. In this case, MnDOT clearly articulated the definition of contractor qualification. It helped its case by stating that it would narrow down the field to a short list of the “most highly qualified firms in accordance with qualifications criteria described in the request for qualifications.” The requirements for prequalification were clear, and each potential competitor could compare itself with the competition and make an informed decision as to its ultimate competitiveness in the known field of players. This transparency serves to reduce the element of subjectivity that is inherent in best-value award evaluation systems by spelling it out rather than hiding it. Second, as the requirements for prequalification were clear, it was easy for the courts to find that MnDOT had followed its own evaluation plan. This speaks to the first point in the earlier Parvin quote. Once an owner publishes its prequalification program, it loses all flexibility in applying it to the competitors that respond. If it wants to be able to defend against a protest, it exhibits that it follows its own rules. If it does, the second Parvin requirement regarding leaving no doubt about the honesty and integrity of the evaluation team is satisfied. Finally, the use of the terms “most highly qualified” gave MnDOT great latitude about determining the final size of the short list. The resultant scores showed that three competitors fell within a range of three points of each other, whereas the plaintiff was 10 points below the third highest score. This created a solid argument that the three competitors on the short list were indeed the most highly qualified. MnDOT Prequalification Evaluation Criteria Possible Points Legal and Financial Qualifications Pass/Fail Organization and Experience 15 Key Personnel 30 Project Understanding 10 Project Approach 25 Project Management 20 Total 100 TABLE 25 MNDOT DESIGN-BUILD PREQUALIFICATION EVALUATION CRITERIA

was unintentionally doing the plaintiff a favor by not being conservative in forming the short list with four firms. The cost of preparing a technical and price proposal for $110 mil- lion project would probably be in the range of $300,000 to $500,000 (Shane et al. 2006). If MnDOT had arbitrarily set a minimum point score to be considered “qualified” of 70 (i.e., minimally qualified), then three instead of two firms would have had to invest that significant amount of money in a losing effort. Thus, the logic of short listing only the “most highly qualified” instead of all the minimally qualified is compelling in an economic sense and in the long run is fairer to industry. Thus, several lessons can be drawn from this case and applied to an agency’s CMR program. First, the owner publishes transparent prequalification criteria along with its procedures for using the input from contractor’s proposals in determining the outputs of the evaluations. This puts all the contractors on an even footing and makes the defense against a possible protest stronger. Second, once published, the owner follows its evaluation procedures to the letter, collecting documentation along the way to prove that the decisions made for the project flow directly from the published evaluation plan and its attendant criteria. Finally, the CMR selection program is logical and the decisions that flow out of it are based on defensible logic. Avoiding Protests “The ability to prequalify and select your construction man- ager on the basis of its reputation and record in controlling costs, meeting deadlines, and satisfying customers” (Rojas and Kell 2008) is a benefit of CMR project delivery touted in the literature (Kwak and Bushey 2000). Therefore, an agency would not want to invalidate its ability to accrue this benefit by promulgating a procurement system that is plagued by protests of award. Taking the lessons learned in the MnDOT case leads to possible practices that can be followed to make an agency’s CMR selection system as immune to protest as possible. The first is the assurance of evaluation and award transparency. By publishing the details of its evaluation plan, the competing contractors have an opportunity to measure their competitive potential against that of the others in the race. It allows them to make business decisions on whom to propose for key positions and how, if successful, this project will be coordinated with the contractor’s other work. Finally, it provides a time for questions about the system. This period serves to highlight to the agency those aspects that the con- struction community, through its questions, does not fully understand. If a major problem with interpretation arises, the agency can move to ameliorate the problem before it becomes a dispute. Once published, the agency can train its evaluation panel to ensure that the evaluation proceeds in accordance with its published plan. FDOT does just this. As previously mentioned, it restricts its interview process to the answering of a pre-published list of questions, scores the answers, and selects the CMR with the highest score. An interview with a 48 contractor that does business with FDOT confirmed that there are no concerns about either selection equity or favoritism in the way they assess qualifications in their process (Gransberg and Riemer 2009). The second practice is to seek means to add probity to the process and ensure the credibility of the selection panel’s fairness and integrity. One method in use is to publish the names of the panel members in the selection documents. UDOT uses this technique and furnishes a list of all panel members and their affiliations. Knowing the personalities and professional qualifications of the panel members is valu- able to the contracting community for two reasons. First, in many if not most highway projects, the agency’s personnel are well known to the contractors that furnish it construction. Past experience with one of the panel members can directly influence the contractor’s choice of key personnel for a given CMR job. If a specific panel member has had a past conflict with one of the contractor’s superintendents, then a prudent business person would not assign that individual to this project. The same theory holds true for assigning the company’s best superintendent if there are panel members who can attest to the person’s competence and professionalism. Second, if the panel is staffed with members whose technical qualifications or construction experience is limited given the type of project under competition, then the proposers can take care to explain the technical/constructability aspects of their approach in a manner that can be understood by the evaluators. UDOT expe- rienced this issue with their first CMR project that entailed accelerated bridge construction methods and was able to alert the competitors merely by naming the panel. Another method to generate the appearance of fairness and to guard against favoritism is to empanel members of the construction and/or consulting engineering industries on the selection board. The Arizona DOT and UDOT require that all selection panels include a contractor and a design professional (Ladino et al. 2008). UDOT established an industry advisory board to assist it in implementing CMR in Utah. That board assisted UDOT in the development of its selection procedures and, as a result, a member of the AGC is on every selection board in addition to a local member of the ACEC. Each non- UDOT member is required to sign a statement promising not to reveal the contents of any of the competing proposals to ensure confidentiality of their contents. The minutes of the advisory board that had reviewed the performance of on-going CMR projects included this statement: The AGC representative should continue to participate in con- tractor selections. UDOT will develop a method to verify the confidentiality statement (Alder 2007). Instances of CMR transportation project protests are isolated as a result of the delivery method’s relatively recent appearance in the industry. The ODOT case study project had an unsuccessful protest lodged on the award of its first CMR project. The basis of the protest was that ODOT had failed to pursue clarifications to the scope of work that materially affected the scoring. Although the protest was denied, it did

49 point ODOT to areas in its CMR selection procedures that could be improved to avoid an appearance of unfairness. The building sector has much greater CMR experience and hence a longer period in which to make selection system design errors that encourage protests. The content analysis found three transportation and one non-transportation project where the procedure for protest was contained in the RFQ/RFP. It also found two non-transportation projects that stated that protest of the selection was not permitted. “The decision of the selec- tion committee will be final and no appeal will be allowed or permitted” (Parker, Colorado; italics added). The most com- prehensive study on the subject was done for the Washington State Joint Legislature Audit and Review Committee (Septelka and Goldblatt 2005). The Washington study found that in 82 CMR projects awarded over 14 years, 97% of the respondents had no protests; only two protests lodged, and they were 10 years apart. The first protest in 1993 was based on the state award- ing the contract to a contractor that was not the low bidder when price was one of the selection criteria. The protest was unsuccessful. Ten years later a protest was filed on an award where the basis for cost evaluation was to award to the “median proposal price” rather than the lowest price. In this case, the plaintiff had forgotten to include the stipulated preconstruction services fee in its total and as a result was low owing to an “obvious mathematical error.” This protest was also unsuccess- ful. The report concludes that “protests over the GC/CM selec- tion process have been rare” (Septelka and Goldblatt 2005). CONSTRUCTION MANAGER-AT-RISK SELECTION IMPACT ON COMPETITION One of the criticisms that have been raised is the potential that alternative project delivery methods such as CMR will reduce competition and decrease the ability for small and/or new companies to be able to compete with the larger compa- nies. The case study project contractors interviewed were asked to furnish their opinion on this question. Two-thirds indicated that the impact on the number of bidders was positive, and the other segment believed it would be worse. The two negative opinions were from contractors on a DOT’s first CMR project; therefore, their responses may have been influenced by the feeling of uncertainty that some members of the industry experience when the status quo is changed. The positive opinions came from contractors who work for agencies that have experience with multiple CMR projects and hence may have worked the bugs out of their system to the point where the industry is no longer believed to be uncomfortable with the change from DBB. The response from the national survey of 47 DOTs resulted in 8 DOTs that believed their local construction industry supported CMR and 8 that believed that there was no support for CMR. The majority, 31, had no opinion on that question. That group contained 15 respondents who indicated that they did not know what CMR project delivery was. Interestingly, none of the respondents indicated that industry opposition was the reason they did not use CMR in their state. Ignorance of the method and lack of enabling legislation were the two major reasons for non-use across the country. The case study agencies were not asked to comment on CMR impact on competition; however, some information directly related to that aspect was collected, and it furnishes a means to speculate about CMR project delivery’s impact on competition in the case study projects. Each agency was asked about its policy for establishing a short list. Eight of ten reported that they do short list and all indicated that their short lists are of three or more qualified contractors. Thus, they apparently are receiving at least three responsive proposals that would indicate that a reasonable level of competition is being achieved by those agencies. The two DOTs that did not short list on their projects were Alaska and Michigan. Alaska received only two proposals and evaluated both. The nature and size of the construction industry in Alaska may account for the low response rate. Michigan’s policies prohibit short listing, and this case study project evaluated five proposals. The Washington State study of CMR public building proj- ects specifically sought to identify if the project delivery method had a discernable impact on competition. The study found that projects typically received an average of five proposals and the number of firms that had not previously competed for a CMR project declined over time. It found that smaller companies (annual revenue under $100 million) were less competitive, with 8% of the projects being awarded to small construction contractors (Septelka and Goldblatt 2005). This study also measured a metric they termed “competition intensity.” It is defined as the ratio of the number of firms competing to the number of projects available. A number greater than one indicates more competition, and a number less than one indicates less competition. The study found that the mean competition intensity over 14 years was 2.59 and that it dropped below two in only three of those years. Therefore, the conclusion is that CMR project delivery did not negatively impact competition in that market (Septelka and Goldblatt 2005). SUMMARY The CMR selection process is an important component of a successful CMR project. It requires planning, forethought, and careful design to make it transparent, fair, and defensible. Conclusions The following conclusions were reached in this chapter: • The CMR procurement process can be standardized by using one of the three models presented in this chapter. • Making the CMR project delivery method selection decision as early as possible provides an opportunity to bring the contractor on board at a point where it has the maximum opportunity to add value to the project.

• Selecting the CMR as soon as practical maximizes its opportunity to add value to the project and to minimize wasted design effort. • The probability of protest is low and the agency can bear this in mind as it creates its selection process. • The impact on competition appears to be minimal and transient as the project delivery method matures in a given market. Effective Practices The following effective practices were reported in this chapter: • The agency can select which of the three CMR selection models best fits its statutory constraints and the project’s 50 requirements and then design a procurement process that is both transparent in its detail and defensible in its logic. • Publishing as much information as practical about the content of the selection process and how the competing contractors will be evaluated enhances the transparency of procurement and avoids the appearance of favoritism. • Publishing the role of the designer in the selection process as well as the required content of the interview, if there is one, reduces the probability of protest. Lessons Learned The MnDOT protest defense provides the main lesson from this chapter.

Next: Chapter Five - Preconstruction Services »
Construction Manager-at-Risk Project Delivery for Highway Programs Get This Book
×
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

TRB’s National Cooperative Highway Research Program (NCHRP) Synthesis 402: Construction Manager-at-Risk Project Delivery for Highway Programs explores current methods in which state departments of transportation and other public engineering agencies are applying construction manager-at-risk (CMR) project delivery to their construction projects.

CMR project delivery is an integrated team approach to the planning, design, and construction of a highway project, to help control schedule and budget, and to help ensure quality for the project owner. The team consists of the owner; the designer, who might be an in-house engineer; and the at-risk construction manager. The goal of this project delivery method is to engage at-risk construction expertise early in the design process to enhance constructability, manage risk, and facilitate concurrent execution of design and construction without the owner relinquishing control over the details of design as it would in a design-build project.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!