Construction Manager-at-Risk Project Delivery for Highway Programs (2010)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

100 APPENDIX B Case Study Project Questionnaire Two types of structured interviews were conducted in conjunction with this research. The first was with the case study project agency personnel. The second was with the contractors that built the case study CMR projects. The questionnaires are provided in the following sections to this appendix. CASE STUDY PROJECT AGENCY STRUCTURED INTERVIEW NCHRP 40-02: Agency Structured Interview Questionnaire CONDITIONS: This interview can either be conducted in person or via telephone. The following protocol shall be followed during its administration: 1. The questionnaire shall be sent to the respondent at least 2 weeks prior to the interview via e-mail. 2. Two days prior to the interview, a follow-up message with the questionnaire attached will be sent to confirm the date and time of the interview. 3. To maximize the quality and quantity of information collected, the primary respondent should be encouraged to invite other members of his/her organization to be present during the interview. Thus, a single transportation agency response can be formulated and recorded. 4. The interviewer will set the stage with a brief introduction that emphasizes the purpose of the research, the type of information expected to be collected, and the ground rules for the interview. 5. Once the interviewees indicate that they understand the process at hand, the interview will commence. 6. The interviewer will read each question verbatim and then ask if the interviewee understood the question before asking the interviewee to respond. 7. Each question contains a specific response that must be obtained before moving to the next question. Once that response is obtained, the interviewer can record as text additional cogent information that may have been discussed by the interviewees in working their way to the specific response. 8. Upon conclusion of the interview, the interviewer will ask the interviewees if they have additional information that they would like to contribute and record those answers as text. 9. The interviewer will assemble a clean copy of the final interview results and return them to the interviewee for verification. STRUCTURED INTERVIEW I. General Information: 1. City and state in which the respondent agency is headquartered: a. Name of agency: 2. What type of organization do you work for? 3. State DOT Other public transportation agency Other, please describe: 4. Annual construction budget: 5. Average annual number of projects: 6. Project monetary size range: $ to $ 7. Average monetary size of a typical project: $

101 Agency CM-at-Risk Project Delivery Experience Project Delivery Experience CM-at-Risk 1 Has your agency awarded a project under one of these project delivery methods? A. If yes, how many projects? B. If yes, what percentage of your total construction budget? C. How long have you used these methods? Yes No A. 1–5 6–10 >10 B. <10% 11–25% 26–50% >50% C. < 2 years 3–5 years > 5 years 2 Is your agency restricted on the use of these project delivery methods? A. If yes, what is the restriction? Explain “other” B. If yes, are you able to obtain a waiver for CM-at-Risk? Explain “other” Yes No A. Legislative Regulation Policy Other B. Yes No Other General remarks about agency program that might affect use of CM-at-Risk: 9. Does your agency use CM-at-Risk contracting to augment its existing workforce during program funding spikes? a. Yes No Please explain if necessary: 10. Do other public agencies in your state have authority to use CM-at-Risk? Yes No Please explain if necessary: 8. Percentage of out-sourced design effort versus in-house design: %

102 2. What project factors are considered when making the project delivery method decision? 3. Which of the following were reasons that your agency uses to select CMR delivery methods? Check all that apply. Which of the below is the single most significant reason for selecting CMR delivery method? (Interviewer circle the check box.) Reasons CM-at-Risk Reduce/compress/accelerate project delivery period Establish project budget at an early stage of design development Constrained budget Get early construction contractor involvement Encourage innovation Facilitate Value Engineering Encourage constructability Encourage price competition (bidding process) Complete different design solutions through the proposal process Redistribute risk Complex project requirements Flexibility needs during construction phase Third party issues (permits, utilities, etc.) Reduce life cycle costs Provide mechanism for follow-on operations and/or maintenance Innovative financing Encourage sustainability Project is a revenue generator Reduced agency staffing requirements Reduced agency review/inspection requirements Other (explain below) Project Factor Considered in decision Drives use of CMR Project monetary size Project budget control issues Project schedule issues Project technical complexity Project type (typical agency project vs. non-typical agency project) Project type (bridge vs. road project) Project technical content (i.e., ITS, seismic features, tolling equipment, etc.) Project location (urban vs. rural) Project environmental issues Project third party interface issues (utilities, business access, railroads, etc.) Project traffic control issues Project quality assurance requirements Project life cycle issues (maintenance/operations) Project sustainability issues Incentives for obtaining federal or state funding Project generates revenue (tolls, special taxes, etc.) Agency staff design review/construction inspection requirements Agency staff experience with delivery method Agency staff availability to oversee project development Desire to include specific innovation Other II. Case Study Agency Project Delivery Method Decision-making Information: 1. Who ultimately makes the project delivery method selection decision: Agency design personnel Agency construction personnel Agency upper management Entity outside the agency’s organization; Explain:

103 6. Do your project cost estimates involve an analysis of uncertainty (i.e., was a range cost estimate developed)? Yes No 7. Do you employ any formalized risk allocation techniques to draft the contract provisions? Yes No If yes, please describe: Interviewer: Collect sample copies of the provisions referred to in this question. Review them with the interviewee. III. Case Study Agency CM-at-Risk (CMR) Procurement Process Information: 1. At what point in the project design development process is the project delivery method decision made? 0% 1%–30% 31%–50% 51%–80% >80% actual percent 2. W hat type of procurement process do you use to advertise a CMR project? Request for Qualifications (RFQ) only [no proposed fees] Request for Proposals (RFP) [includes some or all fees] only RFQ + RFP Request for letters of interest Other, please explain: Interviewer: Collect sample copies of the documents referred to in this question. Review them with the interviewee. 3. Once CM-at-Risk project delivery has been selected, who does the design? In-house design personnel Consulting engineers Combination of both 4. Once CM-at-Risk project delivery has been selected, which entity is brought to the project first? the designer (including in-house design) the CMR 5. If the answer to 4 is “the designer,” does the designer assist in the CMR selection process? Yes No Other, please explain: 6. If the answer to 5 is yes, which of the below tasks is the designer involved in? Evaluation of CMR qualifications Checking/validating CMR references CMR interviews/presentations Developing short list Voting member of CMR selection panel Evaluation of CMR fees Non-voting member of CMR selection panel Negotiation of CMR fees Other, please explain: 7. If the answer to 4 is “the CMR,” does the CMR assist in the designer selection process? Yes No Other, please explain: 8. If the answer to 7 is yes, which of the below tasks is the CMR involved in? Evaluation of designer qualifications Checking/validating designer references Designer interviews/presentations Developing short list Voting member of designer selection panel Evaluation of design fees Non-voting member of designer selection panel Negotiation of design fees Other, please explain: 9. In your CMR selection process, do you develop a shortlist? Yes No 4. Please explain the process that you use to choose of the project delivery method for a typical project. 5. Is a formal risk analysis conducted on a typical project in any of the following areas? Project scope Project schedule Project cost Contracting risk

104 12. Which of the following pieces of information are required to be submitted in response to a typical RFQ/RFP/advertisement? Question 12 Matrix Required submittal? If YES: Is it evaluated to make the CMR award decision? If NO: Is it a required submittal after contract award? Do either the RFQ or the RFP require the following to be submitted as part of the CMR’s statement of qualifications or proposal? Yes No Yes No Yes No Organizational structure/chart Past CMR project experience Past related project experience (non-CMR) References from past projects Qualifications of the CMR’s Project Manager Qualifications of the CMR’s preconstruction services manager Qualifications of the CMR’s general superintendent Qualifications of the CMR’s estimator/scheduler Qualifications of the Construction Quality Manager Qualifications of other key personnel (list below) Construction quality management plan Construction traffic control plan Other key project plans (list below) Preliminary project schedule Declaration of self-performed work Subcontracting plan DBE plan Proposed preconstruction services fee Proposed post-construction services fee (profit) Proposed general conditions fee Rates for self-performed work Critical analysis of project construction budget List of proposed subcontractors Do either your RFQ or your RFP contain the following? Description of scope of work Preliminary plans/specifications Construction testing matrix Quality management roles and responsibilities Design criteria checklists Other critical material (list below) 13. Do you interview CMRs as part of the selection process? Yes, in person Yes, remotely (video teleconference or other means) No 10. If the answer to 9 is yes, how many CMRs are on your typical shortlist? 1 2 3 >3 11. If the answer to 9 is no, why don’t you develop a shortlist? Legislation prohibits shortlist Agency policy prohibits shortlist Avoid possible protest Other, please explain:

105 15. When selecting a CMR, what method do you use to identify the winner? Direct point scoring in unweighted categories published in the RFQ/RFP/advertisement Direct point scoring in weighted categories published in the RFQ/RFP/advertisement Adjectival rating in unweighted categories published in the RFQ/RFP/advertisement Adjectival rating in weighted categories published in the RFQ/RFP/advertisement Cost-technical trade-off (proposed fees compared to qualifications/other factors and best value selected) Comparative evaluation (i.e., “CMR A is more experienced than CMRs B and C” etc. for each category) Unscored selection panel consensus (i.e., panel discusses evaluation results and makes selection on consensus view) Unscored selection panel vote (i.e., each panel member selects his/her choice and CMR with most votes wins) Other, please explain: 16. Is price used as a selection criterion in your CMR award method? Yes No 17. If price is used as a selection criteria, what weight is assigned to construction related-price factors compared to all other factors? 0–25% 26–50% >50% Actual percentage: 18. If price is a selection factor, how are the construction-related price factors incorporated in the GMP? Please explain: 19. Have you ever had a protest of your CMR selection process? Yes No 20. If the answer to 19 is yes, what was the basis of the protest and how was it resolved? Basis of protest: (interviewer: collect details of protests including any documents that may be available) Protest was sustained (in favor of the protestor) Protest was denied (in favor of the agency) 21. What type of contract do you use for the CMR contract? Lump sum GMP Lump sum, fixed price (no GMP) Unit price GMP Unit price (no GMP) Cost plus fee with G-Max Preconstruction services only with hard bid construction packages Other, please explain: 22. If you out-source the design, do you modify the design contract to include CMR-specific clauses? Yes No 23. If the answer to 22 is yes, what types of CMR-specific clauses are included? (Check all that apply.) Design packages to be reviewed by CMR Design milestones to facilitate preconstruction services package Requirements to incorporate/respond to CMR review comments Budget review points Requirement to notify CMR of significant design changes Value engineering with CMR Coordination of design packages with construction bid packages Material availability/selection decisions Construction means and methods decisions Coordination with 3rd party Other, please explain: 14. If the answer to 13 is yes, what of the following are parts of your interview process? (Check all that apply.) Formal presentation of corporate qualifications/past projects Formal presentation of qualifications/past project experience for key CMR personnel Formal presentation of project-specific issues (right-of-way availability, schedule compression, bid packaging, etc.) Formal presentation of preconstruction services components (constructability process, estimating process, scheduling process, etc.) Review and discussion of checklists/documents used by the CMR in preconstruction services components (constructability process, estimating process, scheduling process, etc.) Informal review/discussion of any of the above. (interviewer: circle the appropriate items above) Other, please explain:

106 26. Does your GMP include transparent contingencies? Yes No 27. If the answer to 26 is yes, which types are included? Single project contingency Separate owners and CMR’s contingencies Management reserve in addition to contingencies 28. Does your contract contain a shared savings below the GMP incentive? Yes No 29. If the answer to 28 is yes, does the incentive include all contingency funds? Yes No Explain if necessary: 30. What process do you use to award the construction if the you and the CMR are unable to agree on the GMP? Convert to DBB and bid publicly Other, please explain: 31. If you convert to DBB, is the CMR allowed to bid for the construction contract? Yes No 32. Do you allow the CMR to prequalify subcontractors? Yes, CMR is free to pick and choose its own subs No, CMR is required to accept bids from all subs and award to lowest bidder Other, please explain: 33. Do you allow the CMR to self-perform any of the construction? Yes No 34. If the answer to 33 is no, why not? Legislation prohibits self-performance Agency policy prohibits self-performance Avoid possible protest due to potential conflict of interest Other: 35. If the answer to 33 is yes, how do you determine what work the CMR may self-perform? CMR declares bid packages it wants to self-perform & there are no further constraints CMR declares bid packages it wants to self-perform & must bid against industry to win them 36. If the answer to 33 is yes, are limits (%) on the amount of work the CMR may self-perform? Yes limit is: % No 37. What preconstruction services are included in your CMR contracts? (Check all that apply.) Validate agency/consultant estimates Validate agency/consultant schedules Validate agency/consultant design Prepare project estimates Prepare project schedules Assist/input to agency/consultant design Constructability review Cost engineering reviews Value analysis Market surveys Coordinate with 3rd party stakeholders Assist in right-of-way acquisition Assist in permitting actions Other, please explain: 24. At what point in the process is the GMP negotiated/established? Before 100% final design After 100% final design As early as possible, given the risk analysis (agency’s call) As early as possible, given the risk analysis (CMR’s call) After subcontractor bid packages are awarded by CMR Other: 25. Do you use a progressive GMP? Yes No Other, please explain:

107 40. Does your agency have a specific training or certification program for Project Managers engaging in CMR projects? Yes No 41. Does your agency encourage or require a formal partnering process on CMR projects Yes No IV. Case Study Quality Assurance Program for CM-at-Risk Projects: 1. Do you use a different QA program for CMR projects than you do for DBB projects? Yes No If yes, what is the major difference? 2. Have you used CM-at-Risk to reduce the size of your construction inspection staff? a. Yes No Please explain if necessary: 3. Have you used CM-at-Risk to reduce the size of your design review staff on out-sourced design? a. Yes No Please explain if necessary: 4. Who performs the following construction quality management tasks in your CMR projects? Check all that apply Does not apply Agency personnel Designer’s staff CMR’s constr. Staff Agency- hired consultant Technical review of construction shop drawings Technical review of construction material submittals Checking of pay quantities Routine construction inspection Quality control testing Establishment of horizontal and vertical control on site Verification testing Acceptance testing Independent assurance testing/inspection Approval of progress payments for construction progress Approval of construction post-award QM/QA/QC plans Report of nonconforming work or punchlist 5. How do you rate the final quality of work on CMR projects compared to DBB projects? Better Same Worse No opinion 6. If the answer to 5 is either Better or Worse, explain primary reason for difference: 38. How do you establish the fee for preconstruction services? Agency has fixed rate CMR proposes fee as part of selection process (winner’s fee is accepted as best value) CMR proposes fee as part of selection process and final fee is negotiated after contract award Fee is negotiated with winner after contract award (no proposed fee prior to award) Other, please explain: 39. How do you establish the CMR’s fee for post-construction services (i.e., profit on construction project)? Agency has fixed rate CMR proposes fee as part of selection process (winner’s fee is accepted as best value) CMR proposes fee as part of selection process and final fee is negotiated after contact award Fee is negotiated with winner after contract award (no proposed fee prior to award)

108 9. Please rate the following factors for their impact on the quality of the CMR project. Factor Very High Impact High Impact Some Impact Slight Impact No Impact Qualifications of the CMR’s staff CMR’s past project experience Quality management plans Level of agency involvement in the QA process Use of agency specifications and/or design details Level of detail expressed in the procurement documents Use of performance criteria/specifications Early contractor involvement in design GMP contract Preconstruction services Warranty provisions V. Achieving Value through Project Delivery Method Selection: This section’s purpose is to collect expert opinions on each project delivery system’s ability to add value to the agency operator’s capital project delivery process. If there is more than one person in the interview, the interviewer should require the group to achieve a consensus opinion for the impact of each project delivery system on the agency’s final constructed product. 1. In your opinion how does CMR project delivery method impact the quality of the following project aspects for typical projects at your agency? For each method, assign one of the following ratings based on the agency consensus: Worst = 1; Worse = 2; Neutral = 3; Better = 4; Best = 5 Project aspects CM-at-Risk Completeness of final design deliverables Accuracy of design calculations Accuracy of quantities Acceptance of design deliverables Accuracy of specifications Accuracy of as-built documents Accuracy/applicability of O&M manuals, etc. Implementation of approved QA/QC plans Accuracy of preconstruction cost estimates Ability to achieve post-award budgets Cost growth during design (scope creep, claims, etc.) Accuracy of preconstruction schedules Ability to achieve post-award schedules Material quality Workmanship quality Aesthetics Sustainability Maintainability Operability Security during construction Impact on property owners during construction Traffic flow during construction Interest to potential bidding community 7. Do you formally evaluate the CMR’s performance quality and use that for future CMR selections? No Yes Yes 8. If the answer to 7 is yes, do you believe that the performance rating creates an incentive to achieve quality? No, Why?

109 2. In your opinion how does CMR project delivery method impact the value of the following preconstruction services for typical projects at your agency? For each method, assign one of the following ratings based on the agency consensus: Not valuable = 1; Some value = 2; Valuable = 3; Very valuable = 4; Of highest value = 5 Preconstruction service CM-at-Risk Conceptual estimating Value analysis/value engineering Design charrettes Design reviews Regulatory reviews Security impact studies Environmental studies Early contractor involvement Scope definition/clarification Cost engineering reviews Budget validation Constructability reviews Biddability reviews Operability reviews Life cycle cost analysis Subcontractor bid packaging Schedule validation Coordination with 3rd party stakeholders Early awards for critical bid packages (i.e., asphalt materials, steel, etc.) Size of plan set Other 3. Is there anything else about the CMR process that you would like to share? CASE STUDY PROJECT CONTRACTOR STRUCTURED INTERVIEW Contractor Structured Interview Form Name: Location: Annual Volume: Specialization, if any: Construction Manager-at-Risk Experience Information: 6. How many CMR projects has your company completed? 1–5 6–10 11–20 >20 7. What percentage of your workload are CMR projects in dollar terms? 1–5% 6–25% 26–50% >50% 8. Do the owners your organization works for require some form of contractor prequalification to be able compete for a CMR project? Yes No If you answered “No” skip to question 10

110 9. Please check those prequalification factors that are used in a typical contractor prequalification program in the matrix below. Check all that apply Prequalification Factor Prequalification Factor Financial capability Past illegal behavior Calculated capacity factor from financials Performance evaluations Bonding capacity Claims history Surety statements Past project experience Detailed financial analysis Timely completion of past projects Bank statements Quality of material and workmanship Insurances Workman’s compensation modifier Managerial ability Quality assurance plans Resumes for key personnel Safety plans Professional licensing for key personnel Environmental plans Key personnel past project experience Traffic control plans Equipment and plant Level of subcontracting Technical ability Use of DBEs Calculated ability factor from financials Other Please use this space to elaborate on any of the above responses: 10. Please list those aspects of CM-at-Risk contractor prequalification programs that you are familiar with that you think are particularly effective. Contractor CM-at-Risk Project Execution Information: Project Title: General Composition: Road construction Road rehabilitation Bridge rehabilitation Bridge construction Other: Contract GMP Value: $ Contract Duration: Preconstruction services fee: $ or % 1. What type of solicitation documents were used to advertise the CMR project? Request for Qualifications (RFQ) only [no proposed fees] Request for Proposals (RFP) [includes some or all fees] only RFQ + RFP Request for letters of interest Other, please explain: 2. At what point in the project design development process was the CM-at-Risk selected? 0% 1%–30% 31%–50% 51%–80% >80% 3. At what point in the process is the GMP negotiated/established? Before 100% final design After 100% final design As early as possible, given the risk analysis (agency’s call) As early as possible, given the risk analysis (CMR’s call) After subcontractor bid packages are awarded by CMR Other, please explain: 4. Did the GMP include transparent contingencies? Yes No 5. If the answer to the previous question is yes, which types were included? Single project contingency Separate owners and CMR’s contingencies Management reserve in addition to contingencies

111 6. Did the contract contain a shared savings below the GMP incentive? Yes No 7. If the answer to the previous question is yes, does the incentive include all contingency funds? Yes No Explain if necessary: 8. Were you allowed to prequalify subcontractors? Yes, CMR is free to pick and choose its own subs No, CMR is required to accept bids from all subs and award to lowest bidder Other, please explain: 9. We re you allowed to self-perform any of the construction? Yes No 10. If the answer to the previous question is no, why not? Legislation prohibits self-performance Agency policy prohibits self-performance Avoid possible protest due to potential conflict of interest Other, please explain: 11. If the answer to 9 is yes, how did the agency determine what work the CMR may self-perform? CMR declares bid packages it wants to self-perform & there are no further constraints CMR declares bid packages it wants to self-perform & must bid against industry to win them Other, please explain: 12. If the answer to 9 is yes, were limits (%) on the amount of work the CMR may self-perform? Yes limit is: % No 13. What preconstruction services are included in the CMR contract? (Check all that apply.) Validate agency/consultant estimates Validate agency/consultant schedules Validate agency/consultant design Prepare project estimates Prepare project schedules Assist/input to agency/consultant design Constructability review Cost engineering reviews Value analysis Market surveys Coordinate with 3rd party stakeholders Assist in right-of-way acquisition Assist in permitting actions Other, please explain: 14. How do you establish the fee for preconstruction services? Agency has fixed rate Estimated hours similar to a design fee estimate Historical cost data from previous CMR jobs Business decision based on what the agency would consider reasonable Other, please explain: 15. How do you establish your fee for post-construction services (i.e., profit on construction project)? Agency has fixed rate Historical cost data from previous CMR jobs Business decision based on what the agency would consider reasonable Other, please explain: 16. Did the agency encourage or require a formal partnering process on the project? Yes No 17. If the answer to the above is yes, do you believe that formal partnering adds value to the CMR process? Yes No 18. How do you rate the final quality of work on CMR projects compared to DBB projects? Better Same Worse No opinion 19. If the answer to 18 is either better or worse, explain primary reason for difference:

112 20. Does the agency formally evaluate your performance quality and use that for future CMR selections? Yes No 21. If the answer to 20 is yes, do you believe that the performance rating creates an incentive to achieve quality? Yes No, Why? 22. Please rate the following factors for their impact on the quality of the CMR project. Factor Very High Impact High Impact Some Impact Slight Impact No Impact Qualifications of the CMR’s staff CMR’s past project experience Quality management plans Level of agency involvement in the QA process Use of agency specifications and/or design details Level of detail expressed in the procurement documents Use of performance criteria/specifications Early contractor involvement in design GMP contract Preconstruction services Wa rranty provisions Legal and Political Barriers to Implementing Construction Manager-at-Risk 1. Is the construction industry in the areas where you operate opposed to CM-at-Risk project delivery? Yes No Don’t know If yes, please explain: 2. What is the single largest barrier to implementing CM-at-Risk project delivery in the areas where your organization works that don’t already use it? State/local laws Organizational procurement regulations Internal organizational policies Industry opposition Public opposition Political opposition Don’t know Other; please specify: 3. In your opinion, how does the construction industry in your area of operations view CM-at-Risk project delivery? They support it They are neutral to it They oppose it No opinion 4. In your opinion, how do the subcontractors in the construction industry in your area of operations view CM-at- Risk project delivery? They support it They are neutral to it They oppose it No opinion 5. Do the CMR projects on which your organization does work utilize contractor quality assurance acceptance testing on any of its projects? Yes No 6. If the answer to the previous question is yes, do they use a performance-based prequalification process in conjunction with the contractor acceptance testing program? Yes No

113 7. Based your experience with CMR, in your opinion what impact does CMR project delivery have on the following project aspects? Project Aspect Better No Change Worse No Opinion Number of bidders Material quality Workmanship quality Safety Maintenance of traffic Level/amount of agency inspection required Timely project completion Timely construction submittal completion Timely punchlist completion Personnel experience Personnel competence Number of contractor initiated change order requests Number of claims/disputes Responsiveness on warranty call-backs Achievement of DBE goals Environmental compliance Contractor cooperation with agency Contractor cooperation with property owners Contractor cooperation with third party stakeholders Contractor cooperation with public concerns Other Other Other Final Comments: