Implementing Alternative Technical Concepts in All Types of Highway Project Delivery Methods (2020)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

33 Chapter 5 Guidebook Development and Outline 5.1 Introduction The ATC Guidebook is the primary deliverable produced by this research project. As such, the Phase 2 activities focused on identifying, classifying, and evaluating practices that were found to be effective in the field. Additionally, all intermediate products produced by the academic research team were reviewed by the industry advisory panel and practitioner members of the research team before being finalized for submission to the NCHRP panel for review and comment. 5.2 Guidebook Framework The “strategy-method-tool” framework used to develop the guidebook is one that has been successfully applied to many NCHRP and AASHTO products, including NCHRP Report 826: Estimating Highway Preconstruction Services (Gransberg et al. 2016) and NCHRP 10-85: Guide for Implementing Construction Manager/General Contractor Project Delivery (Gransberg et al. 2013). The framework was first proposed for NCHRP Report 574: Guidance for Cost Estimation and Management for Highway Projects During Planning, Programming, and Preconstruction defined by Anderson et al. (2006). The definitions are as follows: “A strategy is a plan of action intended on accomplishing a specific goal. Strategies typically address a specific problem and are formulated to address a problem’s root cause… The strategy is implemented through a method. A method is a means or manner of procedure, especially a regular and systematic way of accomplishing something… A method is then implemented using a tool. A tool is something used in the performance of an operation” (Anderson et al. 2006). For purposes of this research the framework will be used to synthesize the research findings into three interrelated categories as follows: 1. Strategies: Program level plans of action to implement ATCs in a manner that is relatively consistent throughout the agency. 2. Methods: The various project delivery methods in which ATCs are being implemented. 3. Tools: Effective practices found in the research for project-level ATC implementation, execution, and administration. The methodology described by Anderson et al. begins by identifying the causal factors for the topic of interest that the strategies developed must address. The research team will attempt to draw the causal factors primarily from the literature and refine them based on the survey and content analysis results. The first Phase 2 task will involve identifying the major factors for which a strategy is needed to resolve common issues present in most ACM projects with the use of ATCs. The final list will not be comprehensive but rather be general enough to apply to most, if not all, ACM project delivery environments. Justification for the causes of inefficiency in the ATC project development and delivery process are contained in the benchmarking report. 5.3 Guidebook Development Methodology The methodology was developed to ensure that the content of the guidebook is presented in a manner that fulfills the needs of a typical DOT with little formal ACM experience, as well as

34 adding value for DOTs with long standing ACM programs that can benefit by adjusting their process. The effort was the result of a critical analysis of the case study output, as seen through the lens of the Task 1.1 critical review and analysis of the state-of-the-practice. Lastly, the results of the US and international effective practice evaluation were used to provide content to the final draft guidebook. Authoring an AASHTO guidebook is a complex task. In addition to the technical material contained in the guidebook, there are three issues of importance: (1) target audience; (2) structure, format, and layout; and (3) applicability of the materials in practice. The guidebook was written to address audiences across multiple agencies and across multiple roles within each agency. The guidebook also describes the business case for making the change, discusses the barriers to make the change, provides tools for structuring ATC risk analysis and determining if ATCs are appropriate, as well as implementation guidance. The team took a structured approach to authoring the guidebook. First, the team revised the Task 1.3 annotated outline reviewed by NCHRP panel as required to comply with panel comments. The team then developed the various sections. The research team members worked concurrently on the various sections throughout the process, exchanging the sections as they were completed and jointly contributing to a glossary of key terms during the process to ensure consistency. The panel directed that the first section of the guidebook be written for agency upper management and contain the business case and key management messages. The next sections then explain the key principles of ATC process management as currently practiced. A section is devoted to matching project-specific characteristics with agency project delivery method constraints. An appendix of contract administration tools and effective practices was included for implementation examples. The team used the same protocol that was used to prepare the AASHTO Guidebook for CMGC Contracting on NCHRP Project 10-85. The primary feature was mapping information collected in the literature review, structured interviews, and case studies directly into the guidance in a manner that captures those practices that are found in multiple research instruments and represent converging lines of independent information. The teams’ past guidebooks have established a rigorous definition for classifying practices observed in the research as “best” or “effective” practices. That practice is contained in a paper by Gransberg et al. (2017) in the Transportation Research Record. The researchers ensured that the practices recommended in the guidebook passed a rigorous analysis by the researchers, review and confirmation by the industry team members and most importantly, approved by the NCHRP project panel. Lastly, the case study effective practice findings were categorized by project delivery method, permitting a comparison of the costs and benefits of ATCs on traditional DBB projects to those found in projects delivered using alternative methods. That comparison included the perceptional data acquired in the case studies and allowed the researchers to identify trends and disconnects between what the respondents “think” is beneficial and what the analysis “finds” is adding value. This was used as a tool to screen the content developed in Task 2.2 that guided the development of the Task 2.3 vetting protocol for the draft guidebook. The final Task 2.2 step was to submit the 1st draft guidebook to the NCHRP panel for review and meet with that body to receive their comments and guidance for the remainder of the project.

35 5.4 Guidebook Content Development The development of the content and format of the guidebook was governed by a structured process that involved mapping the information found during the various components of the research and carrying it into the guidebook’s content based on the protocol established in the Transportation Research Record paper entitled: “A Framework for Objectively Determining Alternative Contracting Method Best Practices,” (Gransberg et al. 2017). That protocol defines effective practices as those “found in the literature, which are validated as actually in use based on DOT survey and documentation information.” Therefore, the test to be included in the guidebook is as follows: • The given practice was identified in a research study found in the peer-reviewed literature. • The given practice was found in more than one DOT ACM policy or procedure document such as a DB manual or project development handbook. • The given practice was validated as effective based on the interviews conducted during the case study data collection. Figure 5.1 graphically illustrates the process used to assemble the content of the guidebook. The guidebook was then vetted by the Minnesota and Alabama DOTs to ensure that its content was both complete and understandable. Revisions from the vetting process were applied to the final draft submitted to the panel for review.

36 Figure 5.1 Mapping the Research Results into the Guidebook Content ATC Guidebook – Topical Content Ch ap te r 1 : In tr od uc tio n Ch ap te r 2 : A TC Pr oc ur em en t Co ns id er at io ns Ch ap te r 3 : Ac co un tin g fo r AT Cs in D es ig n Ch ap te r 4 : R ev ie w a nd Ev al ua tio n of A TC s Ch ap te r 5 : P ro je ct Ad m in ist ra tio n Key Chapter Sections • Project delivery definitions • ATC legal matters • Characteristics of ATC delivery • Guide to the guidebook and its use • Open and transparent procurement • Confidentiality • Design liability • Project ATC scope of work • Timing of ATC Review • Allowable ATC areas • Project oversight/ inspection • ATC payment provisions • Change orders on ATC projects • Quality management • Future ATC implementation • ATC review process • ATC evaluation process • ATC submittal documentation • Liquidated damages, incentives/disincentives • Clarifications and modifying the baseline scope • Amendments to the solicitation triggered by ATC submittals • Characteristics of an ATC project • Risk analysis/mitigation • Scope development • Design packaging • NEPA/permitting constraints MoDOT Case Study ALDOT Case Study WSDOT Case Study MnDOT Case Study ODOT Case Study Caltrans Case Study GDOT Case Study FDOT Case Study CDOT Case Study UDOT Case Study Literature Review Agency Expert Interviews ATC Content Analysis

37 5.5 Guidebook Table of Contents The content of the final guidebook is provided below to furnish a more detailed description of the information contained in it. The guidebook is published as NCHRP Research Report 937 and is available on the TRB website. The page numbers have been included to give the reader a feeling for the information content’s order of magnitude. Summary 1 Chapter 1 – Introduction 3 1.1 A Bold Approach 4 1.2 Project Delivery Methods 5 1.2.1 Design-Bid-Build 5 1.2.2 Construction Manager/General Contractor 5 1.2.3 Design-Build 7 1.2.4 Public-Private Partnerships 7 1.2.5 Project Development Process 9 1.3 ATC Legal Matters 9 1.4 Characteristics of ATC Delivery 11 1.5 How to Use This Guidebook 14 Chapter 2 – ATC Procurement Considerations 15 2.1 Conducting an Open and Transparent Procurement 15 2.2 Maintaining Confidentiality During the Procurement Process 16 2.2.1 The Impact of Governmental Disclosure Requirements 17 2.2.2 Ownership Rights Associated with ATCs 17 2.2.3 Making the “Equal to or Better” Determination 18 2.3 Design Liability Arising from an ATC 19 2.4 ATC Roles and Responsibilities 20 2.5 Developing an ATC Project Scope of Work 23 2.5.1 Allowable ATC Areas 23 2.5.2 Timing of ATC Review 24 Chapter 3 – Accounting for ATCs in Design 27 3.1 Characteristics of an ATC Project 27 3.2 Risk Analysis and Mitigation 28 3.2.1 Cost and Schedule Risk 29 3.2.2 ATC Performance Risk 30 3.3 Scope Development 33 3.4 Design Packaging 34 3.5 NEPA/Permitting Constraints 34 3.6 Summary 35 Chapter 4 – Agency Review and Evaluation of ATCs 36 4.1 ATC Review Process 36 4.1.1 Review Process Design and Team Selection 37 4.1.2 Preconstruction Milestone Development 39 4.2 ATC Evaluation Process 40 4.2.1 ATC Evaluation Factors 40 4.2.2 ATC Evaluation Plan 41 4.3 ATC Submittal Documentation 42 4.4 ATC Liquidated Damages and Incentives/Disincentives 43 4.5 Clarifications and Modification of the Baseline Scope for Approved ATCs 43 4.5.1 Pre-Award Clarifications of Project Scope 43 4.5.2 Contractual Modifications to the Baseline Design 44

38 4.6 Amendments to the Solicitation Triggered by ATC Submittals 44 Chapter 5 – Project Administration 46 5.1 Incorporating ATCs from Unsuccessful Proposals 46 5.2 Project Oversight/Inspection 47 5.3 ATC Payment Provisions 47 5.4 Change Orders on ATC Projects 48 5.5 ATC Impact on Quality Assurance Procedures 48 5.6 ATC Impact on Agency Resources 48 5.7 The Future for ATC Implementation 49 Chapter 6 – ATC Implementation Toolkit 50 6.1 ATC Tool #1: ATC Project Selection 50 6.2 ATC Tool#2: ATC Implementation 51 6.3 ATC Tool #3: ATC Evaluation 52 6.4 ATC Tool #4 ATC Performance Assessment 57 References 60 Abbreviations 62 Appendix A—DB ATC Information 63 A.1 DB ATC Flowchart 64 A.2 DB CATC Submittal Forms 65 A.3 DB Formal ATC Submittal Forms. 66 A.4 DB ATC Evaluation and Response Forms 69 Appendix B—P3 ATC Information 72 B.1 P3 ATC Flowchart 73 B.2 CATC Submittal Format 74 B.3 ATC Submittal Format 75 B.4 ATC Evaluation and Response. 76 Appendix C—DBB ATC Information 77 C.1 DBB ATC Flowcharts 78 C.2 DBB CATC Submittal Format 80 C.3 DBB ATC Submittal Format 80 C.4 DBB ATC Evaluation and Response 81 Appendix D—CMGC ATC Information 82 D.1 CMGC PTC Flowchart 83 D.2 Example CMGC Evaluation Plan 83 Appendix E—Example ATC Solicitation and Contract Clauses 85 E.1 Nondisclosure Agreements 85 E.2 Example ATC RFP Clauses 86 E.3 ATCs and Differing Site Conditions 89 E.4 Pre-Accepted Elements Clause 90 Appendix F—Useful Links 93 5.6 Guidebook Vetting The most important aspect of developing a research deliverable like the ATC Guidebook is ensuring that it can be easily understood and properly applied by the target audience. The Guidebook is oriented toward decision-making and a “process” view of ATC management. It is meant to support decision-making by the project team and does not provide a prescriptive recipe for how to use the ATC toolkit output.

39 The vetting workshops were facilitated by Dr. Rueda, involving a range of stakeholders from the selected DOT. The participants were asked to brainstorm through the implementation of a given topical ATC area of a typical project in each project delivery method with which that agency has past experience. The next stage of the vetting is to receive feedback detailing recommendations for changes to the draft guidebook content. The final vetting stage involves presenting the group’s findings to the participants to coordinate each topical area’s feedback with all other areas. After the workshop, a meeting with the select DOT representatives was be held to assess the results and separate the feedback that applies only to the given DOT and that content which has broader application. The results were then used to develop a 2nd draft of the guidebook which is then vetted in another DOT with little or no ATC experience. A final revision was made, and the result is the final draft of the proposed guidebook. The objective of the vetting was to test the draft guidebook to ensure that the materials are applicable for practice. The research team conducted a vetting workshop of the Guidebook with two DOTs with varying degrees of ATC experience. For this study the research group approached Alabama DOT (ALDOT) and Minnesota DOT (MnDOT). These agencies were selected because they were both featured in case studies for this project, the research team was familiar with their practices and real ATC data from each could be used for exercises. ALDOT had only tried ATCs on one project. The lack of a formalized system or familiarity with ATC project selection made the agency a great location to pilot the Guidebook as it tested the Guide’s ability to communicate concepts to an introductory level. MnDOT uses ATCs and PAEs for typical DB projects. This agency provided a good environment to assess whether the Guidebook and the Implementation Toolkit models were understandable and added value if implemented. The workshops consisted of a mixed program of presentations of Guidebook material and interactive exercises to apply the concepts of the Guide in practice and included DOT staff typically involved with ATC implementation. The second section of the workshop involved a focus group with DOT personnel. This meeting provided a platform to assess the day’s results and separate the feedback that applies only to the given DOT and that which has broader application. 5.6.1 Measuring Results The purpose of conducting the workshops was to evaluate how effectively the Guidebook communicated ATC practices and whether the research could be implemented by highway agencies using the resources provided within it. The workshops also aimed to collect feedback detailing recommendations for changes to the draft Guidebook content. Evaluation was conducted during the workshop using observational techniques, a post-workshop series of questions, and the application of the material to a typical ATC project via focus group methods. Using three methods to evaluate the performance of the guidebook allowed the research team to “detect recurrent patterns or consistent relationships” within the feedback received (Abowitz and Toole 2010). Observations were made during Day 1 of the workshop. Of particular interest to the research team was participant’s engagement with the material provided. Specifically, the observers were looking to see if participants were able to relate the Guidebook’s concepts to their prior ATC experience and found value in the information provided. Observations were also made during the exercises to see how well the Guidebook assisted in completing typical ATC implementation tasks.

40 If the group had to ask additional questions, this was noted as it implied that the guide was not comprehensive enough in that particular area. A post-workshop questionnaire was conducted at the end to measure participant’s views on the Guidebook and the concepts presented within it. The survey was developed following principles of Taylor-Powell and Renner (2009) and aimed to quantify perceived changes in motivation, knowledge and estimating skills as a result of using the Guidebook. At the end of the day, participants answered the questions, and an open discussion was held about the merits and problems of the Guidebook. “Focus groups are carefully planned discussions stimulated within a predefined group environment to obtain perceptions about a defined area of interest in a permissive, nonjudgmental environment” (Yu et al. 2006). The post-workshop focus group session involved a structured discussion about the strengths and weaknesses of the Guidebook and the realities of implementing its concepts within an agency. It was important for the research team to steer discussion to issues relating to the guidebook’s applicability to all highway agencies and limit focus on just its application to the DOT participating in the vetting. The final stage of the vetting process was to combine the feedback provided and address any issues within the Guidebook to improve its quality. This was an iterative process. Feedback for the MnDOT vetting was used to revise the Guidebook before it was provided to ALDOT for the second vetting. A summary of the feedback received from the two workshops is detailed in the following sections. 5.6.2 Minnesota DOT Vetting The first draft of the revamped Guidebook was sent to MnDOT two weeks prior to the workshop to allow participants an opportunity to read it and bring any questions they had to the vetting. Dr. Rueda and Ms. Brisk conducted the workshop which had 3 DOT participants. Observations made throughout the workshop noted that the participants were very engaged and receptive to the concepts presented. The final post-workshop questionnaire confirmed this with very positive feedback. The focus group after the workshop was also very positive, and there was consensus that the Guidebook contained very practical approaches that would significantly improve the preconstruction phase of a project. The major findings of this vetting were: • Strengths of guidebook:  The Guidebook is generally aligned with current MnDOT practices.  The appendices organized by project delivery method with examples were particularly valuable. • Improvements:  The Guidebook is repetitive in some sections, this redundancy should be removed.  While the Guidebook contains many great figures, there is still a lot of text; using call boxes to draw out the important information would make it more readable.  The Executive Summary should be aimed at higher management as they are the people who will make the decision on how these practices are implemented.

41  The ATC Toolkit instructions need to be written less academically. The same comment is applied to the entire document.  MnDOT uses a 16-point checklist to open every one-on-one meeting. This has proved to be quite effective and would add value to the Guidebook. • Comments  MnDOT does not ask contractors to submit CATCs; instead, this agency holds one-on- one meetings before the submission of ATCs. Both CATCs and the one-on-one meetings before the submission of ATCs serve the same purpose: to ensure that upcoming ATCs align with the needs of the agency, maximizing the ATC acceptance rate.  MnDOT considers that their ATC practices are having a great positive impact on its DB program, facilitating better value for taxpayer’s money. As a result of this feedback the Guidebook was revised. Redundant text was removed, and call-out boxes were introduced. The executive summary was refocused, and the style of the entire document was modified to read more like a typical DOT policy document. The 16-point checklist for opening one-on-one meetings was added to the Guidebook. 5.6.3 Alabama DOT Vetting An updated draft of the Guidebook was then provided to ALDOT one week ahead of their vetting. This workshop was led by Dr. Rueda and two graduate students. Again, there was lively discussion throughout the workshop. The group believed the research was very timely but would require a change in agency culture to work effectively. ALDOT’s first experience with ATCs has not been positive, but the feeling was that the sequence of events that led the agency to apply ATCs to a DBB project was not optimum. There was agreement that had the Guidebook been available the agency may well have made different decisions. The major findings of this vetting were: • Strengths of guidebook:  Good organization – very methodical  It steps through process beginning to end very well  Easy to read  Thorough coverage of all the functions (survey, enviro, roadway, etc.) • Improvements:  The need for an early start on ATC-DBB projects could use a more emphasis and explanation.  The portion on coordinating the environmental review with potential ATCs is important and should be heavily emphasized. • Comments:  This research fits well with MAP21 requirements for performance measurement  There is an increasing need for accountability from FHWA, and these tools will help justify resource allocation. • Training:  Participants felt that a training session for all involved at ALDOT would help implement the tools of the guidebook. For an inexperienced agency, a workshop is needed to ensure the guidebook makes sense.

42  A consulting session with the research would be good to help an agency establish a more formal ATC program and then implement it for future projects. 5.6.4 Vetting Summary The vettings accomplished the objective by validating its content as complete and cogent and identifying those aspects of the guide that needed improvement. Both the guide and the implementation toolkit were vetted. The results of the toolkit vettings are contained in Chapter 6.