Guidebook for Implementing Alternative Technical Concepts in All Types of Highway Project Delivery Methods (2020)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

50 C H A P T E R 6 This chapter describes a toolkit designed to assist agencies in implementing ATCs on ACM projects. The actual toolkit is an interactive spreadsheet and can be found on the TRB website by searching on “NCHRP Research Report 937”. The toolkit includes four tools designed to cover the life cycle of the ATC process—from making the decision as to whether ATCs are appropriate for a given project to assessing the performance of approved ATCs after construc- tion is complete and producing feedback based on lessons learned. The following sections will provide background on each tool, its input, its output, and a brief discussion of its utility within the overall ATC process. The toolkit spreadsheet is designed to stand alone. Thus, much of the explanation that follows is also contained in the introductory section of each tool in the spreadsheet. The spread- sheet is also designed to be tailored to the specifics of each agency’s process and the context of its constraints. For example, ATC Tool #2 requires an agency to develop a set of design work pack- ages for the project under analysis. Most agencies have a standard set of disciplinary design pack- ages within their project development and design documentation. So, the set of design packages currently shown in the tool is a placeholder and meant to be replaced by the agency as appropriate. Finally, Tools #1 through #3 lead to a recommendation, but this recommendation should not be interpreted as the final decision. At each step in the process, the agency project team will use the tool first as a checklist to make sure that all the necessary factors have been considered and included in the process and, secondly, as a structured method to inform the dialogue necessary to make the decision required at each step in the process. For example, Tool #3 will produce a recommendation regarding whether to approve the ATC as submitted, approve with conditions, or reject. The agency team must not regard that output as definitive, but merely as indicative and leading the team to making its final decision outside the tool. Lastly, a word of caution: the semantics of the project development and delivery process are not standard across the nation. Common terms, such as “preliminary engineering,” include different activities from state to state. To surmount this issue, a generic model for the project development and delivery process from a previous research project was adopted in the toolkit (see Figure 12). This generic model is based on the tasks that are completed at each stage in the process. An agency can overlay its own process with its own terminology on the process and ter- minology shown in Figure 12 to provide an accurate reference timeline when using the toolkit. 6.1 ATC Tool #1: ATC Project Selection The first tool in the toolkit seeks to determine whether a given project is well suited for employment of ATCs. It consists of a checklist of areas that could potentially be addressed by ATCs. It is meant to be used in conjunction with a project’s risk register, if there is one, and ATC Implementation Toolkit

ATC Implementation Toolkit 51 essentially lays out the major risks and opportunities faced by most typical projects. Tool #1’s underlying assumption is that a project should have sufficient flexibility in its technical content and administrative arrangements to permit viable options to be generated by industry during procurement. If there is no apparent flexibility, then the tool will not recommend ATC usage. The tool also captures the notion that since different proposers will possess differing preferred means and methods, the ATC process can capture potential benefits due to allowing each com- petitor to tailor their proposal to their preferred approach. Lastly, Tool #1 includes a factor regarding an agency’s internal capacity to support the ATC process in a timely manner given the project’s procurement timeline. Figure 13 is a screenshot of the main portion of Tool #1. 6.2 ATC Tool #2: ATC Implementation The second tool in the toolkit seeks to assist an agency team in making a comprehensive analysis of the factors that require consideration within the specific ACM chosen for project delivery. This tool can also be used to compare more than one ACM’s suitability for ATC usage. The tool requires the agency select one of five ACMs: 1. DBB 2. DB 3. PDB 4. CMGC 5. P3 Once the ACM is selected, a conceptual timeline is displayed that includes the major ATC milestones overlaid on the procurement phases. Figure 14 is a screenshot of Tool #2 once the user has chosen DB as the ACM with which it intends to use ATCs. The next sections of the tool consist of a series of checklists regarding the following ATC- related aspects: • Use of CATCs • Industry outreach and one-on-one meetings • Confidentiality considerations • Environmental review and permitting constraints • Design liability considerations • ATC ownership and incorporation of ATCs from unsuccessful proposals • Review and evaluation considerations For example, Figure 15 is a screenshot of the checklist for CATCs. It shows that the agency team works through each checkpoint and determines whether it is applicable to the specific project. The checklists found in this and the remaining sections of Tool #2 are meant to Operation & Maintenance Advertise and AwardFinal DesignPreliminary Design Conceptual Design - Project start up - Scoping and budget - Conceptual design - Conceptual estimate - Feasibility study - Funding approval - NEPA evaluation - Environmental clearance - Preliminary design and approval - ROW plans - ROW acquisition - Utility relocation - Detailed design and approval - Engineer’s estimate - Final plan package - Request for proposals - Advertise for bids - Evaluate bids - Award contract Construction Figure 12. Generic project development and delivery phases.

52 Guidebook for Implementing Alternative Technical Concepts in All Types of Highway Project Delivery Methods stimulate discussion that will eventually inform the decision on whether full-scope or limited- scope ATCs will be chosen for the project in question. 6.3 ATC Tool #3: ATC Evaluation The third tool in the toolkit offers a structured evaluation process for ATCs. Like all the tools, the agency can revise or adjust this to meet its specific needs and constraints. Ultimately, Tool #3 will document the agency’s rationale for making the “equal to or better than” decision. The tool uses a simple process based on multi-attribute utility theory. Multi-attribute utility theory provides a rigorous mechanism for assessing the importance of various rated attributes in rela- tion to each other. The output is an index that can be compared to the baseline design. The baseline design’s index is equal to zero. If the ATC’s index is greater than or equal to zero, then the ATC would appear to have passed the test as a viable alternative. If it is less than zero, then it does not. Figure 16 is a flowchart that illustrates how Tool #3 operates. 1.2 1.2.1 Yes 1.2.2 1.2.2.1 Yes 1.2.2.2 No 1.2.2.3 No 1.2.2.4 No 1.2.2.5 No 1.2.2.6 No 1.2.2.7 Yes 1.2.2.8 No 1.2.2.9 No 1.2.2.10 No 1.2.3 1.2.4 Right-of-Way Does the ATC Project Selection Panel foresee risk events in the project's Risk Register that could potentially be mitigated/overcome by ATCs proposed by competing contractors? Note: All orange boxes in this ATC Tool contain drop-down menus. Please select the appropriate choice from each drop-down menu. Check Yes or No in every box. ATC Project Selection Checklist Does the ATC Project Selection Panel foresee major project specific risks or challenges on the areas listed below that could potentially be mitigated/overcome by ATCs proposed by competing contractors? Are the anticipated benefits of potential ATCs expected to justify the time and resources required to review ATC submittals and incorporate approved ATCs into the final design? Yes Recommended for ATCs Potential Differing Site Conditions Environmental/Permitting Constraints Agency Design Capabilities Agency Contract Administration/Oversight Capabilities Does the Project Selection Panel foresee potential means and methods ATCs that could facilitate the accomplishment of the project objectives? Yes Recommendation: Utilities and Railroad Arrangements Project Scope Life-Cycle Cost Project Schedule Quality Assurance/Control Figure 13. ATC project selection checklist in Tool #1.

ATC Implementation Toolkit 53 2.2 2.2.1 Contracting Method Considerations Note: All orange boxes in this ATC Tool contain drop-down menus. Please select the appropriate choice from each drop-down menu. Which project delivery is being considered? Design-Build (DB) DB Figure 14. Contracting method considerations in Tool #2. • • • • Upon receipt of CATCs, the agency conducts confidential one-on-one meetings with each proposer to discuss and clarify the content of the CATCs to determine whether or not the CATC is feasible and of a nature that could potentially gain approval. A typical CATC submittal would include a detailed narrative of the change being proposed; and a description of the impact of the intended ATC on the ATC Evaluation Attributes listed in the RFP (see Tool 3. ATC Evaluation). When CATCs are requested as part of the ATC process, the agency should consider scheduling a longer procurement period to allow for the receipt, review, and discussion of CATCs. Applicable Applicable Do you plan to accept conceptual ATCs (CATCs) from proposers? Yes The ATC proposer retains the option to bid the DOT-furnished baseline design or its own approved ATC-modified design. Applicable Please check whether the following current practices are going to be applicable to your project: Upon receipt of formal ATCs, the agency conducts confidential one-on- one meetings with each proposer to discuss and clarify the content of the ATCs. The agency then proceeds to evaluate each ATC using the third tool of this ATC Toolkit (Tool 3. ATC Evaluation). DB projects require that ATCs be evaluated, approved, and priced prior to contract award. If approved, the proposer, then advances the ATC-modified design to a point where it can be priced. Figure 15. CATC checklist in Tool #2.

54 Guidebook for Implementing Alternative Technical Concepts in All Types of Highway Project Delivery Methods Experience has shown that many agencies have a third outcome from their ATC evaluation methodologies, and that is to issue a conditional approval. This occurs when the ATC is deemed attractive but has one or more shortcomings that must be corrected before it can be approved. As a result, the capability was added to Tool #3 to allow it to yield a recommendation for conditional approval. This is done during the selection of evaluation attributes. Each selected attribute is also assessed on whether conditions would be allowed or not. An example of one typical attribute of this nature might be an ATC that would require re-review of the environ- mental clearance. In Tool #3, the agency could designate this attribute as one that would allow no conditional approval. Figure 17 is a screenshot of an evaluation attribute selection matrix used in Step 1 of Tool #3 that provides the initial input for computing the ATC index. It should be noted that the agency can add attributes without restriction to the standard list. The next step in the ATC evaluation framework, to determine whether an ATC is equal to or better than the baseline design, develops the input for making trade-offs among various ATC objectives (e.g., maximize quality, minimize traffic disruptions, maximize constructability, minimize project duration, and minimize operation and maintenance goals). The tool populates a matrix with the selected evaluation attributes, and the agency must then rank each attribute from most to least important. Step 2 develops a relative importance for each attribute by using an assessment of the benefit that might be realized in each evaluation attribute if the ATC were implemented. In this step, a 5-point Likert scale is used with the following descriptions for each level: 1. No change from baseline benefit 2. Slight enhancement 3. Some enhancement 4. Definitive enhancement 5. Significant enhancement Figure 18 is a screenshot showing the rating for the hypothetical example shown in Figures 13, 14, 15, and 17. The tool combines the input from Step 2 with the previous input using a tech- nique called swing weighting and computes a relative weight for each evaluation attribute. Figure 16. Flowchart describing the ATC evaluation process in Tool #3.

ATC Implementation Toolkit 55 Evaluation Attribute Select Life Cycle Cost Road User Costs (during construction) Constructability Project Duration Asset Structural Performance Asset Operational Performance Aesthetics Environmental Impact No Impact to Environmental Permits Impact to Surrounding and Adjacent Communities Construction Inspection and Testing Efforts Traffic Control and Disruptions Post Construction Safety Workzone Safety Cost Certainty Schedule Certainty Third-Party Stakeholder Risk --- add attribute --- --- add attribute --- --- add attribute --- No Conditions Allowed Figure 17. ATC evaluation attribute selection matrix example from Tool #3. Step 3 involves establishing the baseline design expectations against which the ATC will be compared to make the equal or better determination. The process is very project specific and agency specific; therefore, Tool #3 only provides a space where the project team can record those aspects of the baseline design that define its individual functional expectations. An agency should list at least one expectation for each evaluation attribute. The final list is then used to make the comparative assessment found in Tool #3’s Step 4. In Step 4, the attributes are directly compared to the baseline design using the criteria shown in Figure 19. Remembering that an ATC can be given a conditional approval, which allows the Evaluation Attribute Benefit Weight Lifecycle Cost 2 7% Constructability 5 18% Project Duration 3 11% Environmental Impact 1 4% No Impact to Environmental Permits 1 4% Impact to Surrounding and Adjacent Communities 1 4% Traffic Control and Disruptions 4 14% Post Construction Safety 1 4% Workzone Safety 1 4% Cost Certainty 2 7% Schedule Certainty 4 14% Third-Party Stakeholder Risk 3 11% Figure 18. ATC evaluation attribute weighting example from Tool #3.

56 Guidebook for Implementing Alternative Technical Concepts in All Types of Highway Project Delivery Methods Figure 19. ATC evaluation attribute comparison with baseline design example in Tool #3. proposers to correct those deficiencies that do not meet the “equal to” standard. The team then conducts the comparative assessment. If the ATC is found to be deficient in an attribute and the team believes that it can be corrected, then it is rated as “correctable” and the tool captures that attribute as needing to be included in the conditional approval. If the deficiency is not cor- rectable, then it is rated as such and the tool will generate a recommendation to disapprove the ATC. Finally, if all attributes are assessed as “equal to or better than,” then a recommendation of approval without conditions is generated. The final step is automatically generated by the tool using the input from all the previous steps. A weighted index is computed for the ATC, and, based on the conditions stipulated by the agency team in previous steps, a recommendation is made (as shown in Figure 20). The figure shows that the example ATC’s index is greater than zero, which means it is viable. Evaluation Attribute Weight Weighted Rate Lifecycle Cost 7% 0.00 Constructability 18% 0.18 Project Duration 11% 0.00 Environmental Impact 4% 0.00 No Impact to Environmental Permits 4% 0.00 Impact to Surrounding and Adjacent Communities 4% -0.02 Traffic Control and Disruptions 14% 0.07 Post Construction Safety 4% 0.00 Workzone Safety 4% 0.00 Cost Certainty 7% 0.00 Schedule Certainty 14% 0.07 Third-Party Stakeholder Risk 11% 0.00 ATC 0.30 Recommendation: Recommend approval with conditions The ATC should be improved in the following areas in order to be recommended for approval: - Impact to Surrounding and Adjacent Communities Figure 20. ATC evaluation attribute comparison with baseline design example with recommendation in Tool #3.

ATC Implementation Toolkit 57 However, because the Community Impact attribute was assessed to not be equal to the baseline, but was believed to be correctable, a conditional approval is recommended. The value of Tool #3 is not only in its functional output but also in the documentation that is generated as an agency team develops the input that is used to generate the recommendations. Research and experience have shown that the use of consistent processes for evaluation during procurement reduces the chance of needing to defend those processes if challenged. Therefore, as an agency gains experience with the tools and tailors them to better fit its systems and constraints, the process will grow more consistent from project to project. A secondary benefit of the documentation is the ability to compare the pre-award assess- ment of ATCs to their actual performance after a project is completed. Chapter 5 discussed the benefits of capturing lessons learned and feeding them back into the agency’s design process, and that is the purpose of Tool #4. 6.4 ATC Tool #4: ATC Performance Assessment Tool #4 has two primary purposes. First, it strives to capture the benefits accrued that are directly attributable to ATCs as a means of providing information to make the business case for ATCs to both internal and external stakeholders. This is an important facet of the agency’s continuous improvement process and provides the ammunition for making changes to long- standing policies and preferences. Secondly, the tool leverages the ideas and innovations that are generated by the industry during the ATC process to capture potential enhancements to the agency’s design, procurement, and construction processes. Thus, the next projects will be able to replicate the successes of past projects by including those technical concepts that were found to add value to the future baseline design. Tool #4 has two components. The first is a quantitative summary of ATCs and their impact on project cost. This component also tracks the cost of stipends paid to unsuccessful proposers and uses that as the cost component of a benefit-cost analysis. This is done to document the business case for both ATCs and stipends. Tool #4 records the total number of ATCs proposed by each competitor, the number that were eventually approved, and the number that found their way into the final proposals. Lastly, it documents the ATCs that the winning proposer incorporated into its proposal, as well as ATCs added from unsuccessful proposals. Since some agencies do not consider cost or time savings in their current ATC processes, the second component is a qualitative summary of the ATC innovations proposed and the outcomes realized by implementing them. This component also seeks to capture proposed changes to the project delivery administration process that might later be considered for agency-wide imple- mentation. An example of this type of change would be a request to permit the contractor to work during periods that are excluded in the agency boilerplate. Tool #4 includes the following elements: • ATC key performance indicators • Lessons learned template • Factors to take into consideration: – Cost-benefit – Savings-stipend Tool #4 requires the agency to input the ATC history of previous projects over a given period that it would like to evaluate. This is done on a worksheet labeled “Table 4.1 ATC Inventory.” Once the project under analysis is complete, its ATC history is added to the ATC inventory. This allows Tool #4 to calculate ATC performance indicators at both the project level and agency level. Figure 21 is a screenshot of an excerpt from an example Table 4.1, and Figure 22 is a screenshot of the output that Tool #4 produces.

Figure 21. Excerpt of Table 4.1 ATC Inventory. Project ID Contractor ID Winning Contractor Contractor ATC No. Approved/ Disapproved Non- Winning ATC Used Stipend ($K) Engineers Estimate ($M) Winning Proposal Amount ($M) Estimated Project Duration (Work Days) Apparent Time Savings (Work Days) ATC Innovation 180104 342303 x 1 Approved 105 102.5 285 45 Reduced work zone length 180104 342303 x 2 Approved 105 102.5 285 -5 Reduced construction of temporary structures 180104 342303 x 3 Disapproved 105 102.5 285 - - 180104 193884 1 Approved x 105 102.5 285 10 Flexamat for channels in locations with high riprap prices 180104 193884 2 Approved 105 102.5 285 - - 180104 216128 1 Approved x 105 102.5 285 - Reduced work zone length 180104 216128 2 Disapproved 105 102.5 285 - - 25 0 25

ATC Implementation Toolkit 59 4.2 • Project ID 180093 $11 M $5 M 4.5% 6.0% 4.3 • Project ID 180093 $80 K $0 K $138 - 4.4 • Project ID 180093 100 WD 25 WD 13% 10% Project Level Agency Level • Apparent Time Savings (%) • Apparent Time Savings (WD) • Apparent Time Savings (WD) • Apparent Time Savings (%) ATC Cost Performance Project Level Agency Level • Apparent Project Savings ($M) • Apparent Agency Savings ($M) • Apparent Agency Savings (%) Stipend Performance Project Level Agency Level • Total Stipends Paid ($K) • Apparent Project Savings (%) • Stipends Paid ($K) • Agency Savings/$1 of Stipend • Project Savings/$1 of Stipend ATC Schedule Performance Figure 22. Project-level and agency-level ATC performance indicators.