National Academies Press: OpenBook
« Previous: Chapter 5 Guidebook Development and Outline
Page 49
Suggested Citation:"Chapter 6 ATC Implementation Toolkit." National Academies of Sciences, Engineering, and Medicine. 2020. Implementing Alternative Technical Concepts in All Types of Highway Project Delivery Methods. Washington, DC: The National Academies Press. doi: 10.17226/25865.
×
Page 49
Page 50
Suggested Citation:"Chapter 6 ATC Implementation Toolkit." National Academies of Sciences, Engineering, and Medicine. 2020. Implementing Alternative Technical Concepts in All Types of Highway Project Delivery Methods. Washington, DC: The National Academies Press. doi: 10.17226/25865.
×
Page 50
Page 51
Suggested Citation:"Chapter 6 ATC Implementation Toolkit." National Academies of Sciences, Engineering, and Medicine. 2020. Implementing Alternative Technical Concepts in All Types of Highway Project Delivery Methods. Washington, DC: The National Academies Press. doi: 10.17226/25865.
×
Page 51
Page 52
Suggested Citation:"Chapter 6 ATC Implementation Toolkit." National Academies of Sciences, Engineering, and Medicine. 2020. Implementing Alternative Technical Concepts in All Types of Highway Project Delivery Methods. Washington, DC: The National Academies Press. doi: 10.17226/25865.
×
Page 52
Page 53
Suggested Citation:"Chapter 6 ATC Implementation Toolkit." National Academies of Sciences, Engineering, and Medicine. 2020. Implementing Alternative Technical Concepts in All Types of Highway Project Delivery Methods. Washington, DC: The National Academies Press. doi: 10.17226/25865.
×
Page 53
Page 54
Suggested Citation:"Chapter 6 ATC Implementation Toolkit." National Academies of Sciences, Engineering, and Medicine. 2020. Implementing Alternative Technical Concepts in All Types of Highway Project Delivery Methods. Washington, DC: The National Academies Press. doi: 10.17226/25865.
×
Page 54
Page 55
Suggested Citation:"Chapter 6 ATC Implementation Toolkit." National Academies of Sciences, Engineering, and Medicine. 2020. Implementing Alternative Technical Concepts in All Types of Highway Project Delivery Methods. Washington, DC: The National Academies Press. doi: 10.17226/25865.
×
Page 55
Page 56
Suggested Citation:"Chapter 6 ATC Implementation Toolkit." National Academies of Sciences, Engineering, and Medicine. 2020. Implementing Alternative Technical Concepts in All Types of Highway Project Delivery Methods. Washington, DC: The National Academies Press. doi: 10.17226/25865.
×
Page 56
Page 57
Suggested Citation:"Chapter 6 ATC Implementation Toolkit." National Academies of Sciences, Engineering, and Medicine. 2020. Implementing Alternative Technical Concepts in All Types of Highway Project Delivery Methods. Washington, DC: The National Academies Press. doi: 10.17226/25865.
×
Page 57
Page 58
Suggested Citation:"Chapter 6 ATC Implementation Toolkit." National Academies of Sciences, Engineering, and Medicine. 2020. Implementing Alternative Technical Concepts in All Types of Highway Project Delivery Methods. Washington, DC: The National Academies Press. doi: 10.17226/25865.
×
Page 58
Page 59
Suggested Citation:"Chapter 6 ATC Implementation Toolkit." National Academies of Sciences, Engineering, and Medicine. 2020. Implementing Alternative Technical Concepts in All Types of Highway Project Delivery Methods. Washington, DC: The National Academies Press. doi: 10.17226/25865.
×
Page 59
Page 60
Suggested Citation:"Chapter 6 ATC Implementation Toolkit." National Academies of Sciences, Engineering, and Medicine. 2020. Implementing Alternative Technical Concepts in All Types of Highway Project Delivery Methods. Washington, DC: The National Academies Press. doi: 10.17226/25865.
×
Page 60

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

43 Chapter 6 ATC Implementation Toolkit This chapter describes a set of tools designed to assist agencies’ implementation of ATCs on ACM projects. The actual toolkit is an interactive spreadsheet. The toolkit includes five tools designed to cover the life cycle of the ATC process from making the decision as to whether ATCs are appropriate for a given project to assessing the performance of approved ATCs after construction is complete and producing feedback based on lessons learned. The following sections will describe the background on each tool, its input, its output, and a brief discussion of its utility within the overall ATC process. 6.1 Introduction The toolkit spreadsheet is designed to stand alone. Thus, much of the explanation shown below is also contained in the introductory section of each tool in the spreadsheet. The spreadsheet is also designed to be tailored to the specifics of each agency’s unique process and within the context of its constraints. For example, ATC Tool #2 requires the agency to develop a set of design work packages for the project under analysis. Most agencies have a standard set of disciplinary design packages within their project development and design documentation. So, the set of design packages currently shown in the tool is a placeholder and meant to be replaced by the agency as appropriate. Finally, the Tools #1 through #3 lead to a recommendation and are not intended to make the final decision. At each step in the process, the agency project team will use the tool first as a checklist to make sure that all the necessary factors have been considered and included in the process and secondly, as a structured method to inform the dialog necessary to make the decision required at each step in the process. Hence, Tool #3 will produce a recommendation regarding whether to approve the ATC as submitted, approve with conditions, or reject. The agency team must not regard that output as definitive but merely indicative and leading the team to making its final decision outside the tool. One issue that must be addressed is the fact that the semantics of project development and delivery process are not standard across the nation. A previous study by members of the research team on the topic of estimating preconstruction costs (NCHRP 15-51) first encountered this issue when it discovered that the term “preliminary engineering” had over five different definitions among AASHTO member DOTs (Gransberg et al. 2016). Additionally, DOT design professionals also use different percentages to describe the level of design at the end of each phase. For example, the term “30% design” is often associated with the end of “preliminary engineering.” Thus, to develop a meaningful research product that could be both precisely understood and immediately implemented on a national basis required the creation of a generic set of project development and delivery phases. Each phase is defined by the various tasks that are typically completed on a chronological basis. The fact that the NCHRP 15-51 deliverables have been successfully implemented in at least four states (California, Iowa, Minnesota, and Montana) demonstrates the generic model’s utility and provides the rationale for adopting it to be applied on NCHRP 08-112 project. Figure 6.1 is adapted from the NCHRP 15-51 final report and is used to provide a general task-based timeline against which the ATC process can be overlaid.

44 Figure 6.1 Generic Project Development and Delivery Phases (after Gransberg et al. 2016). 6.2 ATC Tool #1 ATC Project Selection. The first tool in the toolkit seeks to determine if a given project is well-suited for employment of ATCs. It consists of a checklist of areas that could potentially be addressed by ATCs. It is meant to be used in conjunction with the project’s risk register, if there is one, and essentially lays out the major risks and opportunities faced by most typical projects. Tool #1’s underlying assumption is that a project should have sufficient flexibility in its technical content and administrative arrangements to permit viable options to be generated by industry during procurement. If there is no apparent flexibility, then the tool will not recommend ATC usage. Lastly, Tool #1 includes a factor regarding the agency’s internal capacity to support the ATC process in a timely manner given the project’s procurement timeline. Figure 6.2 is a screenshot of the main portion of Tool #1. Figure 6.2 Tool #1: ATC Project Selection Checklist Operation & Maintenance Advertise and AwardFinal DesignPreliminary Design Conceptual Design - Project start up - Scoping and budget - Conceptual design - Conceptual estimate - Feasibility study - Funding approval - NEPA evaluation - Environmental clearance - Preliminary design and approval - ROW plans - ROW acquisition - Utility relocation - Detailed design and approval - Engineer’s estimate - Final plan package - Request for proposals - Advertise for bids - Evaluate bids - Award contract Construction

45 6.3 ATC Tool #2 ATC Implementation. The second tool in the toolkit seeks to assist the agency team in making a comprehensive analysis of the factors that require consideration within the specific ACM chosen for project delivery. It can also be used as a tool to compare more than one ACM’s suitability for ATC usage. The tool requires the agency to select one of five ACMs: 1. DBB 2. DB 3. Progressive DB 4. CMGC 5. P3 Once the ACM is selected, a conceptual timeline is displayed that includes the major ATC milestones overlaid on the procurement phases. Figure 6.3 is a screen shot of Tool #2 after the user has chosen DB as the ACM with which it intends to use ATCs. Figure 6.3 Tool #2 – Contracting Method Considerations. The next sections of the tool consist of a series of checklists regarding the following ATC related aspects: • Use of CATCs • Industry outreach and one-on-one meetings 2.2 2.2.1 Contracting Method Considerations Note: All orange boxes in this ATC Tool contain drop-down menus. Please select the appropriate choice from each drop-down menu. Which project delivery is being considered? Design-Build (DB) DB

46 • Confidentiality considerations • Environmental review and permitting constraints • Design liability considerations • ATC ownership and incorporation of ATCs from unsuccessful proposals • Review and evaluation considerations For example, Figure 6.4 is a screenshot of the checklist for CATCs. It shows that the agency team works through each check point and determines whether it is applicable to the specific project. The checklists found in this and the remaining sections of Tool #2 are meant to stimulate discussion that will eventually inform the decision on whether full scope or limited scope ATCs will be chosen for the project in question. Figure 6.4 Tool #2 – CATC Checklist. 6.4 ATC Tool #3 ATC Evaluation. The third tool in the toolkit offers a structured evaluation process for ATCs. Like all the tools, the agency can revise or adjust this to meet their specific needs and constraints. Ultimately, Tool #3 will document the agency’s rationale for making the “Equal to or Better Than” decision. The tool uses a simple process based on multi-attribute utility theory (MAUT). MAUT furnishes a rigorous mechanism for assessing the relative importance of various rated attributes between each other. • • • • Upon receipt of CATCs, the agency conducts confidential one-on-one meetings with each proposer to discuss and clarify the content of the CATCs to determine whether or not the CATC is feasible and of a nature that could potentially gain approval. A typical CATC submittal would include a detailed narrative of the change being proposed; and a description of the impact of the intended ATC on the ATC Evaluation Attributes listed in the RFP (see Tool 3. ATC Evaluation). When CATCs are requested as part of the ATC process, the agency should consider scheduling a longer procurement period to allow for the receipt, review, and discussion of CATCs. Applicable Applicable Do you plan to accept conceptual ATCs (CATCs) from proposers? Yes The ATC proposer retains the option to bid the DOT-furnished baseline design or its own approved ATC-modified design. Applicable Please check whether the following current practices are going to be applicable to your project: Upon receipt of formal ATCs, the agency conducts confidential one-on- one meetings with each proposer to discuss and clarify the content of the ATCs. The agency then proceeds to evaluate each ATC using the third tool of this ATC Toolkit (Tool 3. ATC Evaluation). DB projects require that ATCs be evaluated, approved, and priced prior to contract award. If approved, the proposer, then advances the ATC-modified design to a point where it can be priced.

47 The output is an index that can be compared to the baseline design. The baseline design’s index is equal to zero. If the ATC’s index is greater than or equal to zero, then the ATC would appear to have passed the test as a viable alternative. If it is less than zero, then it does not. Figure 6.5 is a flowchart that illustrates how Tool #3 operates. Figure 6.5 Tool #3 – ATC Evaluation Flowchart. Experience has shown that many agencies have a third outcome from their ATC evaluation methodologies, and that is to issue a conditional approval. This occurs when the ATC is deemed attractive but has one or more shortcomings that must be corrected before it can be approved. As a result, the added capability was added to Tool #3 to allow it to yield a recommendation for conditional approval. This is done during the selection of evaluation attributes. Each selected attribute is also assessed on whether conditions would be allowed or not. An example of one typical attribute of this nature might be an ATC that would require re-review of the environmental clearance. In Tool #3, the agency could designate this attribute as one that would allow no conditional approval. Figure 6.6 is a screenshot of the Tool #3 Step 1 evaluation attribute selection matrix that provides the initial input to computing the ATC index. It should be noted that the agency can add attributes without restriction to the standard list.

48 Figure 6.6 Tool #3 – ATC Evaluation Attribute Example. The next step in the ATC evaluation framework is to determine if an ATC is equal to or better than the baseline design and develop the input for making trade-offs among various ATC objectives (e.g. maximize quality, minimize traffic disruptions, maximize constructability, minimize project duration, minimize operation and maintenance goals). Step 2 develops the relative importance for each attribute by using an assessment of the potential benefit that might be realized in each evaluation attribute if the ATC were implemented. In this step, a 5-point Likert scale is used with the following descriptions for each level: 1. No change from baseline benefit 2. Slight enhancement 3. Some enhancement 4. Definitive enhancement 5. Significant enhancement Figure 6.7 is a screenshot showing the rating for the hypothetical example in the previous figures. The tool combines the input from Step 2 with the previous input using a technique called swing weighting and computes a relative weight for each evaluation attribute.

49 Figure 6.7 Tool #3 – ATC Evaluation Attribute Weighting Example. Step 3 involves establishing the baseline design expectations against which the ATC will be compared to make the equal or better determination. The process is very project-specific and agency-specific; therefore Tool #3 only furnishes a space where the project team can record those aspects of the baseline design that define its individual functional expectations. The agency should list at least one expectation for each evaluation attribute. The final list is then used to make the comparative assessment found in Tool #3’s Step 4. In Step 4, the attributes are directly compared to the baseline design using the criteria shown in Figure 6.8. Remembering that an ATC can be given a conditional approval which allows the proposers to correct those deficiencies which do not meet the equal to standard, the team then conducts the comparative assessment. If the ATC is found to be deficient in an attribute and the team believes that it can be corrected, then it is rated as “correctable” and the tool captures that attribute as needing to be included in the conditional approval. If the deficiency is not correctable, then it is rated as such and the tool will generate a recommendation to disapprove the ATC. Finally, if all attributes are assessed as equal to or better, then a recommendation of approval without conditions is generated. Figure 6.8 Tool #3 – ATC Evaluation Attribute Comparison with Baseline Design Example.

50 The final step is automatically generated by the tool using the input from all the previous steps. A weighted index is computed for the ATC and based on the conditions stipulated by the agency team in previous steps, a recommendation is made as shown in Figure 6.9. The figure shows that the example ATC’s index is greater than zero, which means it is viable. However, because the Community Impact attribute was assessed to not be equal to the baseline, but was believed to be correctable, a conditional approval is recommended. Figure 6.9 Tool #3 – ATC Evaluation Attribute Comparison with Baseline Design Example. The value of Tool #3 is not only in its functional output but also in the documentation that is generated as the agency team develops the input that is used to generate the recommendations. Both research and experience has shown that the use of consistent processes for evaluation during procurement reduces the chances of needing to defend those processes if challenged. Therefore, as an agency gains experience with the tools and tailors them to better fit their systems and constraints, the process will grow more consistent from project to project. A secondary benefit of the documentation is the ability to compare the pre-award assessment of ATCs to their actual performance after the project is completed. Chapter 5 discussed the benefits of capturing lessons learned and feeding them back into the agency’s design process, and that is the purpose of Tool #4. 6.5 ATC Tool #4 ATC Performance Assessment. Tool #4 has two primary purposes. First, it strives to capture the benefits accrued that are directly attributable to ATCs as a means of providing information to make the business case for ATCs to both internal and external stakeholders. This is an important facet in the agency’s continuous improvement process and furnishes the ammunition for making changes to long standing policies

51 and preferences. Secondly, the tool leverages the ideas and innovations that are generated by industry during the ATC process to capture potential enhancements to the agency’s design, procurement, and construction processes. Thus, the next project will be able to replicate the successes of the past project by including those technical concepts that were found to add value into the next baseline design. Tool #4 has two components. The first is a quantitative summary of ATCs and their impact on project cost. This portion also tracks the cost of stipends paid to unsuccessful proposers and uses that as the cost component of a benefit-cost analysis. This is done to document the business case for both ATCs and stipends. It records the total number of ATCs proposed by each competitor, the number that were eventually approved, and the number that found their way into the final proposals. Lastly, it documents the ATCs that the winning proposer incorporated into its proposal, as well as ATCs added from unsuccessful proposals. Since some agencies do not consider cost or time savings in their current ATC processes, the second component is a qualitative summary of the ATC innovations proposed and the outcomes realized by implementing them. This portion also seeks to capture proposed changes to the project delivery administration process that might later be considered for agency-wide implementation. An example of this type of change would be a request to permit the contractor to work during periods that are excluded in the agency boilerplate. Tool #4 includes the following elements: • ATC key performance indicators. • Lessons learned template • Factors to take into consideration: - Cost-benefit - Savings-stipend Tool #4 requires the agency to input the ATC history of previous projects over a given period that it would like to evaluate. This is done on a worksheet labeled “Table 4.1 ATC Inventory.” Once the project under analysis is complete, its ATC history is added to the ATC Inventory. This allows Tool #4 to calculate ATC performance indicators at both the project-level and agency-level. Figure 6.10 is a screen shot of an excerpt from an example Table 4.1 and Figure 6.11 is a screen shot of the output that Tool #4 produces. Figure 6.10 Excerpt of Table 4.1 ATC Inventory. Project ID Contractor ID Winning Contractor Contractor ATC No. Approved/ Disapprove d Non- Winning ATC Used Stipend ($K) Engineers Estimate ($M) Winning Proposal Amount ($M) Estimated Project Duration (Work Days) Aparent Time Savings (Work Days) ATC Innovation 180104 342303 x 1 Approved 105 102.5 285 45 Reduced work zone length 180104 342303 x 2 Approved 105 102.5 285 -5 Reduced contruction of temporary structures 180104 342303 x 3 Disapproved 105 102.5 285 - - 180104 193884 1 Approved x 105 102.5 285 10 Flexamat for channels in locations with high riprap prices 180104 193884 2 Approved 105 102.5 285 - - 180104 216128 1 Approved x 105 102.5 285 - Reduced work zone length 180104 216128 2 Disapproved 105 102.5 285 - - 25 0 25

52 Figure 6.11 Project-Level and Agency-Level ATC Performance Indicators. 6.6 Results of the DOT Vetting of the Toolkit The vettings included both the guidebook and the toolkit. One of the key aspects was to determine if the information contained in the guide was both consistent with and supportive of the toolkit spreadsheet. The spreadsheet was demonstrated to the DOT participants and then they used it with their vetting project and evaluate its outcomes. The results of the vetting of the ATC implementation toolkit are as follows: • Agencies with less experience in the use of ATCs, like ALDOT, will see more overall value in the ATC Toolkit since it would help them to address various ATC implementation 4.2 • Project ID 180093 $11 M $5 M 4.5% 6.0% 4.3 • Project ID 180093 $80 K $0 K $138 - 4.4 • Project ID 180093 100 WD 25 WD 13% 10% Project Level Agency Level • Apparent Time Savings (%) • Apparent Time Savings (WD) • Apparent Time Savings (WD) • Apparent Time Savings (%) ATC Cost Performance Project Level Agency Level • Apparent Project Savings ($M) • Apparent Agency Savings ($M) • Apparent Agency Savings (%) Stipend Performance Project Level Agency Level • Total Stipends Paid ($K) • Apparent Project Savings (%) • Stipends Paid ($K) • Agency Savings/$1 of Stipend • Project Savings/$1 of Stipend ATC Schedule Performance

53 challenges that otherwise would be addressed on a trial-and-error basis. It has probably been the case of most agencies currently using a mature set of ATC procedures, like MnDOT. • Even though ALDOT showed interest in the ATC Toolkit as a whole, this agency was particularly interested in Tool #3 ATC Evaluation and the lessons learned capturing system in Tool #4 ATC Performance Assessment. Regarding Tool #3, ALDOT indicated that ensuring a consistent and objective evaluation of ATCs was a major challenge faced by the project team in the ATC-DBB project, a challenge that could have been largely overcome with the use of this tool. On the other hand, ALDOT sees the lessons learned capturing system in Tool #4 as a mechanism to mitigate the impacts of the brain drain affecting transportation public owners across the country. High retirement rates and the migration of experienced DOT staff to the private sector are challenging these agencies’ ability to retain critical knowledge. Tool #4 would also help DOTs to minimize the amount of critical knowledge lost when staff retire or leave. • ALDOT recognized that Tool #2 recommendations would have helped to facilitate a smoother initial implementation of ATCs. For example, ALDOT confirmed beginning the ATC process earlier suggested in the DBB contracting timeline would have prevented a number of issues encountered by the agency in its first ATC-DBB project. Likewise, the recommendations on how to handle environmental review and permitting constraints would had been particularly valuable for ALDOT’s since the project team in its ATC-DBB project invested great efforts to address conflicts between attractive ATCs and previous environmental commitments made by agency. • MnDOT is satisfied with its current ATC process, and would be less likely to invest efforts and resources to adjust the ATC Toolkit to align it with its well-established DB program and ATC practices. However, MnDOT was very interested in Tool #4, particularly in the lessons learned capturing system and the ATC Inventory linked to this tool. In fact, MnDOT is currently using a preliminary version of a similar inventory. • MnDOT’s current ATC process aligns with most of the recommendations provided by Tool #2 for the implementation of ATCs in DB contracts, which has allowed this agency to confirm their practical effectiveness. A major difference between MnDOT’s current ATC practices and those used in Tool #2 is associated with the timing of the one-on-one meetings with contractors. While Tool #2 recommends agencies hold one-on-one meetings to discuss submitted Conceptual ATCs (CATCs) and ATCs, a practice adopted by other agencies, MnDOT does not ask contractors to submit CATCs; instead, this agency holds one-on-one meetings before the submission of ATCs. Both CATCs and the one-on-one meetings before the submission of ATCs serve the same purpose: to ensure that upcoming ATCs align with the needs of the agency, maximizing the ATC acceptance rate. Thus, while MnDOT is only discussing the content of ATCs with contractors before their formal submission, Tool #2 is recommending DOTs to hold these discussions before (through CATCs) and after the submission of ATCs (though one-on-one meetings). It should be noted that the ATC Toolkit must be tailored to the needs and procedures of each DOT, and this is an example of an adjustment that MnDOT could make to the toolkit if the agency decides to continue following its current practices on one-on-one meetings.

54 • MnDOT indicated that a challenging part of the ATC process is to ensure an effective and prompt evaluation of ATCs in order to prevent it from delaying other aspects of the procurement process. This is actually one of the purposes of Tool #3. Even though MnDOT seems to prefer to continue using its current subjective ATC evaluation approach, which is based on engineering judgement rather than the objective methodology provided in Tool #3, this agency recognized that the proposed tool would facilitate relevant discussions among the ATC evaluation team and could help if the DOT’s decision is challenged.

Next: Chapter 7 Conclusions and Recommendations »
Implementing Alternative Technical Concepts in All Types of Highway Project Delivery Methods Get This Book
×
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

There is an emerging view in the construction industry that better performance or better value for money can be achieved by integrating teamwork for planning, design, and construction of projects.

The TRB National Cooperative Highway Research Program's NCHRP Web-Only Document 277: Implementing Alternative Technical Concepts in All Types of Highway Project Delivery Methods seeks to assist integrated construction projects to include the construction contractor in the design process in some meaningful manner.

The report is released in association with NCHRP Research Report 937: Guidebook for Implementing Alternative Technical Concepts in All Types of Highway Project Delivery Methods.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!