Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
16 Chapter 2: Research Approach 2.1 Research Framework The research framework springs directly from the NCHRP 24-44 RFP and is divided into two phases: â¢ Phase 1 â Benchmark the state of the practice in geotechnical risk management â¢ Phase 2 â Quantify the costs and benefits and Develop the proposed Guidelines for Managing Geotechnical Risk on Design-Build Projects. Hereafter referred to as the âGuidelines.â The outcome of the research is a set of guidelines, an interim research report, which was submitted in January 2017 and approved by the NCHRP panel in April 2017, and this final research report based on a rigorous analysis of a state-of-the-practice review. The state-of-the-practice findings function as a baseline from which the new contributions to this area are built. The research approach was founded on the following research instruments: ï· Comprehensive review of the literature on the topics of geotechnical risk and DB contract formation and administration. ï· A content analysis of DOT DB and geotechnical engineering manuals of practice, guidelines, and policies from 36 states. ï· A content analysis of 59 DOT DB project solicitation documents. ï· A legal review of the case law on the subject: 17 cases that establish the legal foundations for the issue of subsurface risk allocation and responsibility. ï· A nation-wide benchmarking survey of state DOTs that received 38 responses for an overall response rate of 76%.
17 ï· A directed survey of DOT (22 responses) and industry experts (24 responses) on managing geotechnical risks in DB projects (overall response rate of 31%). ï· Case study data collection, synthesis, and analysis on 11 projects in nine states. The details of Phase 1 and 2 research approaches are shown in graphic form in Figure 2.1 and Figure 2.2 below. Figure 2.1 Phase 1 Research Work Plan Ph as e 1: B en ch m ar k th e St at e- of -t he -P ra ct ic e an d O ut lin e Pr op os ed H an db oo k U pd at e
18 Figure 2.2 Phase 2 Research Work Plan 2.1.1 Phase 1: Benchmark the State of the Practice in Geotechnical Risk Phase 1 evaluated current applications of geotechnical risk in traditional low bid DBB transportation projects, while maintaining a keen focus on how DOTs have modified the traditional NCHRP 24-44: Guidelines for Managing Geotechnical Risks in Design-Build Projects Research Framework Phase 2: Tasks 5, 6, and 7 Ph as e 2: D ev el op a nd V et P ro po se d Gu id el in es PRODUCE GEOTECHNICAL RISK MANAGEMENT GUIDELINES Develop RiskÂ AnalysisÂ Plan Conduct GeotechnicalÂ RiskÂ SimulationÂ â Evaluate OutputÂ DataÂ â Quantify GeotechnicalÂ RiskÂ Task 7 FinalÂ ResearchÂ ReportÂ &Â RecommendedÂ Guidelines NCHRP Panel Review Project Complete Industry Advisory Panel Review Draft Guideline Vetting Plan Literature Review Output Case Study Project Output Survey Output Task 5 Quantitative Risk Analysis Task 6 Prepare Geotechnical Risk Management Guidelines Develop 1st DraftÂ Guidelines Document Content Analysis Output Assemble RiskÂ AnalysisÂ InputÂ Data Generic Geotechnical Risk Register Geotechnical Risk Model Draft Geotechnical Risk Management Guidelines Develop GuidelineÂ VettingÂ Plan Conduct DOTÂ Vetting Workshops Final Draft Geotechnical Risk Management Guidelines Industry Advisory Panel Review From Phase 1 Revise Guidelines as required
19 process to accommodate alternative project delivery. It evaluated the state of the practice with respect to alternative project delivery selection and the way it is applied on a variety of project types. Due to the interdependent nature of the tasks in Phase I, the research overlapped Tasks 1 and 2 with much of the work performed concurrently. The output of the literature review, the content analyses of DOT DB solicitation documents, agency risk management policy guidelines, and the information contained in agency geotechnical design manuals were synthesized and documented in Task 3b. Task 2 involved of a survey of DOTs to provide direction on how to proceed with Task 3a case study interviews, document and data collection. The survey also captured real-time perceptional data regarding practitionerâs definitions of the tangible and intangible costs and benefits associated with geotechnical risk management on their typical DB projects. 220.127.116.11 Task 1 Review of the Literature and Relevant Case Law The Task 1 effort focused on updating the NCHRP Synthesis 429: Geotechnical Information Practices in Design-Build Projects (2011) and 455: Alternative Technical Concepts for Contract Delivery Methods (2013) literature reviews and content analyses of US DOT geotechnical risk policy documents and guidelines. Additionally, the review and analysis of US case law that is pertinent to geotechnical risk was completed, which served to identify legal barriers to US implementation of promising geotechnical risk practices found in the international literature. 18.104.22.168 Task 2 Survey of Current Practice Task 2 involved a survey concerning the use of geotechnical risk with a keen focus on its variants applied to alternative project delivery methods. The team used the Synthesis 429 DB geotechnical
20 risk survey as a starting point, and modified it to include the cogent findings of the Synthesis 455 ATC survey. The survey for this study was industry-wide, so the team could compare highway agency practices with those of other agencies and private sector firms. The research team used the following as sources of contacts for the web-based survey: ï· Appropriate AASHTO committees ï· Appropriate TRB committees ï· International Society for Soil Mechanics and Geotechnical Engineering ï·Design-Build Institute of America ï·Austroads (the Australia-New Zealand version of AASHTO) ï·ASCE Geo-Institute ï·National Council for Public Private Partnerships An invitation to volunteer potential case study projects was included in the survey. Finally, a set of potential case study projects and case study agencies was assembled for use in Task 3a. Both the draft survey and the draft case study interview questionnaires were submitted for panel approval before they were issued. The forms are contained in Appendix C. A second survey of practicing professionals experienced in DB geotechnical risk management was added to the experimental plan. Since âriskâ in its purest form is a matter of perception, research has shown that the perception of DB risk decreases as experience increases (DâIgnazio et al. 2011). It was felt that a survey of experienced expertsâ risk perceptions would provide an insight that would assist practitioners that are new to DB with a benchmark from which to develop geotechnical risk management plans for their initial DB projects. The details of the second survey are found in Appendix C. The output from both surveys was used to form a set of geotechnical risks that are common to DB projects. Additionally, the second surveyâs perceptional output was used to rank those risks on a basis of importance indices. The assumption that the risk perceptions of survey respondents
21 with ten or more DB projects worth of experienced was tested statistically against the respondent that reported experience with less than ten DB projects using the Pearson Chi-Square Test and found to be statistically significant. Details on this are found in Chapter 3. 22.214.171.124 Task 3 State-of-the-Practice Summary for DB Project Geotechnical Risk Management ï· Task 3a. Case study project data collection and interviews, ï· Task 3b. A summary of the state of the practice for highway transportation project geotechnical risk management, both formal and informal, and ï· Task 3c. A detailed outline for the Geotechnical Risk Management Guidelines. Task 3a. Case study project data collection and interviews The output from Tasks 1 and 2 were used to develop a case study protocol and a framework upon which the coding structure was developed by the research team. Ultimately, the documentation was used as a basis from which questionnaires for the case study structured interviews were built. The interview results was used to identify the high-level issues regarding industry acceptance and barriers to implementation that need to be addressed. It also sought to identify and document effective practices to manage risk, such as the use of geotechnical ATCs, which were carried forward into the content of the Guidelines. Lastly, pre-bid pricing implications and their relation to geotechnical risk management and mitigation were also investigated during the case study data collection and interviews. Once the important issues were identified, structured interviews with DOT, consultant, and contractor project participants were conducted to gather information on the concerns and preferences of the stakeholders in DB projects.
22 The first step was to assemble the possible case studies identified in Task 1 and pass them through a filter to ensure that the case study population covered the full spectrum of the research interest. The goal was to have a set of possible case study projects that furnish these attributes: ï· Range of project types â roads, bridges, tunnels, ITS, etc. ï· Range of project size â typical small project to mega-project ï· Range of project complexity â simple to highly complex ï·Range of project location ï·Range of project delivery methods ï·Other factors that may be found in Tasks 1 & 2 Once the potential case study population was developed, the final list, with the rationale for selecting each case, was submitted to NCHRP for approval in a quarterly report. The final approved population consisted of 11 individual case studies were conducted on projects involving significant geotechnical risk on traditional and alternative project delivery methods throughout the country. The case study protocol followed the guidance provided by Yin (2008). Case studies are empirical inquiries that investigate contemporary phenomenon in its real-life context. The case studies protocol was developed to adequately evaluate how the various agencies have successfully implemented geotechnical risk management on projects delivered using alternative methods. These are the primary efforts needed to accomplish this objective: 1. Develop a case study protocol for conducting the case study interviews 2. Conduct the case study interviews 3. Document the raw information collected and integrate it with data from the literature review
23 The key step in Task 3a is the first one: develop a case study protocol for the case study interviews and data collection plan. The protocol (See Appendix C) included a research synopsis of objectives, projects, field procedures that detail the logistical aspects of the investigation, interview questions, and documentation to collect as well as the format for documenting and analyzing the individual case studies (Eisenhardt 1989, 1991; Yin 2008). In addition, a plan was developed for cross-case comparisons to determine similarities and differences between cases (Eisenhardt 1989; Miles and Huberman 1994). The case study protocol permitted the research team to conduct case studies separately in different parts of the country, while maintaining the reliability of the case study results. Internal validity is addressed by attending to multiple sources of evidence and the use of multiple case studies improves the external validity of the project delivery. The case study protocol was pre-tested with the Missouri DOT to ensure that it would provide the desired information. As part of the pre-test, case study participants were asked to comment on the protocol, covering topics such as ease of understanding, ease of response, amount of required pre-interview preparation required, etc. The protocol was then modified to reflect the feedback provided. With limited data found in NCHRP Synthesis 429 concerning the cost effectiveness of geotechnical risk management in alternative delivery, a primary objective of these case studies was to build solid theory for each delivery method. The goal for case study selection is to generate a cross section of cases that permits the analysis of the advantages and disadvantages of the various geotechnical risk models across the various project characteristics. To ensure that this goal is met, the following criteria were placed on the case study selection. ï· Projects delivered using DB ï· Agency solicitation documents are available for content analysis
24 ï· Financial and schedule progress data available ï· Design consultant and construction contractor available for interviews ï· Agency information â construction volume, outsourcing of design, seasonal issues, and so forth ï· Alternative project delivery experience data â number, type, cost/schedule performance, project-specific legal issues, claims, protests, and so forth The case study data collection plan included the procedures for assembling the necessary input data to calculate metrics that quantify the geotechnical risk management cost and benefits for use in Task 5, the quantitative risk analysis. During each case study interview, the DOT was asked to furnish quantitative data on the case study project as well as other past projects that had significant geotechnical issues. The focus was on projects where differing site conditions resulted in delays, change orders and/or claims. The outcomes were mapped to the tools used to mitigate geotechnical risk in Task 5 to furnish definitive guidance that was provided in the final Guidelines. This ancillary goal was not met as all the case study projects were successful and resulted in no DSC claims. Upon completion of the case studies, the team reduced and analyzed the case study data to identify trends and disconnects, gaps in the body of knowledge, needs for contract clause guidance, examples of successful practices, and lessons learned. The primary focus was to matrix specifics of the geotechnical risk model used for each case study for use in the Task 5 quantitative risk analysis. The synthesis of Task 3a findings were also used to develop the state-of-the-practice summary and the annotated outline of the Guidelines in Tasks 3b and 3c respectively.
25 Task 3b. Summary of the state of the practice for managing highway transportation project geotechnical risk on DB projects The state-of-the-practice summary was devoted to the system-level and detailed effective practices for managing geotechnical risk on DB projects. It included such subjects as policy and guidance documentation, design documentation, quality management regulations, resource requirements, training, and contract administration. The summary was included in the interim report and focused on individual methods and specific tools that were found to be effective in the research. It was submitted and reviewed by the NCHRP panel. Task 3c. Prepare annotated outline for the Guidelines for Managing Geotechnical Risk on Design-Build Projects The Guidelines were specifically tailored for the AASHTO audience by the practitioner members of the research team. They also included discussion of the legal pitfalls and remedies, as well as contract DSC clause development and cogent lessons learned from legal case law. Additionally, the findings from the DOT geotechnical engineering design and quality management review and salient information on geotechnical risk analysis was covered. 126.96.36.199 Task 4 Prepare an Interim Research Report and an Updated Phase II Work Plan The objective of Task 4 was to produce a comprehensive summary of findings and conclusions from Tasks 1, 2 and 3, and the updated work plan for Phase II of the research. The interim report was approved by the oversight panel, so that the research could proceed. Team members have prepared numerous interim reports and employed lessons learned from past projects to prepare all documents. Based on the results of Tasks 1 through 3, the research team developed the Phase 2
26 work plan in detail. The Phase 2 work plan included the steps necessary to advance the panel- approved draft Guidelines outline and Final Research Report. 188.8.131.52 Task 5 Quantify Geotechnical Risk Costs and Benefits and Develop the Guidelines The Task 5 data collection plan is founded on the principle of measuring input parameters and correlating them with output metrics. Figure 2. 3 is a conceptual illustration of the process that was be followed to complete Task 5 and shows the research teamâs belief that geotechnical riskâs costs and benefits are unique to a give projectâs attributes and the goals of the project team. The figure describes the notion that given a projectâs description, the analyst can then create a correlation matrix (Rebonato and JÃ¤ckel 1999) containing input measures that describe the resources, such as the cost of additional geotechnical investigation or purchasing a named peril insurance, that will be consumed by adding formal geotechnical risk analysis to the DB project delivery process and the output metrics, like cost or time growth, and statistically evaluate the two data sets in a manner that produces a statistical measure of correlation. Figure 2. 3 Conceptual Research Framework for Quantitative Geotechnical Risk Analysis The team found that importance index theory (Assaf and Al Hejji 2006; Santoso and Soeng 2016) furnished the most robust methodology for identifying trends among the risk data points and Project Input Measures Pr oj ec t O ut pu t M et ric s DB/P3 Project Description Level of Definition Project Complexity Location Required schedule Urban location Foundation conditions Slope stability Utilities Environmental Min. Duration Min. Disruption Max. Life Project Goals Completion Requirement Input does not correlate to output Correlation Factor Input correlates to output Return on Pre-Advertising Geotechnical Investigation Investment Positive Return OR Negative Return Additional Investigation Justified Yes OR No Drop from Analysis Associate Outcome with Project Attributes Project Attributes
27 used it to represent a âcorrelation factorâ (Hong et al. 2012). Those cases where the input is not well correlated to the output are dropped from the analysis. The cycle then completes itself when the outcomes are associated with the project attributes. 184.108.40.206 Importance Index Methodology The analysis of the data for the second survey was conducted in accordance with importance index methodology (Assaf and Al Hejji 2006; Santoso and Soeng 2016). This approach requires the analyst to initially calculate a frequency index and an impact index based on the perceptions of the expert survey respondents. These are then combined mathematically to calculate an importance index for each factor. The underlying theory is that a risk with a high impact that frequently occurs is more important than a low impact risk that rarely occurs. The approach proposed by Assaf and Al Hejji (2006) was adapted for use in this specific context. The importance index is a function of the frequency index (FI) and impact index (II). The indices are computed as shown in equations 2.1, 2.2, and 2.3 below: Frequency Index: This formula was used to rank the geotechnical factors based on the frequency of occurrence as indicated by the contestants % â â 100/5 (Eq. 2.1) Where = is a constant defining the weighting assigned to each response (ranges from 0 for Never up to 4 for Very Often), = is the frequency of the responses, and = is the total number of responses. Impact Index: This formula was used to rank the geotechnical factors based on the impact of as indicated by the respondents.
28 % â â 100/5 (Eq. 2.2) Where: = is a constant defining the weighting assigned to each response (ranges from 0 for No Impact up to 4 for Catastrophic impact), = is the frequency of the responses, and = is the total number of responses. Once having these two indices the importance index (IMPI) can be calculated by the following formula: % % â % /100 (Eq. 2.3) 2.1.2 Phase 2 Geotechnical Risk Assessment The purpose of geotechnical risk assessment in the Phase 2 research is twofold: 1. To provide a project-specific inventory of geotechnical risks, along with each riskâs probability of occurrence and estimated impact on budget and schedule, and 2. To develop an effective plan for coping with geotechnical risks and reducing geotechnical uncertainties where possible. The outcome of this effort is a comprehensive geotechnical risk mitigation plan which would include major geotechnical risks, their impact on cost and schedule and the proposed mitigation method. A by-product of this effort is the risk allocation process through the DB contract. The risk assessment should help the owner establish thresholds for DSC clauses and the limits of liability that can be reasonably expected from the contractor without causing bidders to unduly increase their bids. The Phase 1 research concluded that the presence of significant geotechnical risk did not deter the decision to employ DB delivery, and that DB selection decision is more a function of
29 schedule demands and other non-technical considerations. Therefore, a formal risk analysis to identify and quantify major geotechnical risk factors becomes critical to setting the DB project up for success. Geotechnical risk assessment is conducted using the process set by FHWA (2006). The purpose of a risk assessment is to identify, quantify, and mitigate the significant risks that may affect the project. Phase 2 research used the same geotechnical risk assessment framework concentrated on those specific geotechnical risk factors required to estimate the scale of geotechnical uncertainty. Depending on the magnitude, a decision must be made whether or not to invest in more geotechnical investigation before requesting DB proposals. The general process of risk assessment is well documented and has been described in numerous sources (Touran et al. 2009). Here, the team will follow the below geotechnical risk assessment steps: 1. Identify Risks: For a geotechnical risk assessment, the attention is focused on relevant risk factors. One of the tools used for risk identification is the use of a risk catalog. Developing a catalog of geotechnical risks can facilitate the process of risk identification. The proposed risk catalog was established using information gleaned from the Task 2 survey and Task 3a case study interviews. The catalog was organized according to the ASCE (2011) Geoconstructability Guidelines: dewatering, deep foundations, mass excavation, tunneling, etc. As the risks are identified during this step, they will be added to a template for geotechnical risk register that is part of the final Guidelines. 2. Assess/Analyze Risks: This step involves the analysis of geotechnical risk magnitude and its probability of occurrence. The risk analyst quantifies the cost or schedule impact of the risks in the risk register. The cost and schedule impact of major risks will then be modeled and estimated. The assessment and analysis of geotechnical risks included analysis of the impact of investing in early enhanced site investigative techniques related to geotechnical conditions.
30 This involves the development of an index, like a benefit-cost ratio or other more appropriate parameter, that will provide a quantified value for risk reduction as the investment in early subsurface investigation is increased. This analysis used data collected in Task 3a for the case study projects. A âwhat-ifâ analysis was conducted for each project testing the hypothesis of whether enhanced investigation and analysis, like developing a GBR, would justify the cost of the additional pre-bid investigation by a reduction in the geotechnical risk contingencies in the design-builderâs price proposal. 3. Mitigate and Plan: This step involves studying each major risk factor and deciding what action should be taken to mitigate the negative effect of risk. In a DB geotechnical risk assessment, the action may include accepting, reducing, sharing, transferring, or avoiding the risk. Furthermore, the owner can then decide what portion of the risk has to be assigned to other parties in the DB RFP. The final decision must include differing site condition risks, as well as geotechnical-related design items and temporary works. 4. Allocate: This step consists of assigning the responsibility of major risk factors to various parties, usually through contract documents. A comprehensive allocation of geotechnical risks including differing site condition risks, as well as geotechnical-related design items and temporary works should be included in the DB RFP. 5. Monitor and Control: This last step is to ensure that the identified geotechnical risks listed in the register and proposed mitigation strategies are being implemented. Also, contingency tracking and setting points where risks can be retired as more data becomes available are included. Once the ânumber crunchingâ was complete, the team moved into the analysis portion of this task. The idea was to interpret the correlation output by developing a simplified method to
31 rationally determine a well-understood financial measurement such as return on investment or benefit-cost ratio that can be applied to each case study project. The result is a means to determine whether the resources needed to reduce geotechnical risk for each case study project provided a measurable outcome that justified their expenditure. Lastly, the findings were categorized, permitting a comparison of costs and benefits of geotechnical risk on traditional DBB projects to those found in projects delivered using DB. That comparison included the perceptional data acquired in the Task 2 survey and the Task 3a case studies, and allowed the researchers to identify trends and disconnects between what the respondents âthinkâ is beneficial and what the quantitative analysis âfindsâ adding value. 220.127.116.11 Task 6 Develop Proposed Guidelines for Managing Geotechnical Risk on DB Projects Task 6 entails the research teamâs efforts to flesh out the approved annotated outline for the Guidelines from Task 4 into a fully implementable practice document that can be adapted to align with agency local constraints and preferences. The industry advisory panel was involved in this phase, furnishing direction to the team and an all-important reality check to ensure that the final Guidelines will add value to those agencies that wish to the implement geotechnical risk management tools and techniques contained in the Guidelines. The content of the Guidelines is presented in a manner that fulfills the needs of a typical DOT with little formal DB risk management experience, while adding value for DOTs with longstanding formal risk management programs that may benefit from refining their process. This effort is based on a critical analysis of the case study output, as seen through the lens of the Task 1 literature review and content analysis and the Task 2 survey. Lastly, the results of the US and international effective practice evaluation was used to provide content to the final draft Guidelines.
32 A highly structured approach was taken to author the Guidelines. First, the team revised the Task 3 annotated outline, as reviewed by NCHRP panel as required to comply with panel comments. The team then developed the various sections described in the previous paragraphs. The first section of the Guidelines was written for agency upper management and contains the business case and key management messages. The next sections explain the key principles of geotechnical risk identification and management, as currently practiced, including a selection tool that will match project-specific geotechnical risk characteristics with agency project delivery method constraints. An appendix of contract administration tools and commonly used practices was also included for implementation examples. To ensure that the materials are applicable for practice, a facilitated vetting of the draft Guidelines is scheduled with the Georgia DOT, an agency with significant DB delivery experience. It will then be revised as required and vetted with the California DOT, an agency with limited DB experience. The industry advisory panel furnished oversight and guidance for the vetting session as it was developed. The framework described in the next section is used for both vettings. 18.104.22.168 Guidelines Vetting Framework The framework is based on the five steps in program evaluation developed by UWex (2006) for extension programs. It has been adapted to apply to the evaluation of highway agency guidelines. This section describes the specific tasks that are completed by Guidelinesâ authors. Step 1: Engage Stakeholders The audience for a guidebook is typically DOT employees or independent practitioners. This audience should be involved throughout the evaluation process. This includes considering where and how to conduct the evaluation. Due to restricted funding within a DOT, it is often logical to undertake the evaluation on site, rather than expect staff to
33 travel. Generally, there are multiple people within an organization who will be impacted by the contents of a guidebook; therefore it is wise to include all of these people in the vetting. Step 2: Create a Logic Model A logic model that describes the research projects is created to determine the purpose of the evaluation and what it will indicate when this purpose has been fulfilled. Having a clear idea about what the evaluation results will be used for is important to ensure the entire process stays on track and produces useful information. Step 3: Collect Data The first step is to identify specific DOT staff and practitioners as a key source of data and feedback. Next a workshop to present the key features of the guidebook to this audience was created, ensuring that participants are provided with a copy of the guidebook two weeks before the workshop to allow time for workshop attendees to read it prior. Three methods of data collection were used to establish whether key concepts of the guidebook have been effectively communicated to participants: â End of sessions survey â Observation of participants during the workshop â Focus Group to discuss the practicality of the guidebook Using three data collection/evaluation methods allows the information gathered to be triangulated. Triangulation helps reduce or eliminate the disadvantages of each individual method, while benefiting from the advantages of each (Fellows and Liu
34 2008). Additionally, reoccurring patterns within the feedback should be detectable by the vetting evaluators (Abowitz and Toole 2010). Step 4: Analyze and Interpret First, the collected data was processed into a useful format and analyzed to interpret the key messages from the workshop participants. Next the team established what could be learned from this information within the limitations of the results collected. Step 5: Use Results The knowledge gained from the collected information was used to make decisions on improving the Guidelines. The insights were documented to record how the insights influenced the guidebook revisions. Assess whether the key objectives of the research project have been met. Ask your vetting team the question: Does it fulfill some or all of the outcomes prescribed in within the logic model? The vetting workshop are scheduled take one and a half days, involving a range of stakeholders from the selected DOT such as DOT upper management, design, construction, and contracting personnel; consulting engineering firms with whom the DOT routinely does business; and local construction contractors that would likely compete for the DOTâs DB work. The participants will be divided into integrated teams containing stakeholders from each group and asked to brainstorm through the implementation of a given topical area of a typical project in each project delivery method with which that agency has past experience. For example, the group looking at subsurface utility risk will focus its efforts on ironing out the Guidelines content in that area and identifying those places where the DOT would need to revise the content of geotechnical risk register on the given DB project. The next stage of the vetting would be to prepare feedback detailing recommendations for changes to the draft Guidelines content. The final vetting stage
35 involves presenting the groupâs findings to all the workshop participants and coordinate each topical areaâs feedback with all other areas. The day after the workshop, a meeting with the DOT representatives will be held to assess the previous dayâs results and separate the feedback that applies only to the given DOT and that content which has broader application. 22.214.171.124 Task 7 Prepare Recommended Guidelines for Managing Geotechnical Risk on Design- Build Projects and Final Report Task 7 involves finalizing the Guidelines and the final report. The key deliverable is the Guideline itself. Our team expects that it will be published by either NCHRP or AASHTO. The Final Research Report will likely be a web-based document, to expedite the dissemination of the project results. 2.2 Phase II Work Plan Task Completion The next sections describe how the Phase II work was completed. Identification of major geotechnical risks in NCHRP 24-44 The research team conducted a comprehensive survey of state DOTs followed by interviews of nine DOTs for this effort. One component of this effort was to identify the most significant geotechnical risks in transportation (and specifically highway) projects. The research team then followed through by conducting another survey of geotechnical engineers in various DOTs and contractors and asked them to rank the identified geotechnical risk factors and added any risks as appropriate. A ranked list of risk factors was developed as a result of this effort and is shown in
36 Table 3.18. This ranking is based on responses from 22 DOT employees and 24 contractors. While there were discernible differences between the responses of owners and contractors, this ranking is based on the aggregate rating by the respondents (46 responses). Each respondent was asked to rate the risk factor in terms of frequency and impact on a 1 to 5 scale. A risk score was calculated for each risk factor by multiplying the frequency by impact and risks were ranked according to their risk score. While this listing can be used as an effective checklist to ensure that project team does not disregard major risks, a project risk listing may have more risks identified depending on the projectâs characteristics. Also, interaction between identified risk factors should be considered and accounted for. 2.2.1 Risk Quantification Approach The literature review showed that a range of established methods are available for assessing the geotechnical risk impacts. These methods can largely be divided into two groups: qualitative and quantitative (Figure 2. 4). Risk assessments conducted at the conceptual design level generally involves a qualitative risk assessment where major risks are identified and ranked. Risk assessments conducted in the later design phases require a higher level of detail and the risks are quantified by the risk analysis team. Evaluation of risks from the preliminary design, to the final design phases (or in the case of DB projects from conceptual to preliminary engineering) which goes beyond merely identifying and rating of risk factors and involves quantification of each risk factor, is called quantitative risk assessment. Detailed descriptions of these processes are described in several sources including state DOT risk manuals (WSDOT 2014; VDOT 2015). These approaches are well established and have demonstrated success.
37 Figure 2. 4 Risk Quantification Approaches 126.96.36.199 Qualitative Risk Assessment The qualitative approach is quite subjective and based on the perception of the project development team regarding the type of risks to be encountered and their impact on project. At the very early stages of development, the best that the team can do is to identify some potential risks based on their knowledge of the site conditions and maybe some indication that the risk is a major or a minor one. As more data becomes available and scope gains clarity, the project team might be in a position to rate the risk in terms of frequency and impact. Note that at this early stage, it may not be possible to put a dollar value or delay time on the risk factor so a rating similar to Likert scale (for example from 1 to 5 for frequency and impact) is assigned to each risk factor. A risk score is commonly calculated based on these two factors. This risk score is the expected magnitude of the identified risk (Equation 2.4). Risk score = Risk frequency x Risk impact Eq. 2.41
38 The results of the NCHRP 24-44 survey was used to validate that geotechnical risk assessment is performed as part of the overall project risk assessment by most DOTs. In other words, no independent formal geotechnical risk assessment is conducted. If the appropriate personnel are present at the risk assessment, this approach work wells, as long as the geotechnical risk assessment results are highlighted and used in the decision to identify primary geotechnical design requirements. The traditional risk assessment methods are acceptable for quantifying and mitigating geotechnical risks. 188.8.131.52 Proposed Two-step Approach for Calculating Project Risk Score This approach is developed to help the agency assess the level of geotechnical risk in the project and to determine the approximate cost of developing the geotechnical design document to the level that can be released for DB bidding. Based on this analysis, the project team will be able to assess the magnitude of the geotechnical risk picture very early in project development phase and determine if the DB approach is advisable. The process for calculating the project risk score could be classified as a semi-quantitative approach, because it uses a qualitative approach for rating the risk and then attempts to estimate the cost and schedule impacts. The NCHRP research team has developed a tool for calculating the project risk score by combining a preliminary risk assessment with a conceptual cost estimate (Figure 2. 5). The proposed impact-likelihood matrix is prepared according to practices of various DOTs. The purpose here is twofold: (1) to assess a preliminary risk score for a project and classify the project as high risk, medium risk, or low risk, all in the context of geotechnical risks, and (2) to evaluate the identified risks in terms of their potential impact on the cost and time of required for additional geotechnical investigation.
39 Figure 2. 5 The Proposed Two-step Procedure for Estimating Project Risk Score The zones of high, medium, and low risk in the matrix shown are based a synthesis DOT risk management documentation and published research (WSDOT 2014; Clayton 2001). The synthesis proposes the initial score boundaries shown in Figure 2. 5, which can be fine-tuned in accordance with local policy/practice. For example, Washington State DOT considers every risk factor with very high impact as a high risk and that approach is reflected in the proposed model. Two-step procedure: STEP 1 Likelihood FirstÂ startÂ withÂ aÂ twoâdimensionalÂ matrix. 5 5 10 15 20 25 ConsiderÂ allÂ risksÂ withÂ impactÂ ofÂ 5Â asÂ high risk. 4 4 8 12 16 20 RiskÂ scoresÂ ofÂ 15Â orÂ higherÂ willÂ beÂ consideredÂ high risk. 3 3 6 9 12 15 RisksÂ withÂ likelihoodÂ ofÂ 5Â areÂ ratedÂ asÂ atÂ leastÂ medium. 2 2 4 6 8 10 RiskÂ scoresÂ ofÂ 8Â toÂ 12Â isÂ consideredÂ medium. 1 1 2 3 4 5 RiskÂ scoresÂ ofÂ 6Â orÂ lessÂ areÂ consideredÂ low unlessÂ theÂ likelihoodÂ isÂ 5. 1 2 3 4 5 Risk Likelihood (1-5) Impact (1-5) Risk score (1-25) Liquifaction 1 2 2 HighÂ Risk ScourÂ ofÂ bridgeÂ piers 2 2 4 MediumÂ Risk Settlement 2 2 4 LowÂ Risk RockÂ andÂ boulders 4 4 16 ContaminatedÂ materials 2 4 8 0 0 0 0 SUM 34 Rate Overall Project Geotechnical Risk IfÂ theÂ totalÂ projectÂ riskÂ scoreÂ isÂ 40Â orÂ larger,Â theÂ projectÂ isÂ classifiedÂ asÂ high risk. IfÂ theÂ totalÂ projectÂ riskÂ scoreÂ isÂ betweenÂ 20Â andÂ 39,Â theÂ projectÂ isÂ classifiedÂ asÂ medium risk. IfÂ theÂ totalÂ projectÂ riskÂ scoreÂ isÂ lessÂ thanÂ 20,Â theÂ projectÂ isÂ classifiedÂ as low risk. STEP 2 InÂ thisÂ case,Â firstÂ theÂ mostÂ importantÂ risksÂ areÂ identified.Â OnlyÂ thoseÂ significantÂ risksÂ willÂ beÂ evaluatedÂ forÂ theÂ potential DSCÂ cost.Â ThisÂ potentialÂ costÂ canÂ beÂ estimatedÂ eitherÂ asÂ aÂ $Â valueÂ orÂ percentÂ ofÂ projectÂ cost.Â This potential cost is the cost of further geotech investigations before letting the DB contract. Risk Risk Score Effort to mitigate risk ($) Effort to mitigate risk (duration) RockÂ andÂ boulders 16 50,000$Â Â Â Â Â 2Â months ContaminatedÂ materials 8 190,000$Â Â Â Â Â Â Â Â 3Â months 0 0 0 0 0 0 0 Total 240,000$ 3 months* * TheÂ totalÂ durationÂ impactÂ willÂ beÂ theÂ largestÂ ofÂ individualÂ mitigationÂ durationsÂ assumingÂ concurrentÂ investigation.
40 The method adds the âEffort to mitigate riskâ as the third factor for rating a major risk factor. This came from Missouri DOT practice. It allows the analyst to assess the cost and time impact of rectifying any major risk. As an alternative, one can use this matrix to assess the exposure to budget and schedule changes if further geotechnical investigation is not carried out before finalizing the DB contract. In the first step, the focus is on identifying geotechnical risks and assigning a Likert score of 1 (low) to 5 (high) to the risk likelihood and risk impact. The product of these two ratings (that can be a number between 1 and 25) is the risk score for that specific risk factor. In the second step, those risk factors with a risk score of higher than the desired threshold will be investigated as to the level of effort and cost needed to obtain more geotechnical data and testing before the project can be advertised for DB bidding. This two-step process can be repeated after new information and data becomes available for the geotechnical design. The process is complete when the total project risk score falls below an established threshold. It is believed that this qualitative risk assessment process should be sufficient for projects that are either small in size or less complex in nature. Project complexity has a significant effect on the depth and extent of risk analysis and can be measured along several facets. As an example, the SHRP2 R10 project developed a method for measuring project complexity which depended on cost, schedule, financing, technical issues, and context (Gransberg 2015). While complexity of the project will have a bearing on overall project risk assessment, this research is interested in the geotechnical risks and hence focused on geotechnical complexity. While the two-step process described above should be sufficient for dealing with geotechnical risks in routine projects with limited size and scope, for larger more complex projects
41 it might be necessary to conduct a more detailed quantitative geotechnical risk assessment. The next section describes this process. 184.108.40.206 Quantitative Risk Assessment The proposed quantitative risk assessment approach is a more involved process; however, the basic tools are still the same. The team starts with the list of identified geotechnical. The most significant risks are entered into a risk register. Several examples of risk registers are given in Clayton (2001). For each risk factor in the risk register, the team has to estimate the probability of occurrence (based on subjective knowledge of the geotechnical experts and possibly some statistics from similar past projects) and the impact of risk on project cost and schedule. The cost and schedule impact may be estimated by a single value (deterministic) or with a range (probabilistic). The combined impact of all these risks is the total risk impact on the project. There are several mathematical approaches for calculation of total risk impact, including approximate analytical methods and Monte Carlo simulation techniques. Detailed descriptions of these methods are given in DOT manuals and published books and reports (WSDOT 2015; Hollman 2016; MBTA 2011). The purpose of risk assessment in this research is to quantify the risk of geotechnical factors on project cost and schedule. As such, the same general methodology for whole-project risk assessment can be used, but with emphasis on geotechnical risks. We propose to use a risk register for this purpose. A sample risk register, adapted from Virginia DOT and shown in Fig 2.7 is suggested for this purpose. Any significant geotechnical risks identified for the project should be listed in this table. For each risk, the likelihood (probability of that risk event happening) should be estimated along with the potential cost and schedule of the risk impacts. Probability of occurrence is estimated using established protocols to estimate the likelihoods (Van Staveren
42 2007; Von Winterfeldt and Edwards 1986). One approach commonly used by the risk analysis team is to poll the experts and calculate the average value of the opinions. The impact of the risk if realized, is estimated by a range of possible values. Various probability distributions have been used to estimate the cost and schedule impact of risks. A full discussion of these distributions is beyond the scope of this report but normal distribution, lognormal distribution, PERT, uniform and triangular distributions have been used for this purpose. A common approach is to bracket the cost or delay by providing a pessimistic and an optimistic value and then identify the most likely cost or delay. So, each risk cost or delay is estimated with three estimates representing optimistic, most likely, and pessimistic scenarios. An example of a hypothetical analysis is provided in Fig 2.6. Four risk factors are listed and the probability of occurrence is estimated. The cost impact is modeled as triangular distributions with ranges elicited from subject matter experts (Fig. 2.9). As an example, the potential cost of the risk factor âDamage caused by differential settlementsâ is modeled with a triangular distribution with the following values: Optimistic cost = $25,000; most likely cost = $100,000; pessimistic cost = $250,000. Such estimates are provided for all the risk factors. The estimated costs are arrived at by the subject matter experts that conduct the risk assessment in a workshop setting. Sometimes these values are completely subjective based on subject matter expertsâ personal experience with previous projects and sometimes these costs are estimated based on reasonable assumptions. For each risk factor probability of occurrence is multiplied by the cost distribution. This will result in the expected value of the risk factor. These expected values are summed up. The sum of these expected values is simulated using a Monte Carlo simulation approach and is presented in Fig 2.8. Commercial software is available that can perform Monte Carlo simulation. There are non-
43 simulation options for calculating the total impact of risk also. A detailed discussion of these computational methods is beyond the scope of this discussion. The results in Fig 2.8.a show that the total geotechnical risks can have a cost impact ranging from $0k (in case none of the four risk factors materializes) to $740k with the most likely cost impact being about $200k. This is the extent of damage if project risks are not mitigated. In case the agency is interested in setting up a contingency exclusively for geotechnical risks, then a $300k contingency will provide a confidence level of 80% against geotechnical risks (Fig 2.8.b). This analysis can also be used as a tool to assess the necessity of further geotechnical investigations before going to bid. If the magnitude of risk is beyond the risk tolerance of the agency, then it would make sense to develop the geotechnical risk further and clarify the risk picture. Going back to the flowchart of (Stage 3), one can see that in case the project is classified as high risk, and if the letting of the DB project as a progressive DB is permitted by law, then it could provide a reasonable approach to deal with geotechnical risks.
44 R IS K A SS E SS M E N T R IS K R E SP O N SE P ro ba bi lit y Im pa ct P ro ba bi lit y * Im pa ct R es po ns ib le Te am M em be r M et ho d Ti m e/ B ud ge t Im pa ct C om m en ts /N ot es M iti ga tio n St ra te gy P ro ba bi lit y Im pa ct P ro ba bi lit y x Im pa ct TO TA L PR O JE C T # R IS K R EG IS TE R N o. R isk F ac to r R E SI D U A L R IS K A SS E SS M E N T (A F TE R A P P L IE D S TR A TE G Y ) D at e N o. R is k Fa ct or R IS K A SS ES SM EN T R IS K R ES PO N SE Pr ob - ab ili ty Im pa ct Pr ob ab ili ty * Im pa ct R es po ns ib le Te am M em be r M et ho d Ti m e/ B ud ge t Im pa ct Pr ob - ab ili ty Im pa ct Pr ob ab ili ty * Im pa ct 1 U nid en tif ied u tili tie s c au sin g de lay s du rin g co ns tru ct io n 50 % $ 2 50 ,0 00 12 5, 00 0 $ U tili ty SU E 50 ,0 00 $ 10 % 10 0, 00 0 $ 10 ,0 00 $ 2 D am ag e ca us ed b y di ffe re nt ial se ttl em en ts w ith in br id ge fo ot pr int a fte r co ns tru ct io n 20 % $ 1 25 ,0 00 25 ,0 00 $ G eo te ch A dd itio na l te sti ng 10 ,0 00 $ 10 % 50 ,0 00 $ 5, 00 0 $ 3 U nk no w n gr ou nd w at er fl ow d ire ct io n 5% $ 58 ,3 33 2, 91 7 $ Si te T ea m A cc ep t - $ 5% 58 ,3 33 $ 2, 91 7 $ 4 D am ag e to n eig hb or ing p ro pe rti es b y he av e du e to e xc av at io n 10 % $ 1 66 ,6 67 16 ,6 67 $ G eo te ch Tr an sfe r 20 ,0 00 $ 10 % - $ - $ TO TA LS 16 9, 58 3 $ 80 ,0 00 $ 17 ,9 17 $ R IS K R E G IS TE R PR O JE C T # EX A M PL E G eo te ch ni ca l/S ub su rf ac e R is ks D at e G eo te ch ni ca l E xt ra ct (A FT ER A PP LI ED S TR A TE G Y) R ES ID U A L R IS K A SS ES SM EN T Fi gu re 2 . 6 R is k R eg is te r f or Q ua nt ita tiv e R is k A ss es sm en t ( A da pt ed fr om V D O T R is k M an ag em en t M at rix ) Fi gu re 2 . 7 P ar tia lly fi lle d- ou t R is k R eg is te r w ith id en tif ie d R is k Fa ct or s
45 Fi gu re 2 . 8 C os t E st im at e fo r R is k Fa ct or s Fi gu re 2 . 9 T ot al C os t f or G eo te ch ni ca l R is ks (a - P ro ba bi lit y di st rib ut io n; b - C um ul at iv e di st rib ut io n)
46 220.127.116.11 Risk Mitigation and Monitoring The last phase of risk management exercise is risk mitigation and monitoring. In the design-build approach, the ownerâs control is somewhat limited compared to the traditional DBB or construction manager/general contractor (CMGC) delivery methods. However, it is imperative that the contractor keep track and implement the risk mitigation measures during the construction. The general objective of risk mitigation is shown in Fig 2.10. The collective effect of mitigation measures is to reduce the uncertainty in project costs. The graph shows that due to risk mitigation efforts, the owner hopes to reduce the overall costs. More realistic (and maybe more important) is to expect that mitigation efforts will reduce the overall risk variance and in effect reduce project uncertainty. Figure 2.10 The Effect of Mitigation Effort Distribution for Expected Values Values in Thousands Original CostÂ Distribution Mitigated costÂ distribution 0.000 0.200 0.400 0.600 0.800 1.000 5% 90% 5% 414.0026 425.8417 Mean=4.198461E+05
47 The DOT survey found that the owners usually do not often require the contractor to maintain a risk register and to implement and monitor the ownerâs risk management program. The premise that optimizing the exposure to geotechnical risk demands that the owner involve the DB Contractor, resulting in a joint post-award formal risk assessment and implement the mitigation efforts will benefit both the owner and the contractor. There are a number of strategies for mitigating the negative impact of risks on project schedule and cost. These include: ï· Avoiding the risk: this strategy can be used in cases where the risk is of such magnitude that re-planning the design is required, for example not using a ground stabilization technique, such as ground freezing, because of lack of precedence or expertise in an area. This can apply to risks with high probability of occurrence and serious consequences. ï· Sharing the risk: the agency can share the risk with the design-builder. As an example, many DOTs will not compensate the contractor for DSC claims unless the amount surpasses a certain threshold. ï· Reducing the risk: this strategy can help the project move forward in the face of uncertainty. If the site characterization is incomplete, the owner may opt to perform more testing and spend some upfront money to better scope out the site conditions. This results in more competitive bids and a reduction of contingency by the bidders. ï· Transferring the risk: typical example is the use of insurance. Damage to adjacent properties due to settlements, catastrophic events, etc. could be examples of risk transfer insurance. It should be noted that insurance only transfers the ownership of risk and the associated liability but does not reduce the effect of risk factor (or hazard) on the project.
48 ï· Accepting the risk: finally, this strategy can be used by the owner to deal with risks that are relatively rare and/or have limited impact. The owner usually accepts some risks by establishing a contingency budget. A DSC clause in DBB contracts is an example of risk acceptance by the owner.