National Academies Press: OpenBook

A Guidebook for the Evaluation of Project Delivery Methods (2009)

Chapter: Chapter 6 - Tier 3 Optimal Risk-Based Approach

« Previous: Chapter 5 - Tier 2 Weighted-Matrix Delivery Decision Approach
Page 87
Suggested Citation:"Chapter 6 - Tier 3 Optimal Risk-Based Approach." National Academies of Sciences, Engineering, and Medicine. 2009. A Guidebook for the Evaluation of Project Delivery Methods. Washington, DC: The National Academies Press. doi: 10.17226/14238.
×
Page 87
Page 88
Suggested Citation:"Chapter 6 - Tier 3 Optimal Risk-Based Approach." National Academies of Sciences, Engineering, and Medicine. 2009. A Guidebook for the Evaluation of Project Delivery Methods. Washington, DC: The National Academies Press. doi: 10.17226/14238.
×
Page 88
Page 89
Suggested Citation:"Chapter 6 - Tier 3 Optimal Risk-Based Approach." National Academies of Sciences, Engineering, and Medicine. 2009. A Guidebook for the Evaluation of Project Delivery Methods. Washington, DC: The National Academies Press. doi: 10.17226/14238.
×
Page 89
Page 90
Suggested Citation:"Chapter 6 - Tier 3 Optimal Risk-Based Approach." National Academies of Sciences, Engineering, and Medicine. 2009. A Guidebook for the Evaluation of Project Delivery Methods. Washington, DC: The National Academies Press. doi: 10.17226/14238.
×
Page 90
Page 91
Suggested Citation:"Chapter 6 - Tier 3 Optimal Risk-Based Approach." National Academies of Sciences, Engineering, and Medicine. 2009. A Guidebook for the Evaluation of Project Delivery Methods. Washington, DC: The National Academies Press. doi: 10.17226/14238.
×
Page 91
Page 92
Suggested Citation:"Chapter 6 - Tier 3 Optimal Risk-Based Approach." National Academies of Sciences, Engineering, and Medicine. 2009. A Guidebook for the Evaluation of Project Delivery Methods. Washington, DC: The National Academies Press. doi: 10.17226/14238.
×
Page 92

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Introduction The Tier 3—Optimal Risk-based Approach leverages risk-based cost-estimating methods that have emerged in transit and highway agencies in the past few years (Touran, Bolster, and Thayer 1994; Parsons, Touran, and Golder 2004). Tier 1 and Tier 2 approaches should be completed before the Tier 3 approach is introduced. Most of the time, it will be possible to make the deliv- ery method decision by completing Tiers 1 and 2. Even if a clear choice cannot be established after going through the first two tiers, at least the completion of the first two tiers will yield a short list of viable choices. It is expected that by the time decision-makers get to Tier 3, they are looking at only two delivery method candidates. It is important that there are only two delivery method candidates because the effort involved in using Tier 3 (especially the quantitative approach) is considerably larger than effort involved in either Tier 1 or Tier 2. The Tier 3 approach consists of two phases. The first phase involves a qualitative analysis: devel- oping a risk-allocation matrix that clearly portrays an owner’s risk under competing delivery methods. Through review of these risks, the owner (in this context, mostly transit agencies) will have an opportunity to decide whether a specific delivery method is more appropriate than others. If the qualitative analysis does not provide a definitive answer to the delivery selection question, the second phase—a quantitative analysis—should be considered. The quantitative approach emphasizes the effect of the project delivery method on project cost and schedule. The two-phase process (depicted in Figure 6.1) should be repeated for each project delivery method that survives the Tier 2 process. Due to cost escalation on large transit projects, since 2002, the FTA has required that each “New Starts” project undergo a formal risk-based cost estimate. Specific requirements for these risk assessments are provided in FTA guidance documents such as “PMO Operating Procedures No. 40, Risk Management Products and Procedures” (2007). A risk-based cost estimate gener- ates a range of possible project costs rather than a single point estimate, as shown in Figure 6.2. This distribution represents the combined effect of various risks that affect project cost. Using this distribution, the project owner would be able to estimate the probability of finishing the project within a specified budget. Alternatively, the owner can establish a sufficient contingency budget to keep the probability of cost overrun or schedule delay below a specified threshold. The same modeling method (and much of the same data) that is used to generate the cost and schedule risk analysis can be used to make more informed decisions and allocate risks appropri- ately, in essence, optimizing the project delivery and contracting decisions. One of the major findings of the structured interviews (conducted with transit agencies as part of this research effort) was the apparent effect of a rigorous risk analysis on project success. It 87 C H A P T E R 6 Tier 3—Optimal Risk-Based Approach

was found that projects in which more attention was paid to risk analysis fared better than other projects in terms of meeting budget and schedule goals. The following sections describe the qualitative and quantitative phases of the Tier 3 approach in more detail. Qualitative Analysis Figure 6.3 shows the risk-based approach superimposed on the project lifecycle. The most likely times to decide on the project delivery method are at the end of the Conceptual Design Phase or during the Preliminary Engineering Phase. If a project goes into the Final Design Phase without a decision on a project delivery method, the agency will lose the opportunity to effec- tively use alternative delivery methods and will be limited to the traditional DBB approach. At the end of the Conceptual Design Phase, the agency usually has not done a detailed risk analy- sis. If an agency is unable to select a project delivery method upon completion of the Tier 1 and Tier 2 approaches, it would need to conduct a preliminary risk analysis in order to make an informed choice of project delivery method. The result of this preliminary risk analysis is a risk-allocation matrix. The risk-allocation matrix has become an industry standard for legal teams when authoring alternative contracts for large infrastructure projects. For example, a risk-allocation matrix was a first step in creating the contract for the T-REX multimodal DB project in Colorado. Table G-1 in Appendix G (avail- able on the TRB website at http://trb.org/news/blurb_detail.asp?id=10054) presents a generic risk-allocation matrix that can be used for accomplishing the qualitative analysis. It should be noted that the matrix of Table G-1 will most likely consist of only two (and in rare cases maybe three) delivery methods because the completion of the Tier 1 and Tier 2 approaches should 88 A Guidebook for the Evaluation of Project Delivery Methods Qualitative Approach: Risk-allocation matrix Quantitative Approach: Cost/ schedule impact Document Results Select project delivery method; Document Results Selection? No Yes Figure 6.1. Overview of the risk-based qualitative and quantitative approaches. Total Project Cost (Year of Expenditure $M) 0 0.02 0.04 0.06 0.08 25 00 27 00 29 00 31 00 33 00 35 00 37 00 39 00 0.1 Pr ob ab ili ty Figure 6.2. Distribution of project costs.

reduce the number of possible alternatives. Table 6.1 shows a risk-allocation matrix for a hypo- thetical project. A description of the development of this risk-allocation matrix is given below. Risk factors (shown the first column in Table 6.1 and typically arranged in a matrix according to either their impact [rank] or chronology) are major events or conditions that can affect a proj- ect in a negative way (the events that can affect the project in a positive way are called “opportu- nities,” and traditionally there are far fewer opportunities than risks). Only significant risks should be considered because identifying and measuring all project risks would be a major effort. Under each project delivery method listed in the matrix, a main responsible party should be identified for each risk factor. For example, in the matrix shown in Table 6.1, the party responsible for design defects in a DBB contract is the owner, whereas in the DB contract, the responsible party is the constructor. Risk factors are rated, always from the perspective of the owner agency, according to the effect of a particular project delivery method on that risk factor. In the hypothetical case shown in Table 6.1, from the agency perspective, DBB is seen as having a favorable effect on the risk fac- tor of “permits/approvals.” The agency thinks that it is the best party to obtain permits/approvals and that it can most effectively do this using a DBB approach. Therefore, the risk factor of Tier 3—Optimal Risk-Based Approach 89 Pr oje ct Lif ec yc le Pr oje ct De liv er y M et ho d (P DM ) S ele cti on Risk AnalysisPreliminaryRisk Analysis PDM Selection Verify Selected PDM Preliminary Engineering Final DesignConceptual Design Risk-Allocation Matrix Qualitative Evaluate Decision Document Results and Stop Successful? Document Results and Stop Quantitative Figure 6.3. Risk-based approach superimposed on project lifecycle. Table 6.1. Risk-allocation matrix for a hypothetical project. DBB DBRisk Factor Responsible Party Rating Responsible Party Rating Permits/Approval Owner + Constructor/Owner Different Site Conditions Owner 0 Constructor/Owner + Design Defects Owner Constructor + Quality Assurance/ Quality Control Constructor/Owner 0 Constructor + Exchange Rate Risk Owner Owner Other Risk Factors

permits/approvals has received a rating of +. The same risk factor, under a DB delivery method, is seen as unfavorable from the agency’s point of view because the agency thinks that the DB con- structor is not the best party to obtain various permits and approvals (such as environmental permits). Therefore, a rating of − is assigned. Another risk factor in the hypothetical example, “design defects,” has a rating of − under the DBB arrangement because in this delivery method the agency is responsible for the accuracy of design. A DB approach, on the other hand, is rated + because it transfers this risk to the constructor. If the choice of a project delivery method has no effect on a particular risk factor, then a rating of 0 will be assigned. In rating each risk factor, one can refer to the contents of Chapter 3 of this guide, where the advantages and disadvantages of various project delivery methods in relation to 24 pertinent issues are documented. No attempt is made at this stage of the Tier 3 analysis to quantify the impact of these risk fac- tors (in terms of $ value or project delay). After the matrix is developed and the risk factors rated, the evaluation team can review the outcome and see if any project delivery method seems supe- rior in terms of its capacity in dealing with these risk factors. For example, a review of the matrix in Table 6.1 may suggest that DB is the better choice for the owner agency because of the num- ber of favorable ratings that it obtained. Preparation of the risk-allocation matrix and rating the risk factors can be accomplished in a reasonable amount of time. If the outcome suggests a “most appropriate” project delivery method, then the decision is finalized and the results, along with justification, are documented. If, after going through the process, the choice is still not clear, then the Tier 3 process should con- tinue on to the second phase—the quantitative analysis. Quantitative Analysis The quantitative approach should be attempted only if the qualitative approach does not result in a clear delivery method choice for a project. As shown in Figure 6.3, it is suggested that the Tier 3 quantitative analysis occur at the conclusion of the preliminary engineering phase, after the agency has conducted the FTA-mandated probabilistic risk analysis of project cost and sched- ule. The risk analysis is a major undertaking that requires hundreds of person-hours over the course of several weeks. Also, the outcome of the risk analysis can inform the project delivery method selection process (see Figure 6.4). The quantitative phase of Tier 3 would then be con- tingent on the availability of the complete risk analysis. If this risk analysis is not a requirement (for example in projects that do not apply for federal funding), then it is suggested that the proj- 90 A Guidebook for the Evaluation of Project Delivery Methods Risk Analysis Cost/Schedule - Ranked Risk Factors - Risk Factor 1 - Risk Factor 2 … Quantitative Approach (See Fig. 6-5) Figure 6.4. Risk analysis outcome as an input to project delivery method selection.

ect delivery method selection decision be made without this phase as the cost of this phase could be prohibitive. The outcome of the probabilistic risk analysis required by the FTA consists of a distribution (range of possible values) for project cost and duration. Also, a list of the most important risk factors, ranked according to their impact on budget or schedule, is provided as part of the risk mitigation report. Usually, the number of these ranked risks is limited (e.g., in several risk assess- ments conducted by the project management oversight (PMO) consultants on behalf of the FTA, the list of significant risk factors included 10 to 15 risk factors). The FTA analysis follows the logic of Pareto’s law (also known as the 80-20 rule and the law of the vital few), which states that for many events, 80% of the effects come from 20% of the causes. In the context of project risks, rel- atively few risks are responsible for most of the project cost or schedule overruns. The project cost distribution and the list of ranked risks serve as inputs to the process of selecting the best project delivery method. For each ranked risk, a distribution of risk costs is usually estimated. The highest ranked risks are those with large expected values and large ranges (an indication of high variability in the risk factor). The proposed process, called the quantitative approach in this work, will involve estimating the effect of each major risk factor on the agency’s budget, given a specific delivery method. The process starts by reviewing all the risk factors and selecting the risk factors whose value will be affected by the choice of project delivery method. Only the risk factors that are sensitive to the project deliv- ery method will be selected for further analysis. For each of these risk factors, the range of cost will be estimated under a given project delivery method. This estimation can best be accomplished by some of the same experts involved in the risk analysis. Figure 6.5 provides an example of a hypo- thetical project in which four major risk factors have been identified as the risk factors that are affected by the choice of project delivery method and the two remaining candidates for delivery method are DBB and DB. The risk factors are the following: permits, utility relocation, differing site conditions (DSC), and third-party issues. The cost of each risk is estimated using a triangular distri- bution, although many other distributions can be used depending on the nature of the risk factor.4 The sum of these risk costs will give the distribution for the total risk costs. There are statistical methods that can be used to calculate this sum with relative ease. Comparison of distributions of these total risk costs will give the owner agency a valuable tool for assessing the effect of project delivery method on project cost. A similar approach can be used to assess the effect of risks on project schedule. If the purpose of the risk analysis is to examine the effect of delivery method on project duration, all the distributions depicted in Figure 6.5 would have durations on the X-axis and the total effect will be the total impact on project schedule instead of on project cost. The quantitative analysis is a powerful tool for comparing competing project delivery methods. It focuses on those differences between project delivery methods that affect cost and schedule and provides a consistent way of evaluating each project delivery method vis-à-vis major risk factors affecting the project. This analysis allows the decision-maker to document the reasons for the selection of a specific project delivery method. The drawback of this analysis is its dependency on the availability of expensive risk analysis results and the higher skill level required for pricing out each risk under various project delivery methods. However, the choice of the project deliv- ery method is a natural outcome of a risk analysis exercise because one of the most important benefits of any risk analysis is risk allocation/mitigation. A properly selected project delivery method is an effective risk mitigation instrument that can help keep project costs low and min- imize project delays. Tier 3—Optimal Risk-Based Approach 91 4In a triangular distribution, the range of possible values is estimated with a lower bound (optimistic), an upper bound (pessimistic), and a most-likely value. The triangular distribution is commonly used in probabilistic risk analysis because of its simplicity.

Conclusion The Tier 3 approach may be needed in cases where the Tier 1 and Tier 2 approaches cannot provide a clear best choice for the project delivery method. In such a case, the Tier 3 provides a two-phased approach: first, a qualitative analysis and then, if necessary, a quantitative one. Both analyses are based on a risk-allocation exercise that will determine major risks to the agency under various project delivery methods. In the qualitative approach, the decision-makers care- fully examine each risk factor and deliberate the anticipated effect of each risk factor on project cost and schedule. This critical review can help the agency decide on the most appropriate deliv- ery method. If the qualitative analysis does not yield a final choice of delivery method, the agency can then proceed with the quantitative analysis. In this analysis, the cost and schedule effect of each risk factor is estimated within an appropriate range, summed up, and used in comparing the total effect of risks under competing delivery methods. The agency can then select the deliv- ery method that results in the most favorable outcome considering both cost and schedule. 92 A Guidebook for the Evaluation of Project Delivery Methods DBB DB $ Pr ob . Utility DSC Third Party Total Effect on Cost Permits $ $ $ $ $ $ $ $ $ Pr ob . Pr ob . Pr ob . Pr ob . Pr ob . Pr ob . Pr ob . Pr ob . Pr ob . Figure 6.5. Overview of the quantitative analysis.

Next: Chapter 7 - Summary »
A Guidebook for the Evaluation of Project Delivery Methods Get This Book
×
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

TRB’s Transit Cooperative Research Program (TCRP) Report 131: A Guidebook for the Evaluation of Project Delivery Methods examines various project delivery methods for major transit capital projects. The report also explores the impacts, advantages, and disadvantages of including operations and maintenance as a component of a contract for a project delivery method.

A companion publication to TCRP Report 131 isTCRP Web-Only Document 41: Evaluation of Project Delivery Methods, which explores pertinent literature and research findings related to various project delivery methods for transit projects. TCRP Web-Only Document 41 also includes definitions of project delivery methods and highlights the existing selection approaches commonly used by transit agencies.

Appendix A: References and Appendix B: Definitions were published as part of TCRP Report 131. Appendices C through H of the report are available online.

Appendix C: Forms for Project Description and Goals

Appendix D: Forms for the Analytical Delivery Decision Approach (Tier 1)

Appendix E: Forms for the Weighted-Matrix Delivery Decision Approach (Tier 2)

Appendix F: Procedures for Determining the Weights of Selection Factors in the Weighted-Matrix Delivery Decision Approach (Tier 2)

Appendix G: Form for the Optimal Risk-Based Approach (Tier 3)

Appendix H: Application of the Tier 1 and Tier 2 Approaches to a Hypothetical Project

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!