Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Introduction The Tier 3âOptimal Risk-based Approach leverages risk-based cost-estimating methods that have emerged in transit and highway agencies in the past few years (Touran, Bolster, and Thayer 1994; Parsons, Touran, and Golder 2004). Tier 1 and Tier 2 approaches should be completed before the Tier 3 approach is introduced. Most of the time, it will be possible to make the deliv- ery method decision by completing Tiers 1 and 2. Even if a clear choice cannot be established after going through the first two tiers, at least the completion of the first two tiers will yield a short list of viable choices. It is expected that by the time decision-makers get to Tier 3, they are looking at only two delivery method candidates. It is important that there are only two delivery method candidates because the effort involved in using Tier 3 (especially the quantitative approach) is considerably larger than effort involved in either Tier 1 or Tier 2. The Tier 3 approach consists of two phases. The first phase involves a qualitative analysis: devel- oping a risk-allocation matrix that clearly portrays an ownerâs risk under competing delivery methods. Through review of these risks, the owner (in this context, mostly transit agencies) will have an opportunity to decide whether a specific delivery method is more appropriate than others. If the qualitative analysis does not provide a definitive answer to the delivery selection question, the second phaseâa quantitative analysisâshould be considered. The quantitative approach emphasizes the effect of the project delivery method on project cost and schedule. The two-phase process (depicted in Figure 6.1) should be repeated for each project delivery method that survives the Tier 2 process. Due to cost escalation on large transit projects, since 2002, the FTA has required that each âNew Startsâ project undergo a formal risk-based cost estimate. Specific requirements for these risk assessments are provided in FTA guidance documents such as âPMO Operating Procedures No. 40, Risk Management Products and Proceduresâ (2007). A risk-based cost estimate gener- ates a range of possible project costs rather than a single point estimate, as shown in Figure 6.2. This distribution represents the combined effect of various risks that affect project cost. Using this distribution, the project owner would be able to estimate the probability of finishing the project within a specified budget. Alternatively, the owner can establish a sufficient contingency budget to keep the probability of cost overrun or schedule delay below a specified threshold. The same modeling method (and much of the same data) that is used to generate the cost and schedule risk analysis can be used to make more informed decisions and allocate risks appropri- ately, in essence, optimizing the project delivery and contracting decisions. One of the major findings of the structured interviews (conducted with transit agencies as part of this research effort) was the apparent effect of a rigorous risk analysis on project success. It 87 C H A P T E R 6 Tier 3âOptimal Risk-Based Approach
was found that projects in which more attention was paid to risk analysis fared better than other projects in terms of meeting budget and schedule goals. The following sections describe the qualitative and quantitative phases of the Tier 3 approach in more detail. Qualitative Analysis Figure 6.3 shows the risk-based approach superimposed on the project lifecycle. The most likely times to decide on the project delivery method are at the end of the Conceptual Design Phase or during the Preliminary Engineering Phase. If a project goes into the Final Design Phase without a decision on a project delivery method, the agency will lose the opportunity to effec- tively use alternative delivery methods and will be limited to the traditional DBB approach. At the end of the Conceptual Design Phase, the agency usually has not done a detailed risk analy- sis. If an agency is unable to select a project delivery method upon completion of the Tier 1 and Tier 2 approaches, it would need to conduct a preliminary risk analysis in order to make an informed choice of project delivery method. The result of this preliminary risk analysis is a risk-allocation matrix. The risk-allocation matrix has become an industry standard for legal teams when authoring alternative contracts for large infrastructure projects. For example, a risk-allocation matrix was a first step in creating the contract for the T-REX multimodal DB project in Colorado. Table G-1 in Appendix G (avail- able on the TRB website at http://trb.org/news/blurb_detail.asp?id=10054) presents a generic risk-allocation matrix that can be used for accomplishing the qualitative analysis. It should be noted that the matrix of Table G-1 will most likely consist of only two (and in rare cases maybe three) delivery methods because the completion of the Tier 1 and Tier 2 approaches should 88 A Guidebook for the Evaluation of Project Delivery Methods Qualitative Approach: Risk-allocation matrix Quantitative Approach: Cost/ schedule impact Document Results Select project delivery method; Document Results Selection? No Yes Figure 6.1. Overview of the risk-based qualitative and quantitative approaches. Total Project Cost (Year of Expenditure $M) 0 0.02 0.04 0.06 0.08 25 00 27 00 29 00 31 00 33 00 35 00 37 00 39 00 0.1 Pr ob ab ili ty Figure 6.2. Distribution of project costs.
reduce the number of possible alternatives. Table 6.1 shows a risk-allocation matrix for a hypo- thetical project. A description of the development of this risk-allocation matrix is given below. Risk factors (shown the first column in Table 6.1 and typically arranged in a matrix according to either their impact [rank] or chronology) are major events or conditions that can affect a proj- ect in a negative way (the events that can affect the project in a positive way are called âopportu- nities,â and traditionally there are far fewer opportunities than risks). Only significant risks should be considered because identifying and measuring all project risks would be a major effort. Under each project delivery method listed in the matrix, a main responsible party should be identified for each risk factor. For example, in the matrix shown in Table 6.1, the party responsible for design defects in a DBB contract is the owner, whereas in the DB contract, the responsible party is the constructor. Risk factors are rated, always from the perspective of the owner agency, according to the effect of a particular project delivery method on that risk factor. In the hypothetical case shown in Table 6.1, from the agency perspective, DBB is seen as having a favorable effect on the risk fac- tor of âpermits/approvals.â The agency thinks that it is the best party to obtain permits/approvals and that it can most effectively do this using a DBB approach. Therefore, the risk factor of Tier 3âOptimal Risk-Based Approach 89 Pr oje ct Lif ec yc le Pr oje ct De liv er y M et ho d (P DM ) S ele cti on Risk AnalysisPreliminaryRisk Analysis PDM Selection Verify Selected PDM Preliminary Engineering Final DesignConceptual Design Risk-Allocation Matrix Qualitative Evaluate Decision Document Results and Stop Successful? Document Results and Stop Quantitative Figure 6.3. Risk-based approach superimposed on project lifecycle. Table 6.1. Risk-allocation matrix for a hypothetical project. DBB DBRisk Factor Responsible Party Rating Responsible Party Rating Permits/Approval Owner + Constructor/Owner Different Site Conditions Owner 0 Constructor/Owner + Design Defects Owner Constructor + Quality Assurance/ Quality Control Constructor/Owner 0 Constructor + Exchange Rate Risk Owner Owner Other Risk Factors
permits/approvals has received a rating of +. The same risk factor, under a DB delivery method, is seen as unfavorable from the agencyâs point of view because the agency thinks that the DB con- structor is not the best party to obtain various permits and approvals (such as environmental permits). Therefore, a rating of â is assigned. Another risk factor in the hypothetical example, âdesign defects,â has a rating of â under the DBB arrangement because in this delivery method the agency is responsible for the accuracy of design. A DB approach, on the other hand, is rated + because it transfers this risk to the constructor. If the choice of a project delivery method has no effect on a particular risk factor, then a rating of 0 will be assigned. In rating each risk factor, one can refer to the contents of Chapter 3 of this guide, where the advantages and disadvantages of various project delivery methods in relation to 24 pertinent issues are documented. No attempt is made at this stage of the Tier 3 analysis to quantify the impact of these risk fac- tors (in terms of $ value or project delay). After the matrix is developed and the risk factors rated, the evaluation team can review the outcome and see if any project delivery method seems supe- rior in terms of its capacity in dealing with these risk factors. For example, a review of the matrix in Table 6.1 may suggest that DB is the better choice for the owner agency because of the num- ber of favorable ratings that it obtained. Preparation of the risk-allocation matrix and rating the risk factors can be accomplished in a reasonable amount of time. If the outcome suggests a âmost appropriateâ project delivery method, then the decision is finalized and the results, along with justification, are documented. If, after going through the process, the choice is still not clear, then the Tier 3 process should con- tinue on to the second phaseâthe quantitative analysis. Quantitative Analysis The quantitative approach should be attempted only if the qualitative approach does not result in a clear delivery method choice for a project. As shown in Figure 6.3, it is suggested that the Tier 3 quantitative analysis occur at the conclusion of the preliminary engineering phase, after the agency has conducted the FTA-mandated probabilistic risk analysis of project cost and sched- ule. The risk analysis is a major undertaking that requires hundreds of person-hours over the course of several weeks. Also, the outcome of the risk analysis can inform the project delivery method selection process (see Figure 6.4). The quantitative phase of Tier 3 would then be con- tingent on the availability of the complete risk analysis. If this risk analysis is not a requirement (for example in projects that do not apply for federal funding), then it is suggested that the proj- 90 A Guidebook for the Evaluation of Project Delivery Methods Risk Analysis Cost/Schedule - Ranked Risk Factors - Risk Factor 1 - Risk Factor 2 ⦠Quantitative Approach (See Fig. 6-5) Figure 6.4. Risk analysis outcome as an input to project delivery method selection.
ect delivery method selection decision be made without this phase as the cost of this phase could be prohibitive. The outcome of the probabilistic risk analysis required by the FTA consists of a distribution (range of possible values) for project cost and duration. Also, a list of the most important risk factors, ranked according to their impact on budget or schedule, is provided as part of the risk mitigation report. Usually, the number of these ranked risks is limited (e.g., in several risk assess- ments conducted by the project management oversight (PMO) consultants on behalf of the FTA, the list of significant risk factors included 10 to 15 risk factors). The FTA analysis follows the logic of Paretoâs law (also known as the 80-20 rule and the law of the vital few), which states that for many events, 80% of the effects come from 20% of the causes. In the context of project risks, rel- atively few risks are responsible for most of the project cost or schedule overruns. The project cost distribution and the list of ranked risks serve as inputs to the process of selecting the best project delivery method. For each ranked risk, a distribution of risk costs is usually estimated. The highest ranked risks are those with large expected values and large ranges (an indication of high variability in the risk factor). The proposed process, called the quantitative approach in this work, will involve estimating the effect of each major risk factor on the agencyâs budget, given a specific delivery method. The process starts by reviewing all the risk factors and selecting the risk factors whose value will be affected by the choice of project delivery method. Only the risk factors that are sensitive to the project deliv- ery method will be selected for further analysis. For each of these risk factors, the range of cost will be estimated under a given project delivery method. This estimation can best be accomplished by some of the same experts involved in the risk analysis. Figure 6.5 provides an example of a hypo- thetical project in which four major risk factors have been identified as the risk factors that are affected by the choice of project delivery method and the two remaining candidates for delivery method are DBB and DB. The risk factors are the following: permits, utility relocation, differing site conditions (DSC), and third-party issues. The cost of each risk is estimated using a triangular distri- bution, although many other distributions can be used depending on the nature of the risk factor.4 The sum of these risk costs will give the distribution for the total risk costs. There are statistical methods that can be used to calculate this sum with relative ease. Comparison of distributions of these total risk costs will give the owner agency a valuable tool for assessing the effect of project delivery method on project cost. A similar approach can be used to assess the effect of risks on project schedule. If the purpose of the risk analysis is to examine the effect of delivery method on project duration, all the distributions depicted in Figure 6.5 would have durations on the X-axis and the total effect will be the total impact on project schedule instead of on project cost. The quantitative analysis is a powerful tool for comparing competing project delivery methods. It focuses on those differences between project delivery methods that affect cost and schedule and provides a consistent way of evaluating each project delivery method vis-Ã -vis major risk factors affecting the project. This analysis allows the decision-maker to document the reasons for the selection of a specific project delivery method. The drawback of this analysis is its dependency on the availability of expensive risk analysis results and the higher skill level required for pricing out each risk under various project delivery methods. However, the choice of the project deliv- ery method is a natural outcome of a risk analysis exercise because one of the most important benefits of any risk analysis is risk allocation/mitigation. A properly selected project delivery method is an effective risk mitigation instrument that can help keep project costs low and min- imize project delays. Tier 3âOptimal Risk-Based Approach 91 4In a triangular distribution, the range of possible values is estimated with a lower bound (optimistic), an upper bound (pessimistic), and a most-likely value. The triangular distribution is commonly used in probabilistic risk analysis because of its simplicity.
Conclusion The Tier 3 approach may be needed in cases where the Tier 1 and Tier 2 approaches cannot provide a clear best choice for the project delivery method. In such a case, the Tier 3 provides a two-phased approach: first, a qualitative analysis and then, if necessary, a quantitative one. Both analyses are based on a risk-allocation exercise that will determine major risks to the agency under various project delivery methods. In the qualitative approach, the decision-makers care- fully examine each risk factor and deliberate the anticipated effect of each risk factor on project cost and schedule. This critical review can help the agency decide on the most appropriate deliv- ery method. If the qualitative analysis does not yield a final choice of delivery method, the agency can then proceed with the quantitative analysis. In this analysis, the cost and schedule effect of each risk factor is estimated within an appropriate range, summed up, and used in comparing the total effect of risks under competing delivery methods. The agency can then select the deliv- ery method that results in the most favorable outcome considering both cost and schedule. 92 A Guidebook for the Evaluation of Project Delivery Methods DBB DB $ Pr ob . Utility DSC Third Party Total Effect on Cost Permits $ $ $ $ $ $ $ $ $ Pr ob . Pr ob . Pr ob . Pr ob . Pr ob . Pr ob . Pr ob . Pr ob . Pr ob . Figure 6.5. Overview of the quantitative analysis.