Guidebook on Risk Analysis Tools and Management Practices to Control Transportation Project Costs (2010)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

44 5.1 Introduction This chapter presents guidance for risk management in the planning phase. The Planning Phase is defined in terms of its relevance to cost estimating and risk management. Each of the risk management steps is discussed in detail, including in- puts, outputs, and tools used in the risk management process. Tips for tool application and how project complexity impacts risk management tools and practices are also discussed for each step. 5.1.1 Planning Phase Overview The purpose of transportation planning for both states and regional areas is to identify a set of the most cost effective ap- proaches that achieve their stated system goals. Federal law requires that SHAs develop a statewide transportation plan and that MPOs develop a regional transportation plan (RTP). The horizon year for these long-range plans is usually 20 to 25 years into the future. Approaches, or at least terminology, for statewide trans- portation planning vary across the country. While some states do identify major or even unique minor projects, most state- wide transportation plans do not identify specific projects, but rather establish strategic directions for state investment in the transportation system and present future challenges that could constrain the ability of the SHAs to improve the performance of their systems. The metropolitan RTP is very different. The RTP identifies specific projects that are to be implemented over the next 25 years. Federal law requires the RTP to be fiscally con- strained, that is, the sum of the total project costs in the plan cannot exceed the amount of funding that is expected over the next 25 years. This places great importance on having valid and realistic cost estimates for the projects in the MPO’s plan. The term “conceptual estimating” is often used to describe the general method of estimating project costs during the planning phase. Planning level cost estimates can have a sig- nificant effect on the overall transportation program and on the ability of the SHA and MTO to meet their area trans- portation needs. The level of effort regarding cost estimating and risk management is limited due to a lack of detailed proj- ect definition and the type of estimate techniques available to prepare a planning phase cost estimate (e.g., lane mile, etc.). The expectations for accuracy must be realistic in an estimate for a project that may be 25 years in the future and is only defined by project limits (e.g., point A to point B). Proper application of cost estimating and risk management tools will result in the generation of credible ranges of costs and risk management plans to meet these cost estimates. 5.1.2 Planning Phase Risk Management Emphasis The risk management process begins during the planning phase. As shown by the shading in Figure 5.1, risk identifica- tion is the primary focus of the risk management process dur- ing the planning phase. Detailed risk identification is useful in the preparation of planning documents, particularly in the RTP. Risk assessment and risk mitigation and planning can be conducted on the most complex projects when they are iden- tified in the RTP. However, risk assessment and mitigation are limited in the planning phase due to a lack of project defini- tion. Similarly, detailed risk management plans can be devel- oped for identified major projects and groups of minor or moderately complex projects, but they are limited as tools to assist in cost control (again due to the lack of individual proj- ect definition). Although risk allocation and risk monitoring and control are not the focuses of the planning phase, they should be considered when major projects are identified. The remainder of this chapter describes tools and manage- ment practices suggested for use during the planning phase of project development. A discussion of how project complexity impacts risk management tools and practices is also provided. C H A P T E R 5 Guide to the Planning Phase

45 5.2 Planning Phase Risk Identification As stated in Chapter 3, risk identification is paramount to successful risk management and contingency estimation at the planning phase. The objectives of risk identification are: 1) to identify and categorize risks that could affect the project and 2) to document these risks. The outcome of the risk iden- tification is a list of risks. Ideally, the list of risks should be comprehensive and non-overlapping. This list of risks can be the basis for estimating project contingency and setting the baseline cost estimate. Please refer to Chapter 3, Section 3.4.1 regarding risk identification, specifically regarding: • Why risk identification should stop short of risk assessment; • The appropriate level of detail for risk identification; and • When and how to apply risk checklists. 5.2.1 Planning Phase Risk Identification Inputs The planning phase defines overarching transportation needs, groups of projects, and/or individual unique proj- ects. The determination of project risk stems from a review of the planning assumptions made by the planning team and the estimating assumptions made by the project estima- tor. The planning team will necessarily make broad assump- tions of transportation needs to define the groups or projects. Likewise, the estimator will need to make estimating assump- tions at a very high level. These assumptions often will be in the form of a cost per lane mile on projects very early in the planning process. Projects that are identified later in the plan- ning phase, but before they enter programming, may have more detail (e.g., estimate of structure size, right of way pur- chase required, etc.), but these projects generally have to be estimated in a very conceptual manner. Planning and esti- mating assumptions serve as triggers for risk identification. The risk identification process should begin with a review of any risks identified during previous planning phase analy- ses or estimates as projects can remain in the planning phase for many years. When no previous analyses or estimates are available, the risk identification projects can begin with a re- view of the current planning report and cost estimate with a focus on the assumptions that were used to generate these reports and estimates. It should be noted that planning phase project scopes can vary greatly and the risk analysis and assessment varies with the scope accordingly. Those projects that are identified early in the planning phase have minimal scope definition and pro- vide challenges for conducting detailed risk analyses. Projects with clearly identified scopes near the end of the planning phase can benefit more from rigorous risk analysis and the applica- tion of defined contingency estimates. 5.2.2 Planning Phase Risk Identification Tools Risk identification tools that can be used in the planning phase are listed in Table 5.1. Note that complex projects can use all risk identification tools. Refer to the Appendix A for complete tool descriptions. 5.2.3 Planning Phase Risk Identification Outputs The key risk identification output is a list of risks. In its sim- plest form, the list of risks may take the form of a Red Flag Item list (I2.1). On some moderately complex and most major projects near the end of the planning phase, the list of risks Risk Management Process Allocate Monitor and Control Identify Assess/ Analyze Mitigate and Plan Figure 5.1. Risk management focus in the planning phase. Table 5.1. Planning phase risk identification tools. Tool M in or (N /A ) M od er at el y C om pl ex M ajo r I2.1 Red Flag Items I2.3 Risk Checklists I2.4 Assumption Analysis I2.5 Expert Interviews I2.6 Crawford Slip Method I2.7 SWOT Analysis R3.1 Risk Management Plan R3.6 Risk Workshops R3.11 Risk Breakdown Structure R3.12 Risk Register

46 should form the basis of a Risk Management Plan (R3.1) and Risk Register (R3.12) for later risk management and control. Categorization of the risk list can be extremely helpful to ensure that no risks have been missed. Categorization of risks can be accomplished through a review of Risk Checklists (I2.3). On major projects near the end of the planning phase, categorization can be achieved through the application of Risk Breakdown Structure (R3.11). Ideally, the list of risks should be comprehensive and nonoverlapping. This list of risks can be the basis for estimat- ing project contingency and setting the baseline cost estimate. Comprehensive and nonoverlapping lists of risks are required for detailed risk and contingency modeling in the later steps (see Chapter 3 for more details). 5.2.4 Planning Phase Risk Identification Relationship to Project Complexity Minor projects are typically not identified as individual projects until the programming phase or later. During the planning phase, minor projects are grouped into what can be classified as moderately complex or major projects. It is gen- erally not until the programming phase that these are iden- tified as individual projects. Please see Chapter 6 for details regarding risk identification on minor projects. Risk identification on moderately complex projects will benefit from a large number of sources. Expert Interviews (I2.5) will be a key input and formal Assumptions Analysis (I2.4) is typically warranted. A Risk Register (R3.12) should always be employed and a Risk Management Plan (R3.1) can be warranted on moderately complex projects with a high level of uncertainty. Major projects require the highest level of risk identification. All risk identification tools are applicable to major projects. The Risk Workshop (R3.6) is the principal tool employed on major projects that is not typically applied to moderately com- plex projects. Formal Risk Workshops (R3.6) are typically fa- cilitated by a professional risk analyst and can have upwards of 20 experts participating, depending upon the specific project needs. The time to plan, conduct, and document the workshop should not be underestimated. The benefits of a workshop include a comprehensive list of risks and a firm basis for risk analysis and planning. 5.2.5 Planning Phase Risk Identification Tips The use of appropriate risk identification techniques should be intense, but commensurate to project understanding, dur- ing planning. • Risk identification should be a creative brainstorming process. It should not attempt to analyze risks or discuss mitigation procedures, which will be completed in the sub- sequent steps. • Use risk checklists and experience from similar projects only to check for missing risks and to help categorize unique project risks. • Upon completion of the risk list, categorize the risk into logical groupings. Use risk checklists and similar project risk analyses for possible categorizations. 5.3 Planning Phase Risk Assessment and Analysis The primary objective of risk assessment is the systematic consideration of risk events focusing on their probability of occurrence and the consequences of such occurrences. The majority of risk management applications during planning will include only a qualitative risk assessment (or even stop at risk identification without any risk assessment due to the lack of scope definition in planning). Rigorous risk analysis is gen- erally reserved for only those major projects that are identi- fied in the Planning Phase. Recalling from Chapter 3 (Section 3.4.2), risk assessment and analysis is the process of evaluating the risk events documented in the preceding identification step. Risk assessment and analy- sis has two aspects. The first determines the probability of a risk occurring (risk frequency); risks are classified along a contin- uum from very unlikely to very likely. The second aspect judges the impact of the risk should it occur (consequence severity). Typically, a project’s qualitative risk assessment will recog- nize some risks whose occurrence is so likely or whose conse- quences are so serious that further quantitative risk analysis is warranted. A key purpose of quantitative risk analysis is to combine the effects of the various identified and assessed risk events into an overall project risk analysis. The overall risk analysis is used to determine cost and schedule contingency values and to quantify individual impacts of high risk events. Ultimately however, the purpose of quantitative analysis is to not only compute numerical risk values but to provide a basis for controlling project costs through effective risk manage- ment practices. 5.3.1 Planning Phase Risk Assessment and Analysis Inputs The key inputs for risk assessment and analysis are the probability of a risk occurring (risk frequency) and the impact of the risk should it occur (consequence severity). The physi- cal input to risk assessment and analysis is the list of risks from the identification step. In the Planning Phase, this will likely be a high-level list of risks. Determining the risks’ probability and impact allows for qualitatively ranking to help planners

focus on the most critical risks or quantitatively modeled to determine cost and schedule contingency estimates. The risk identification step will have identified risks, but the identification step should stop short of assessment or analysis. In the risk assessment and analysis step, risk inputs will be elicited from planners, functional experts, estimators, and analysts to gain a clear picture of the risks. 5.3.2 Planning Phase Risk Assessment and Analysis Tools Risk assessment and analysis tools that can be used in the Planning Phase are listed in Table 5.2. Note that minor proj- ects are not included in the Planning Phase as explained in Section 5.2.4. Refer to the Appendix A for complete tool descriptions. 5.3.3 Planning Phase Risk Assessment and Analysis Outputs Planning Phase risk assessment and analysis outputs will depend on the type of decision required and the rigor in the selected tools for assessment or analysis. The use of qualita- tive expert input (I2.5 Expert Interviews) and the application of a Risk Priority Ranking (R3.7), Probability x Impact Ma- trix (P x I) (R3.8), or Risk Comparison Table (R3.9) tools will yield a ranked set of risks that can help focus management on managing the highest priority risks. The use of quantitative expert inputs and application of an Estimate Ranges – Monte Carlo Analysis (R3.5) tool will yield a definitive contingency estimate and detailed sensitivity analysis of the risks that con- tribute to the contingency. Recalling from Chapter 3 (Section 3.4.3.4), the type of out- puts that a tool produces is a function of the analytical rigor, na- ture of assumptions, or amount of input data. Results from risk analyses may be divided into three groups according to their primary output: 1) single parameter output measures; 2) mul- tiple parameter output measures; and 3) complete distribution output measures. Please refer to Sections 3.4.3 and 3.4.5 for a full description of risk assessment and analysis outputs. 5.3.4 Planning Phase Risk Assessment and Analysis Relationship to Project Complexity Project complexity is a key indicator of which risk assessment and analysis tools to apply. Moderately complex projects will generally apply a qualitative risk assessment to produce a rank- ing of risks. Risk ranking is commonly accomplished through the use of Risk Priority Rankings (R3.7), P x I Matrixes (R3.8), Risk Comparison Tables (R3.9) and/or Risk Maps (R3.10). Early planning estimates are often described by ranges. An Estimate Range – Three Point Estimates (R3.4) can be used to produce the range on moderately complex projects. On those moderately complex projects that are identified during plan- ning that need a separate contingency estimate, Contingency— Percentages (R3.2) or Contingency—Identified (R3.3) tools can be applied. Major projects that are identified during the Planning Phase can use more rigorous risk ranking and contingency estima- tion tools. The risk management process often begins with Risk Analysis Workshops (R3.6) and generates a stochastic estimate of cost and schedule through an Estimate Range– Monte Carlo Analysis (R3.5). The resulting risk-based range estimate is then updated throughout project development. Please see Chapter 6, Section 6.3.4 for a more detailed de- scription of how risk analysis and contingency estimation are related to project complexity at the programming phase. 5.3.5 Planning Phase Risk Assessment and Analysis Tips A clear understanding of risk significance is helpful in the planning phase. On major identified projects near the end of the planning phase, a rigorous estimate of contingency can also be helpful. • The goal of risk assessment is not to eliminate all risk from the project. Rather, the goal is to recognize the significant Table 5.2. Planning phase risk assessment and analysis tools. Tool M in or (N /A ) M od er at el y C om pl ex M ajo r I2.5 Expert Interviews R3.2 Contingency–Percentage R3.3 Contingency–Identified R3.4 Estimate Ranges–Three Point Estimate R3.5 Estimate Ranges–Monte Carlo Analysis R3.6 Risk Workshops R3.7 Risk Priority Ranking R3.8 Probability x Impact Matrix (P x I) R3.9 Risk Comparison Table R3.10 Risk Map R3.13 Risk Management Information System 47

48 risk challenges to the project and to initiate an appropriate management response. • A comparison of each risk’s probability and impact yields a relative ranking of the risks that can be used for risk man- agement or, if warranted by project complexity, a detailed quantitative risk analysis using probabilistic models to generate ranges of possible outcomes. 5.4 Planning Phase Risk Mitigation and Planning The objectives of risk mitigation and planning are to ex- plore risk response strategies for the high-risk items identi- fied in the qualitative risk assessment or quantitative risk analysis. The process identifies and assigns parties to take re- sponsibility for each risk response. It ensures that each risk re- quiring a response has an owner. The key output is the risk register or risk management plan. As shown in Figure 5.1, risk mitigation and planning in the planning phase requires less effort than in later project devel- opment phases. This lower level of effort stems primarily from the fact that projects, project teams, and project man- agement structures are generally not in place until the pro- gramming phase. Detailed risk mitigation and planning efforts are reserved only for those identified moderately com- plex and major projects in the planning phase. 5.4.1 Planning Phase Risk Mitigation and Planning Inputs The ranked list of risks from the first two risk management steps is the key input to the mitigation and planning effort. Risks mitigation and planning begins by developing and doc- umenting a risk management strategy focused on the key risks. Early efforts establish the purpose and objective; assign responsibilities for specific areas; identify additional techni- cal expertise needed; describe the assessment process and areas to consider; delineate procedures for consideration of mitigation and allocation options; dictate the reporting and documentation needs; and establish report requirements and monitoring metrics. This planning should address evaluation of the capabilities of potential sources as well as early indus- try involvement. The list of risks is used as the basis for the solicitation of mitigation and planning options from key managers and ex- perts. The mitigation and planning options will require cost- benefit analyses (e.g., the cost of implementing a mitigation effort versus the reduction in probability or impact to a risk) to assess the viability or impact of the options. Estimators and risk analysts will therefore have key input into the mitigation and planning process. The planning team will have the final input into the risk mitigation and planning efforts. They will determine who “owns” the risk and is responsible for ensuring that it is man- aged effectively. The risk management plan and/or risk regis- ter should clearly identify who is responsible for managing and resolving each individual risk. 5.4.2 Planning Phase Risk Mitigation and Planning Tools Risk mitigation and planning tools that most generally apply the planning phase are listed in Table 5.3. Refer to the Appendix A for complete tool descriptions. 5.4.3 Planning Phase Risk Mitigation and Planning Outputs The outputs of the risk mitigation and planning step are an organized and interactive risk management strategy and a plan for adequate resources. The Risk Management Plan (R3.1) tool in Appendix A provides a good example of this output from Caltrans. The example risk management plan from Caltrans describes a clear approach to the assignment of responsibility. It also provides items that require resource in- vestment and a method for calculating their costs. Risk mitigation and planning is iterative and includes de- scribing and scheduling the activities and processes to assess (identify and analyze), mitigate, monitor, and document the risk associated with a project. For minor or moderately com- plex projects, the result should be a Risk Register (R3.12). For major projects or moderately complex projects with a high degree of uncertainty, the result should be a formal Risk Management Plan (R3.1). 5.4.4 Planning Phase Risk Mitigation and Planning Relationship to Project Complexity The risk mitigation and planning effort should be congru- ent with project complexity. Each risk plan should be docu- mented, but the degree of documentation will vary with project Table 5.3. Planning phase risk mitigation and planning tools. Tool M in or (N /A ) M od er at el y C om pl ex M ajo r R3.1 Risk Management Plan R3.12 Risk Register

complexity. Not all projects will require a comprehensive and quantitative risk management process. The creation of a Risk Register (R3.12) is a formal identification of risks. A risk reg- ister can be applied to all projects that are identified during the planning phase. The risk register provides planners with a listing of significant risks and includes a variety of informa- tion for each. The risk register creates a concise tool to man- age the risks and the individuals assigned to each risk. Major projects identified in the planning phase will bene- fit from having a detailed and formal Risk Management Plan (R3.1) that records all aspects of risk identification, risk as- sessment, risk analysis, risk planning, risk allocation, and risk information systems, documentation, and reports. Major projects should use a register that is integrated into the for- mal Risk Management Plan (R3.1). 5.4.5 Planning Phase Risk Mitigation and Planning Tips Risk mitigation and planning will apply to those projects that are specifically identified during the planning phase. • Document the risk mitigation and planning efforts at a level of detail that is appropriate for the level of project def- inition, project complexity and resources available for Planning Phase efforts. • Rigorous documentation of risk mitigation and planning in the Planning Phase will assist in scoping during the Pro- gramming Phase. 5.5 Planning Phase Risk Allocation The goal of risk allocation is accomplished by contractu- ally assigning risks to the party that is best able to manage them. Risk allocation efforts do not typically begin until the programming phase and are most intensive during the design phase. However, some major projects that are identified in the planning phase will identify, or explore, project delivery and contracting options. For a detailed discussion of these projects, please refer to the risk allocation discussion in the programming phase, Section 6.5. 5.5.1 Planning Phase Risk Allocation Inputs Risk allocation inputs at the planning phase include the outputs from the first three steps in the risk management process plus an examination of the available/allowable proj- ect delivery options for the project. 5.5.2 Planning Phase Risk Allocation Tools As seen in Table 5.4, the only tool that applies in the plan- ning phase is the Delivery Decision Support (D1.2) tool, which includes descriptions of numerous alternative project delivery methods. 5.5.3 Planning Phase Risk Allocation Outputs The likely output of the risk allocation step at the planning phase will be an analysis of possible project delivery methods. However, the output will vary with the nature of the project. At the highest level of detail, a project delivery decision report can be generated that analyzes risk allocation in terms of proj- ect characteristics (e.g., cost, schedule and quality issues), agency characteristics (e.g., available staffing level, staff expe- rience, etc.), market characteristics (e.g., market availability, market experience, etc.) and stakeholder characteristics. The final project delivery decision will not likely be made until the programming phase. 5.5.4 Planning Phase Risk Allocation Relationship to Project Complexity Alternative delivery methods will only be explored on major projects during the planning phase. 5.5.5 Planning Phase Risk Allocation Tips Risk allocation discussions can assist in appropriate proj- ect delivery method selection. • Document risk allocation discussions in the planning phase to assist in making final project delivery and contract pack- aging decisions in the later phases. 5.6 Planning Phase Risk Monitoring and Control The objectives of risk monitoring and control are to: sys- tematically track the identified risks; identify any new risks; effectively manage the contingency reserve; and capture lessons learned for future risk assessment and allocation efforts. The application of risk monitoring and control will vary greatly with the level of project definition during the PLANNING PHASE. Risk monitoring and control effort Tool M in or (N /A ) M od er at el y C om pl ex M ajo r D1.2 Delivery Decision Support Table 5.4. Planning phase risk allocation tools. 49

50 are more intense during the programming and design phases of project development. 5.6.1 Risk Monitoring and Control Inputs The key inputs to risk monitoring and control are the Risk Management Plan (R3.1) and Risk Register (R3.12). These tools provide a framework for managing risks through a for- malized monitoring and control process. In some cases when fewer planning resources are available, a list of Red Flag Items (I2.1) can also be used for risk monitoring and control. 5.6.2 Planning Phase Risk Monitoring and Control Tools Table 5.5 lists the risk monitoring and control tools sug- gested for the Planning Phase. More information on the tools can be found in Appendix A. 5.6.3 Planning Phase Risk Monitoring and Control Outputs The key output for risk monitoring and control in the planning phase is a strategy or plan that will support moni- toring and control in the programming and design phases. Please see Sections 6.6.4 and 7.6.4 for a more detailed discus- sion of monitoring and control in those phases. 5.6.4 Planning Phase Risk Monitoring and Control Relationship to Project Complexity A Red Flag Item (I2.1) list should only be applied for risk monitoring and control on those projects where planning re- sources are constrained. Risk Registers (R3.12) and Risk Man- agement Plans (R3.1) are more appropriate tools to assist in monitoring and control in major projects with a large amount of uncertainty. 5.6.5 Planning Phase Risk Monitoring and Control Tips Risk monitoring and control should be a continuous and repetitive activity during all phases of project development. • For each project, develop a unique risk monitoring and con- trol process that is appropriate for the size and complexity of the project. However, do not create monitoring and control processes that are burdensome or create undue paperwork. 5.7 Conclusions This chapter discussed guidance for risk management in the planning phase. The outcome of this discussion is a map or guide to the risk management tools provided in Appendix A. The planning phase focuses efforts on risk identification. Moderately complex and major projects that are specifically identified in the planning phase can employ some level of risk assessment and analysis efforts – particularly near the end of the planning phase. Risk mitigation and planning, risk alloca- tion, and risk monitoring and control are typically found only on identified major projects. Rigorous documentation of the risk management process in the planning phase will make the programming and design phase efforts more effective. Table 5.5. Planning phase risk monitoring and control tools. Tool M in or (N /A ) M od er at el y C om pl ex M ajo r I2.1 Red Flag Items R3.1 Risk Management Plan R3.12 Risk Register