Portfolio Risk Management
Most of the discussions of risk assessment and management in this report have been concerned with risks at the individual project level and have focused on the IPT and project director. However, DOE has numerous projects, and program managers and senior managers should be concerned with the management of risks at the overall enterprise level, or project portfolio management. While the portfolio is composed of projects rather than stocks and bonds, the analogy with stock portfolios is intentional. For the investor, high-yield, high-risk stocks and bonds can be balanced by low-yield, low-risk stocks and bonds to achieve the desired level of portfolio risk. The knowledgeable owner, whether of stocks and bonds or of active projects, balances the portfolio to achieve an acceptable overall risk level.
Portfolio risk management does not imply that an owner should not perform risky projects but rather that the knowledgeable owner is aware of an optimum overall level for risk and adjusts project risks accordingly. Project owners do not have the flexibility of investors to trade stocks and bonds on the market and therefore cannot manage project portfolios in the same way that investment portfolios are managed, by buying and selling them. Some of the projects that an organization undertakes may not be freely chosen but rather determined by external forces. For example, an industrial owner may be required by environmental regulations to install scrubbers to reduce air pollution or to clean up contaminated ground water; these are not the owner’s choices and cannot be avoided.
However, how these projects are performed may be controlled by the owner. The fact that some projects may not be completely controllable only reinforces the point that the owner should understand the risks. Building a new plant, developing a new product, and cleaning up a contaminated site all affect the total risk in the owner’s project portfolio.
The primary difference between investment portfolios and project portfolios is that an investor has historical information about the volatility of the stocks (for example, the beta factors computed for stock prices) and, importantly, the correlation between them, with which to make informed decisions. However, there is no available database of volatility factors and correlation coefficients for first-time or one-of-a-kind projects. Therefore other means must be used to assess project risks. A knowledgeable owner who performs a large number of projects can make valid statistical judgments if a database of past projects is maintained and a consistent methodology for assessing risks is implemented across all projects.
PROGRAM-LEVEL RISK ANALYSIS
A knowledgeable owner maintains a program-level risk analysis of all ongoing significant projects in order to monitor the risks and vulnerabilities of project portfolios with respect to schedule, cost, scope, and performance and to control the total organizational risk. A fundamental maxim of modern management is: “If you don’t know it, you can’t measure it; if you can’t measure it, you can’t control it.” Therefore, controlling risk has to start with knowing the risks—that is, measuring them or estimating them by the methods discussed in this report.
The program-level risk analysis should assess the risk status of current projects based on each project’s original baselines and current project schedules and budgets, compared to performance on completed projects. The assessment should evaluate the risks of future scope shortfalls and budget and schedule overruns and should identify ongoing deficiencies in risk management that undermine the owner’s ability to avoid surprises. Managing projects through risk assessment and risk management means looking forward, to anticipate future risks, instead of looking backward at past mistakes. The knowledgeable owner applies this process to all active projects in the portfolio, to minimize surprises.
An inadequate contingency at the program level may lead to project scope reductions, schedule delays, and even terminations. But at the program level, cost underruns could be attributable to an excessive contingency, which ties up the owner’s capital or borrowing capacity unnecessarily in management reserves that are not needed and might have been used to support other desirable projects (Mak and Picken, 2000). Over-estimation of contingencies leads to opportunity costs (i.e., some valuable
projects at the margin will not be undertaken) as well as to excessive project costs if contingencies are routinely used up no matter what happens. The knowledgeable owner uses the best available method to set appropriate contingencies consistently across the project portfolio that are neither too large nor too small for the risks involved.
The knowledgeable owner also undertakes the risk assessments necessary to avoid baseline breaches by predicting the actual cost and time at completion of all ongoing projects. Projects that are the most vulnerable to risks need the most management attention. Consistency is necessary for programwide, cross-project comparisons, so the knowledgeable owner needs consistent procedures for assessing risks across all projects, and these procedures need to be insulated from project biases.
To get started with program-level risk management, an owner needs to have a current risk assessment of all ongoing projects in the portfolio and to establish, on a consistent basis, the vulnerabilities of projects with respect to schedule, cost, and performance risks. This assessment then becomes the baseline for program risk management, and should be updated as:
Projects are completed and removed from the active project portfolio.
Projects make progress and the estimated costs to complete, times to complete, and risks are reevaluated periodically (see the discussion on learning from project progress in Chapter 4).
New projects are authorized, or proposed for authorization, to be added to the active project portfolio.
For proposed new projects, portfolio risk assessment should be used to determine whether the authorization of a particular project would raise the overall portfolio risk to a level unacceptable to enterprise management. If it would, then program managers may wish to terminate the proposed project, modify it, postpone it (until, for example, some active high-risk projects have been completed), or accept the risk of undertaking it. Program management concerns not only doing projects right, but also doing the right projects, and a project that appears to raise the portfolio risk to an unacceptable level may require restructuring before it can proceed.
A very important factor in portfolio risk assessment, as mentioned earlier, is the determination of correlations between projects. In investment portfolio management, for example, the investor needs to understand the correlations between stocks, as the whole point of portfolio management is to ensure that the assets in the portfolio are independent of each other so they do not all lose value at once. Similarly, the program manager needs to know if projects are correlated such that, if one project
goes over budget, others might be more likely to go over budget. Correlations may be due to elementary factors such as material cost: If the price of steel goes up, then the costs of all projects that use steel will go up. Or the dependency between projects may be more subtle; for example, if multiple plants or processes are built to the same design, any design flaw would be likely to occur in all of them, potentially affecting the performance of the entire enterprise. The knowledgeable owner may not be able to avoid such dependencies but must certainly know what they are. Owners also need to understand dependencies among projects and their combined effect on the success of the enterprise. For example, if one project is to build a chemical processing plant and another is to build a waste treatment facility to support the former, then the risks for these projects should be managed as one.
Knowledgeable owners set project contingencies to meet risk management criteria at all levels of the enterprise. Portfolio contingency is difficult to control if the project contingencies are not explicitly stated but rather are buried in the project estimates. Consequently, periodic project reviews (known as scrubbing the estimates) become necessary as a means to uncover and consolidate the buried contingencies.
It is possible to set contingencies to meet defined risks on a consistent basis across all levels of an enterprise, from work packages at the lowest level up through the work breakdown structure to the total project level and then to the program or enterprise level. When this is done on a consistent basis, the budgets at all levels can be set in accordance with acceptable risks of overrunning at these levels, which need not be the same for all levels and all projects. For example, program management might accept a relatively high risk of overrunning at the detailed work package level, less risk of overrunning the project budget, and still less risk of overrunning the program budget at the enterprise level. However, the budget risks cannot be controlled unless the contingencies are explicitly set to match the risks. Knowledgeable owners should have a consistent and explicit policy on the use of contingencies, what level of risk they should reflect, and, of particular importance, who controls them.
PROJECT AND PORTFOLIO BUDGETS
Project portfolio management needs to address the following question: Why do relatively few projects seem to underrun budgets, and why do so few return the unused contingency? There are at least two possible explanations for this: (1) underestimated costs and (2) project budget entitlement.
One hypothesis is that costs are in fact consistently underestimated, such that the actual project budgets are less than the expected values. Under what circumstances might project budgets be generally biased on the low side (less than the expected values)? Some possible explanations include the following:
Projects are intentionally underestimated and pushed through the process by their advocates, who recognize that the likelihood of getting funded decreases with increasing project cost estimates. Project proponents also recognize that if a project is underfunded, the funding may not be enough to complete the project, but will be enough to get it started. They expect to go back to the sponsor to authorize a budget increase once the project is under way, and expect that the sponsor will not terminate it with so much money sunk into it. Project proponents are motivated to lowball the cost estimates and discouraged from providing unbiased estimation. Even a small lowball bias at the work package level leads to a virtual certainty that budgets will be overrun at the project level.
Project estimates are in fact originally accurate at the project level but are arbitrarily reduced at a higher (political) level, in the belief that they are too large or contain too much fat. Or, trying to do more projects than the funds available can support, higher-level managers simply divide their fixed resources among their projects regardless of project estimates. This behavior is also self-reinforcing.
Project Budget Entitlement
Another possible explanation for the apparent fact that more projects seem to overrun than underrun is an asymmetry in how funds are handled. It is typically assumed that cost overruns in some projects are (statistically) offset by underruns in others, and that reserve margins can therefore be proportionally lower when spread over many projects. To take a different view, suppose that every project that overruns its budget and appeals for additional funding receives it, while projects that underrun budgets hold onto all or part of the contingency and use it to enhance the project instead of passing it back to the program.
This result may be rationalized by a sense of entitlement. Project personnel may feel that their work is highly justified—e.g., essential to the national defense, vital to the advancement of science, necessary to cleaning up the environment—and that if, through
their efforts, they have some money left over they deserve it. They can also easily justify spending it on increasing the project scope and quality, improving performance and reliability, getting more and better instruments, upgrading the office space, and the like. Any or all of these options may be much more attractive to the project personnel than giving the money back to the program, especially if they feel they are entitled to it by the value of their work or their suffering through past budget cuts. In fact, many people believe that all contingency funds belong to the project and ought to be spent by the project, whether or not the events on which they were contingent ever occurred.
No underruns may in fact be observed. This happens because more management attention and efforts are typically directed to projects (or work packages) that are underperforming and overrunning than to those that are outperforming estimates. In typical project monthly reports, projects are classified as red (underperforming and overrunning), yellow (trending toward underperforming and overrunning), or green (outperforming estimates), based on cost and schedule performance, and attention immediately turns to those classified as red, not to those classified as green. Because badly performing projects get more upper management attention than problem-free projects, it is not unusual that the good performers get worse in the absence of careful supervision. In addition, a common solution to the problems on one project is to transfer the project manager from another project that is within budget and on schedule, leaving that project under less competent management. Underruns may thus disappear naturally, and no one can say if they ever existed, much less where they went.
These effects would be very hard to observe in the cost records. In the second case, no one ever observes an underrun, even the personnel on the projects. In the first case, underruns might become known if project personnel admit that they could have come in under budget but spent the full budget anyway because it was there. But this admission is unlikely to occur. Thus an outside observer can never observe the probability distribution of costs as they might have been; one can only observe the actual costs after they have been reported and therefore after any potential underruns have been spent.
The costs of one project may influence the budgets for following projects. If contingencies are expended, the whole cost structure will inexorably creep upward. Thus, as overruns are reported accurately but underruns are spent, costs will get higher and higher, even for programs with cost databases from prior experience. This project-to-project cost
creep will doubtless be attributed to construction cost inflation or other uncontrollable factors.
These considerations lead to the conclusion that, not only should contingencies be set objectively based on probability considerations, but also control of these contingent funds should be retained at the enterprise level. Management reserves should be controlled high up the management chain, in order to take advantage of the benefits of larger numbers of projects, and they should be controlled by people who are not proponents of any particular projects to ensure that the reserves are allocated based on actual needs and priorities, not personal bias.
The nature of management reserves and contingencies—how large they are and why—should be made open, rational, and explicit rather than hidden or implicit, at all organizational levels. Rules for the rational setting of management reserves should be published in organizational policies and procedures, along with statistical justifications. Responsible managers should actively manage the reserves. Efforts should be made to reduce costs by controlling the release of contingency funds to projects or activities, by rewarding managers who come in under budget, by sharing any remaining contingency funds between the project and the program, and by giving management attention to prospective risks, whether projects are under budget or over budget. Bringing in a project $1 million under budget should have just the same value and recognition as preventing one from going $1 million over budget.
Some sense of common purpose is required to reinforce cooperation and minimize competition so that project estimates are not manipulated up or down, and surpluses are returned to the higher management level. The likelihood of defensive irrational decisions can be reduced by meeting budget cuts or shortfalls by delaying or canceling the lowest-priority projects and fully funding the rest rather than underfunding, and thus delaying, them all.