Click for next page ( 41


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 40
7 Readiness to Proceed for First-of-a-Kind Projects INTRODUCTION A significant part of the DOE budget is spent on first-of-a-kind and one-of- a-kind environmental restoration, nuclear defense, and scientific research and development projects. By definition, there is no guidance for these projects based on previous history and project experience; as a result, first-of-a-kind projects have shown themselves to be susceptible to large cost and schedule overruns and performance shortfalls. This problem is aggravated by the tendency to measure the degree of preparation (readiness to proceed) for these projects by the same standards and metrics used for projects that are similar to others that have already been executed. First-of-a-kind projects have greater uncertainty simply because they are first. Often the components of first-of-a-kind projects are new and require tech- nical development, scale-up, or even research and development. Even if the individual components are commercial off-the-shelf, their integration into a new, high-performance system may be unprecedented. Senior management, faced with the alternatives whether to wait until all technical uncertainties are resolved or to move forward concurrently with project design and the development of new technologies will often opt for the faster approach. A first-of-a-kind project may have to proceed even with the associated risks, but these should be calcu- lated risks. The view of the committee is not that first-of-a-kind projects should not be undertaken, but rather that management should make an informed decision based on an unbiased understanding of the risks involved and should take active measures to reduce, mitigate, and manage these risks. Unlike conventional 40

OCR for page 40
READINESS TO PROCEED FOR FIRST-OF-A-KIND PROJECTS 41 projects, which generally proceed sequentially in time, with no backtracking, first-of-a-kind projects need to consider and plan explicitly for rework, recycling, and iteration. This chapter addresses some of the issues peculiar to first-of-a-kind projects, their specific planning requirements, and the issues that need to be considered when determining whether a first-of-a-kind project is ready to proceed. CHARACTERISTICS OF FIRST-OF-A-KIND PROJECTS First-of-a-kind projects are, by definition, highly diverse, but regardless of their purpose construction of a new weapons processing plant, decontamination and remediation of a site with unique or unusual conditions, or construction of a new scientific instrument or laboratory that could achieve scientific and techno- logical leadership these endeavors share a number of features. One of the significant issues in environmental restoration projects, among them decontami- nation and remediation, is that conditions at many sites are unknown, owing to inadequate or incomplete characterization of the wastes. Accordingly, many of these projects are one-of-a-kind because there is no site with similar conditions. Uniqueness First-of-a-kind projects are unique. Because they cater to specific needs, no direct historical comparisons are possible and no project managers with previous experience are available. High Degree of Uncertainty First-of-a-kind projects may involve development (design, construction, and operation) of new, complex, and untested structures, systems, and equipment, or substantial scale-up of laboratory or pilot processes. These difficulties are com- pounded by the desire to accomplish the project as soon as possible, so construc- tion is started and many critical issues remain to be resolved once the project is under way. For some high-technology projects, scientific knowledge is developed and tested directly on the capital acquisition project and many of the critical systems are designed during construction. This concurrency of technology devel- opment and project engineering, design, and construction I high degree of uncertainty. High Cost ncreases an already First-of-a-kind project costs generally increase geometrically with the num- ber of technical groups or specialties that must be successfully integrated. Tech- nical and other uncertainties are more difficult to manage and are more visible on

OCR for page 40
42 PROGRESS IN IMPROVING PROJECT MANAGEMENT AT THE DOE multimillion to several billion dollar ventures because of the larger number of scientific and technical specialists. High Visibility and Public Attention New, large projects generally attract attention from the general public and from special interest groups, and first-of-a-kind projects are no exception. This is especially true for DOE environmental restoration projects, which may involve state and local governments, public and private environmental protection groups, and other stakeholders. Information technology enables all stakeholders and concerned parties to keep a close eye on the project and to react if some of their interests are affected. The democratic process promotes a high degree of visibility for government-sponsored projects, which results in a large number of external stakeholders monitoring projects and promoting or resisting project changes. High Impact The success or failure of major first-of-a-kind projects can have a substantial influence on the economic, social, environmental, and developmental atmosphere in the vicinity of the project. Successful completion of the project could bring numerous local benefits, such as job creation and improvement of the local infra- structure. The attractiveness of the area as a place for investment could be enhanced. Consequently, such projects often have political components that affect decisions and make these projects hard to stop or modify once they have begun, regardless of the technological difficulties that may arise. Project failure, cancellation, or delay, for example, would have severe detrimental effects- unexpected layoffs, unusable land and facilities, and millions of dollars lost. CHALLENGES THAT ARISE IN FIRST-OF-A-KIND PROJECTS First-of-a-kind projects have some inherent features that make them much more prone to failure than ordinary public or private projects. Technical Risk The inevitable technical challenges that emerge throughout the project devel- opment and construction process create high levels of uncertainty. As mentioned above, first-of-a-kind projects are by nature concerned with the development and implementation of new technologies. These projects may require the use of nontraditional materials and new, complex, and demanding (high-performance) systems. At the beginning of the project, the only information about these inno- vative systems and materials is derived from scientific experiments and labora- tory or pilot tests, so that construction of a first-of-a-kind facility often involves

OCR for page 40
READINESS TO PROCEED FOR FIRST-OF-A-KIND PROJECTS 43 scaling up laboratory or bench models. The ability of the process to meet the scaled-up design parameters is highly uncertain and depends on the validity of numerous scientific and technical assumptions and projections. This uncertainty creates high financial and environmental risk. Managerial Challenges Uniqueness and uncertainty caused by technical issues must be addressed by an adequate decision-making process (performed by DOE as the project owner) and management (performed by DOE and its contractors). A number of issues make managerial processes much more challenging than the decision-making and management processes for ordinary projects where managers have prior expenence. Uncertainty in Planning The uncertainty caused by technical issues hinders the ability of the project team to perform accurate and realistic project planning. Thus, planning for project execution requires careful attention to risk management. First-of-a-kind projects are typically accompanied by risks of potentially large cost and schedule overruns and performance shortfalls. Risk Management First-of-a-kind projects involve technologies and approaches that lie on the boundary between pure science and practical implementation, which is why they involve a higher degree of uncertainty. To address this issue, an explicit risk man- agement program should be undertaken, from the earliest phases of preconceptual planning and continuing throughout the entire project development process. Cost and Schedule Contingency The inherent uncertainty in project definition for first-of-a-kind projects re- sults in more changes and a corresponding need for larger contingency for both costs and schedules, making the construction control process more difficult. Therefore, special attention must be paid to the development of a rigorous contin- gency tracking system. During the preconceptual and conceptual planning stages, owing in part to the strong desire to proceed with a project, there is a tendency to underestimate the potential risks. This often leads to inadequate contingency allocation. If the contingency is set at a level appropriate for a conventional project with well-defined scope, as has happened in the past, the contingency allowance will not adequately address the schedule and cost variability, resulting in a continual need to revise the project baseline.

OCR for page 40
44 Personnel Resources PROGRESS IN IMPROVING PROJECT MANAGEMENT AT THE DOE Problems may arise because first-of-a-kind projects often need unique scien- tific and engineering resources. The development of new technologies requires highly qualified, experienced professionals to be involved in both design and construction. The scale and the schedule of these projects call for employing highly competent staff experienced in dealing with the uncertainty inherent in first-of-a-kind projects. Another issue that typically creates problems during the execution of first- of-a-kind projects is the creation of a work environment that promotes teamwork. The involvement of large numbers of laboratories, contractors, and specialists with diverse backgrounds and areas of competence could create tensions if the respective tasks, responsibilities, and accountabilities are not clearly and explicitly defined. Vaguely defined roles and responsibilities can make it difficult to assess responsibility for failures. The nature of the work on first-of-a-kind projects makes it difficult to keep the staff focused on the project objectives when there are unexpected technical and scientific challenges. First-of-a-kind projects often take a long time to complete, resulting in turnover of critical, experienced project personnel. Funding The first-of-a-kind projects undertaken by DOE often require huge invest- ments and entail great risks that private companies are not able to accept. Govern- ment undertakes such projects because they benefit the nation. Some projects are jointly funded by other nations to reduce each party's risk, but the principal source of funds is typically the U.S. Treasury. Congress controls these resources, and the availability of funds is subject to budget limitations, annual budget appro- priations, competing interests, and other factors. Availability of funding needs to be assessed at each project decision point. CRITICAL DECISIONS First-of-a-kind projects call for critical go/no-go decisions at various stages of project development and execution. The timeliness of these decisions is criti- cal for the project outcome. For acquisition executives to make these critical decisions, they need adequate information. Therefore, during the preproject planning phase of first-of-a-kind projects, special procedures are needed to assure the development of information that will facilitate a responsive decision-making process. First-of-a-kind projects have been and can be successfully managed with respect to performance, budget, and schedule. Successful project performance depends on adequate planning and management. In first-of-a-kind projects with

OCR for page 40
READINESS TO PROCEED FOR FIRST-OF-A-KIND PROJECTS 45 technological challenges, it is often assumed that the highest priority must go to scientific and engineering issues. This is not so. The critical issues for these projects are project planning and project management, which are essential to provide the basis for successful resolution of the technological issues. The following issues should be addressed in the review of risk assessments, project procurement and execution plans, and other management areas for first- of-a-kind projects. It is unlikely that all of these issues would be faced on any single project, but all of the methods cited here have been used on some DOE projects. The readiness to proceed from one stage of the project to the next depends on the level of definition and uncertainty; a level of definition that is acceptable for approval to proceed at CD-0 will probably not be acceptable at CD- 1 or CD-2. Project Benefits to the Public First-of-a-kind projects may require large public investments. In return, DOE should continually demonstrate how the project will benefit the public. If the benefits to the nation are only marginally greater than the costs, the project may be at risk at any time due to budget constraints, lack of political support, or even public opposition. Benefits as well as costs should be quantified and estimated, and cost-benefit analyses should be made at every phase of the project and reviewed by DOE management. These costs and benefits should be readily comprehensible to the stakeholders that is, the general public, which is paying for the project. Scope In first-of-a-kind projects it may be very difficult to define the scope pre- cisely at the beginning of the project. The project may require decisions at critical decision points with less than desirable definition of the systems to be used. For example, the site plan, footprint, and general arrangements of the facility may not be well defined at CD-0 or CD-1. Only a conceptual plan and general building arrangements may be available until project systems are better defined and sized. The practice of setting contingencies for costs and schedules is appropriate, but the practice of changing project scope to maintain the baseline cost and schedule is definitely not a best practice. Costs and Schedules In first-of-a-kind projects there are no historical cost records that can be referenced from earlier projects. DOE has to depend heavily upon judgment, analogies with previous projects, and independent (external and internal) cost reviews. As costs of first-of-a-kind projects are more uncertain owing to the

OCR for page 40
46 PROGRESS IN IMPROVING PROJECT MANAGEMENT AT THE DOE lower level of technical definition, it is essential to obtain cost estimates by several different and independent methods and sources. Bottom-up cost estimates should of course be made as soon as design information becomes available and should be updated as more information is developed. Adequate attention should be paid to the uncertainties in scope definition and the likelihood that design development will identify additional costs. Top-down cost estimates should be made using whatever parametric or statistical methods may be applicable and used as reality checks on the bottom-up estimates. It is essential that the top-down and bottom-up estimates be completely independent and prepared by different groups with no vested interest in the project. The difference between the top- down and the bottom-up estimates is one indicator of the degree of uncertainty in the project costs. If independent top-down and bottom-up estimates differ sig- nificantly, management should immediately investigate the reasons for these differences. In first-of-a-kind projects it is difficult to schedule activities with precision because some activities may not be clearly defined until the project is well under way. Because a detailed schedule has not been set, schedule risk and contingency analysis require more attention, especially during the early stages of a first-of-a- kind project. The uncertainty inherent in first-of-a-kind projects typically requires flexibility in planning and scheduling. Options should be identified and kept open as long as necessary. Parallel technological developments and multiple suppliers, for example, can address technological risk. More hold points may be required to determine whether or not to go ahead with new or proven technology. Special attention should be given to the schedule impacts of external events and decisions by decision makers not under DOE or contractor control, such as regulators. Constructability Constructability analysis is intended to identify the problems that may be faced during the course of construction. This analysis assesses the probability of the project running over budget and schedule and suggests design changes that would reduce costs and time by making the project easier to construct. Con- structability analyses, performed early and often, assess the ability to carry out the project in the planned manner, within budget and schedule. Constructability analysis should be started in the very first phase of planning, whenever a design or a plan to build something is conceptualized. Prototype Studies In planning any new technology project, it is beneficial to build a pilot or bench model rather than immediately building the full-scale facility, so that the process design may be checked, proven, and refined earlier, at a much lower cost.

OCR for page 40
READINESS TO PROCEED FOR FIRST-OF-A-KIND PROJECTS 47 A decision to proceed with the project at CD-1, CD-2, or CD-3 should depend on the degree of confidence that the full-scale facility will perform as intended based on the performance of the laboratory-scale models. Alternatives Studied First-of-a-kind projects often involve new technologies that have not been proven, and it may not be clear at the outset that all the technologies will perform as required. Hence efforts are needed to identify alternatives and to maintain them until at least one has been proved successful. In many cases, early involve- ment of and input from equipment vendors is essential to making good decisions. The project planning and scheduling process should specify in the project sched- ule the dates by which these technological decisions have to be made to avoid extending the project completion date. MANAGEMENT PLANNING AND CONTROL As discussed above, management planning and control are the areas that need the most improvement for DOE first-of-a-kind projects. The greater tech- nical and organizational complexity, combined with the lack of experience that is characteristic of first-of-a-kind projects, poses additional challenges for project management. The following items address such management challenges. Organizational Breakdown Structure All human resources and specific competencies required for the project should be recognized in the organizational breakdown structure. Consistent with the work breakdown structure, the organizational breakdown structure identifies the key project participants and assigns responsibilities. Of particular concern is the early and clear identification of the owners and the users of the facility. This has not always been clear from the outset. Commitment Tracking System A commitment tracking system monitors the fulfillment of responsibilities. Procedures for assigning and tracking the commitments and responsibilities of all the participating parties throughout the entire project are developed and resources for the implementation are allocated. Integrated Documentation System Integrated information systems designed to collect, store, and process project- related information need to be developed and implemented. These systems should

OCR for page 40
48 PROGRESS IN IMPROVING PROJECT MANAGEMENT AT THE DOE be integrated to serve the needs of DOE project management, DOE users, con- tractors, subcontractors, and regulators. DOE Order O 413.3 requires the use of an earned value management system (EVMS). Effective use of EVMS requires the integration of project costs with the project schedule, through a resource- loaded project network capable of generating and tracking the budgeted cost of work scheduled, the budgeted cost of work performed, and the actual cost of work performed, for all contractors and subcontractors. To meet the require- ments of O 413.3, an integrated cost and schedule system should be established as soon as possible. Risk Management To address the greater uncertainty characteristic of first-of-a-kind projects, an explicit risk management program should be undertaken, starting from the earliest phases of preconceptual planning, and performed as a continuous process throughout the life of the project. In risk identification and assessment, all potential risk factors should be recognized and evaluated. The methods for performing this evaluation depend on the degree of uncertainty and complexity. They could range from the analysis of failure modes and effects to probabilistic risk assessment. Each risk identified as significant should have a risk mitigation and management plan, and the dem- onstration of a satisfactory risk management plan should be a condition for proceeding at every critical decision point. FINDINGS AND RECOMMENDATIONS Finding: First-of-a-kind projects have been and can be successfully managed and executed by DOE, but they require particular care. The higher degree of uncertainty that attends these projects requires managers who are experienced in dealing with uncertainty and ambiguity. Not all project managers have this ability. The best project managers and management systems more than pay for themselves on first-of-a-kind projects by delivering projects on schedule with little budget overrun. Recommendation: DOE managers and acquisition executives should pay par- ticular attention to the unique characteristics of first-of-a-kind projects by consid- ering the issues discussed above for example, costs and benefits, scope, cost and schedule budgets, constructability, alternatives, management planning, and project controls at all critical decision points.