Front-end planning is, in many senses, the most critical phase of a project and the one that often gets least attention. The front-end planning process defines the project. The decisions made in this phase constrain and support all the actions downstream and often determine the ultimate success or failure of the project. Projects with adequate front-end planning do not always succeed, but those with inadequate front-end planning most often fail (CII, 1995). Typically, a project will not be better than its front-end planning process.
The front-end planning stage encompasses determination of the mission need or business objective, the scope for a project to fulfill the mission or objective, project justification, basic project definition, an outline of the general design, approximate benefits and costs, funding sources, risk factors facing the project, a basic organizational structure for the project, and a preliminary project execution plan. Based on the information developed in this phase, senior management must determine whether to approve, terminate, or modify the project. Unfortunately, this activity often takes place with insufficient attention from senior management, who often are unaware of the process and whether it has been adequately performed. Senior managers who do not spend time at the early planning stage to get a project started right will probably spend a lot of time later to fix it.
There are many approaches to front-end planning, which is also known as preproject planning, preconstruction planning, project programming, feasibility analysis, schematic design, scope definition, or conceptual planning. Whatever it is called, successful front-end planning requires the active involvement of senior management before decisions are made that will determine the fate of a project.
The Phase II report identified early project planning as a major factor affecting project success (NRC, 1999). It noted that inadequate definition of project scope and inadequate preconstruction planning lead to cost overruns, schedule overruns, and failure to achieve the intended project scope and performance. The report also found that adequate initial project definition was a continuing problem in DOE: “Statistical studies showed that inadequate project definition (detailed planning of scope, objectives, resources) accounts for 50 percent of the cost increases for environmental remediation projects.” The report also noted that the DOE was setting project baselines too early and based on too little design information.
In October 2000, DOE issued Order O413.3, Program and Project Management for the Acquisition of Capital Assets, which defined the critical decision steps from CD-0 through CD-4 and a fairly detailed project-planning process as part of the capital budget cycle. OECM, in conjunction with Congress, has begun to develop a funding mechanism for project engineering and design (FED) (DOE, 2000a). As noted in the Phase II report, adequate FED funding, preconstruction planning, and project controls are all critical to successful projects. The committee reaffirms the Phase II recommendations for DOE to improve preconstruction planning and performance baselines. The committee applauds the positive steps taken by DOE for implementing preconstruction planning; however, much more management attention to improving front-end planning is needed.
An example of the need for very early project planning is the Next Linear Collider, which has been publicly proposed by the Stanford Linear Accelerator Center (SLAC), Fermilab, the Office of Science, and other laboratories. Although this project is still in the early conceptual stage, and even the country of location is undecided, the need for early front-end planning is demonstrated by the fact that cost estimates (“more than $6 billion”) have already been published in the general press (Glanz, 2001a; Glanz, 2001b; Seife, 2001). It is never too early to start front-end planning, and with cost estimates having been made public, front-end planning should already be under way.
OECM has limited documentation of project planning procedures and expects to revise and expand the descriptions in the draft Program and Project Management manual (PPM) (DOE, 2000b) and the draft Project Management Practices (PMP) (DOE, 2000c), which are reviewed in Chapter 8 of this report. The committee believes that OECM and the PMSOs should seek out and implement the best, most up-to-date front-end planning methodologies. As one example, the Construction Industry Institute (CII) has defined front-end planning as “the process of developing sufficient strategic information with which owners
can address risk and decide to commit resources to maximize the chance for a successful project” (CII, 1995). In this handbook, CII breaks down the front-end planning process into four steps, as shown in Figure 3–1: (1) organize for planning, (2) select project alternative(s), (3) develop a project definition package (which is the detailed scope definition of the project), and (4) decide whether to proceed with the project.
Front-end planning procedures should be focused on the process to be followed by DOE as the owner, user, and operator of the facility even though a contractor may undertake the actions. An appropriate front-end project planning process would help DOE to identify the mission need for the project and aid in identification and evaluation of alternative approaches and assessment of the costs and risks of each. It should lead to a well-defined set of requirements and scope of work that form the basis for effective design. Front-end planning in the DOE project management system includes planning procedures from project conception through approval of the performance baseline (CD-2). The DOE process includes 20 to 30 percent design completion (preliminary design) as the basis for development of a preliminary scope, budget, and schedule for the project; definition of the project performance baseline; and project funding authorization from Congress (DOE, 2000a).
ASSESSMENT OF FRONT-END PLANNING IN DOE
A recent analysis of 65 external independent reviews (EIRs) using the EM project definition rating index (EM-PDRI) included an evaluation of project planning issues common throughout DOE (RCI, 2000). Almost all the projects in the study, including many FY2000 and FY2001 projects, had significant unresolved problems. Of the 65 EIRs evaluated, 26 were missing corrective action plans. Other problems included missing or deficient cost estimates, project schedules, alternative analyses, project risk management plans, and project organization documentation, all of which are key elements in front-end planning.
The committee took steps to assess the effectiveness of the current DOE front-end planning process by requesting CD-0, CD-1, and CD-2 documents for a selected sample of capital projects in the DOE portfolio (11 projects authorized in FY2000 and FY2001, including 5 projects from DP, 5 from EM, and 1 from SC). A list of the projects and the data requested are shown in Appendix B. Unfortunately, the responses to this request were varied, incomplete, and inconsistent. The documents received did not provide the information needed for an assessment of the planning process. The amount of material on each project varied from voluminous to nearly none. Regardless of its volume, the record of
the justification of project need, definition of scope, and history of decisions was typically insufficient to permit identification and assessment of the front-end planning process. It was often impossible to determine from the documentation why a project was being performed or how its scope had been determined. Some projects did have some records of evaluations of readiness to proceed, which provided snapshots of certain steps and decision points, but based on the lack of front-end planning information in the documents that were provided, the committee determined that the current front-end planning process as applied through FY2001 is still incomplete, disorganized, inconsistent, and unreliable.
In addition, presentations on specific projects and program office policies and procedures were made to the committee at various meetings. In general, the projects were selected by the PSOs (see Appendix B for a list of presentations). From the information provided, the committee could not identify any department-wide improvement in front-end planning. It noted that while some projects appeared to be well planned and on the right track, many others demonstrated problems typically associated with inadequate front-end planning. The observed problems occurred irrespective of project size, complexity, or originating program organization.
It should be noted that all the projects reviewed by or presented to the committee were initiated prior to the activation of the OECM, the establishment of the PMSOs, and the issuance of O413.3, so that this negative assessment does not show these initiatives to be ineffective. What it does show, based on the evidence provided to the committee, is that the DOE front-end planning process for projects initiated in FY2000 and FY2001 has not noticeably improved and probably will not improve until the reforms that have been implemented have had time to become effective. Further assessments of the situation, including a review of projects funded in 2002 (when appropriate), will be made by the committee in the future; for the time being, the committee reiterates that improved project performance will require an improved front-end planning process, and that any improvement in front-end planning will require positive, aggressive action by all responsible parties.
Even though the department has issued a new policy (O413.3) and procedures (PPM manual) including front-end planning, not all program offices have incorporated these requirements into their project planning procedures. Differences in approach and attitude among the three major programs are evident.
Defense Programs (DP)—and Military Applications and Stockpile Operations (DP-20) in particular—have recognized the benefits that will accrue by following a rigorous front-end planning process and are making progress toward formalizing a multiyear implementation program. One of the more distinctive features of the DP approach is viewing the front-end planning process (through CD-2) as being a programmatic rather than a project function, thus requiring more departmental involvement. DP is trying to overcome the “wish-list syndrome” by integrating and prioritizing projects over the long term and developing
budgets based on long-range plans. The committee supports the DP-20 approach and encourages its department-wide application.
EM has adapted the CII Project Definition Rating Index (PDRI) for use in its project reviews (DOE, 2001). Tools such as the EM-PDRI can help ensure consistency in front-end planning and give planners a means to assess the probability that projects will perform as planned. However, the EM-PDRI will not achieve its full potential until EM personnel are sufficiently trained in its use and it becomes an integral part of the planning process rather than an after-the-fact review tool.
The committee has observed that SC projects often involve experimental, one-of-a-kind technology, apparently leading the office to believe that documenting planning decisions consistently is not productive and that it would be impossible to devise a process appropriate for all its projects. Also, SC does not seem to have a complex-wide system for integrating and prioritizing projects for future years but validates projects as part of the annual budgetary process. By necessity, the big science projects have longer planning horizons, but nevertheless they demonstrate instances of inadequate or inappropriate front-end planning.
SC projects are typically proposed by the laboratories, individual scientists, or the research community. Projects are planned in a series of workshops in which the project scope, purpose, and research programs are developed and refined, based on inputs from the research community. Workshop participants, who are predominantly scientists and researchers, do not necessarily have recent or extensive experience with project management. The committee believes that workshops need to include project management professionals to provide support for front-end project planning, including cost and schedule estimates and risk management. Effective front-end planning should not wait for scientific consensus on scope and design.
It was noted that all program offices approach front-end planning differently for infrastructure projects and for program mission-driven projects. While smaller, less complex infrastructure projects may warrant less management attention, the components of the planning effort should be the same. It was also noted that while determining mission need should be a program office responsibility, mission need appears to emanate from the contractors and laboratories, with only perfunctory DOE oversight. The committee observes that determination and documentation of mission need are the responsibility and obligation of the owner, even when contractors perform the documentation.
Finding. Compliance with the front-end planning requirements in O413.3 has been inconsistent among PSOs and among individual projects.
Recommendation. OECM should assure that all program offices have a documented front-end planning process that meets the intent of O413.3, and that the information used as input for Energy Systems Acquisition Advisory Boards (ESAABs) and ESAAB-equivalent readiness reviews, as well as the outcomes of
these reviews, is documented and used to assess project performance and progress in improving project planning.
Recommendation. The PMSOs should consider developing tailored checklists such as the EM-PDRI as in-process planning tools, train project personnel to use them, and analyze their effectiveness for projects throughout the DOE complex. Effective and consistent front-end planning should be made mandatory for all projects.
Finding. Tools such as checklists, communications software/methods, planning reviews, third-party audits, economic modeling, objective setting, and team building, if used correctly, can contribute to effective front-end planning. Performance of technical evaluation during planning is essential for projects involving new technology, complex site conditions, and complex project-flow requirements. Consistent documentation and planning structure would increase the effectiveness of front-end planning in the department.
Recommendation. OECM should clarify, expand, and revise the front-end planning procedures in the Program and Project Management manual and Project Management Practices. DOE should use standard industry procedures where applicable; however, the PMSOs should provide supporting policies and procedures tailored to the specific projects and needs of each program. The PMSOs and OECM should assure the adequacy of front-end project planning prior to each critical decision, to assure that projects are not unnecessarily delayed by poor plans and that time constraints do not cause projects to be approved without adequate planning.
Recommendation. The deputy secretary and the designated program acquisition executives should strengthen their interest and support, thereby confirming that truly effective front-end planning will be required without exception. OECM and the PMSOs should pay close attention to documentation of front-end planning decisions.
Project Engineering and Design Funding
The committee is convinced that investing in front-end planning is essential for the success of DOE project management, and it is encouraged by the creation of a funding mechanism to complete preliminary engineering and design as the basis for refined cost and schedule estimates prior to approval of the performance baseline. The committee encourages DOE to continue its development of preliminary engineering and design and other measures to define and manage risks and improve the accuracy and reliability of cost and schedule estimates.
Finding. DOE has established a process to significantly increase the accuracy and reliability of project baselines.
Recommendation. OECM should actively participate in the process and monitor the performance of projects baselined under this new process to document its impact and opportunities for improvement.
Front-end planning will be successful only with the involvement and support of senior management. For success, DOE senior management should insist that every project be effectively planned from its conception. Senior management should understand the process and should assure that effective project planning is being conducted. This can be accomplished by a number of means:
Questioning at project review meetings,
Providing resources to support the process implementation and training,
Maintaining discipline in sticking to the plan, and
Benchmarking results (NRC, 2001).
Because contractors are frequently the users and operators, their involvement in the front-end planning process is appropriate and necessary in most cases. However, DOE, as the project owner, has the primary responsibility for front-end planning. The committee recognizes that improvement in front-end planning cannot be incorporated uniformly in all DOE projects in a short time frame. However, consistency in front-end planning will not be achieved as long as DOE delegates this activity to contractors without also providing prescribed procedures, products, and performance measures, as well as adequate supervision. Effective front-end project planning will require both process and cultural change within the organization.
Finding. Overall, insufficient attention from DOE management is being given to the front-end planning process; however, the committee observed that management was acting in isolated cases and to varying degrees within the program offices.
Recommendation. DOE senior management should emphasize the importance of thorough and complete front-end planning (including written documentation). ESAABs and ESAAB-equivalent reviews should be used to enhance the quality of front-end project planning and assure that the project team is pursuing the right project—that is, that the project has adequate justification and will satisfy a well-conceived need.
FRONT-END PLANNING METRICS
The Phase II committee sought metrics by which to evaluate DOE’s project management functions (NRC, 1999, Appendix A). Without adequate metrics, it was very difficult to evaluate the effectiveness of DOE planning practices or to compare practices among DOE projects or between DOE and private sector projects. It is equally difficult for management to address problems that exist in the diverse pool of projects that DOE performs.
For example, CII developed its project definition rating index (PDRI) to assess and guide front-end planning and related planning practices. The PDRI assesses 70 project scope-definition elements (CII, 1996, 1999). CII’s Benchmarking and Metrics Data Report includes front-end-planning metrics derived from 23 questions in the PDRI (CII, 2000). The CII database includes over 1,000 projects representing approximately $52 billion in construction costs. The CII data show a positive correlation between front-end planning and project performance in terms of cost, schedule, change orders, and operational performance. The mean percentage of total project cost spent on front-end planning activities was 4.3 percent for the industrial projects and 2.4 percent for the building projects in CII’s benchmarking database.
DOE does not currently have enough data to compare its front-end planning and project performance with best industry practices and performance. The DP PMSO has taken some positive steps to develop a benchmarking database to compare its projects with the CII database, and this positive action should be continued and extended by all PMSOs and the OECM. Also, DOE recently joined CII, so it now has access to CII’s database.
Finding. Front-end planning improvement requires metrics for trend analysis. The committee was not able to obtain this information for specific projects because DOE does not have enough data for front-end planning trend analysis.
Recommendation. OECM should begin benchmarking project practices and performance metrics to identify areas in need of improvement and establish a baseline for future evaluation. This benchmarking effort should be systematic, quantitative, and analytical, and it should compare practices in industry and in other government agencies. It should capture both front-end planning and performance metrics, including actual performance versus forecast.
HUMAN RESOURCES FOR FRONT-END PLANNING
The skills needed to effectively manage the different types of DOE projects are based on technical knowledge, management experience, and personal traits. Individuals involved in science or equipment-type projects need a strong background in process engineering, mechanical engineering, or chemical engineering.
Those involved in environmental remediation projects need extensive background in environmental engineering or chemical engineering. A person perfectly at ease working on a relatively low-risk project may be lost on a project that is highly complex and changing extensively during front-end planning.
Effective planners have the technical knowledge to understand the project mission and the facilities, equipment, and processes needed to satisfy the project requirements. The project manager’s experience should correspond to the level of risk and complexity of the project. In addition, project managers need personality traits that facilitate collaborative relationships. Because these skills and abilities are not easily developed after a project manager has been assigned, they should be considered as criteria for hiring and assigning personnel to planning assignments.
Finding. A training program addressing front-end planning and other project management practices is being developed. The completion date of this effort was reported to the committee to be December 2002, with training to start soon afterward. Without immediate improvement in the planning knowledge and skills of personnel and more management emphasis on improving the planning process, projects will continue to have inadequate front-end planning.
Recommendation. OECM should do more than develop policies and procedures—it should become fully engaged in process improvement beginning with front-end planning. To overcome the lack within the department of skilled project planners and the delays in training, and to bridge the gap until a training program takes effect, DOE should establish a cadre of experienced project planners within OECM; they should have a wide variety of planning capabilities and prior experience in different project types, including high-risk projects. These individuals should be a part of the initial integrated project teams and should assist the project originators (as internal consultants) in getting front-end planning done correctly, including planning prior to CD-0. This cadre of internal consultants should champion the DOE front-end planning process, providing just-in-time training for front-end planning to project teams. DOE should benchmark its management of project planning personnel and application of their expertise with that of private sector companies that have successfully undertaken similar activities. In this way, DOE may be able to jump-start an immediate improvement in planning capability.
Recommendation. DOE should eliminate impediments to initiating training for front-end project planning prior to December 2002. Training should begin as soon as possible.
CII (Construction Industry Institute). 1995. Preproject Planning Handbook (Special Publication 39–2). Austin, Tex.: Construction Industry Institute.
CII. 1996. Project Definition Rating Index (PDRI)—Industrial Projects (Implementation Resource 113–2). Austin, Tex.: Construction Industry Institute.
CII. 1999. Project Definition Rating Index (PDRI)—Building Projects (Implementation Resource 155–2). Austin, Tex.: Construction Industry Institute.
CII. 2000. Benchmarking and Metrics Data Report. Austin, Tex.: Construction Industry Institute.
DOE (U.S. Department of Energy). 2000a. Program and Project Management for the Acquisition of Capital Assets (Order O413.3). Washington, D.C.: Department of Energy.
DOE. 2000b. Program and Project Management. Draft. Washington, D.C.: Department of Energy.
DOE. 2000c Project Management Practices. Draft. Washington, D.C.: Department of Energy.
DOE. 2001. Office of Environmental Management Project Definition Rating Index Manual. Washington, D.C.: Department of Energy.
Glanz, James. 2001a. “Physicists Unite, Sort of, on Next Collider.” The New York Times. July 10, 2001, pp. D1-D2.
Glanz, James. 2001b. “To Be Young and in Search of the Higgs Boson.” The New York Times. July 24, 2001, p. D3.
NRC (National Research Council). 1999. Improving Project Management in the Department of Energy. Washington, D.C.: National Academy Press.
NRC. 2001. Improved Project Management in the Department of Energy. Washington, D.C.: National Academy Press.
RCI (Resource Consultants, Inc.). 2000. External Independent Review Analysis. Washington, D.C.: Office of Engineering and Construction Management, Department of Energy.
Seife, Charles. 2001. “Plans for Next Big Collider Reach Critical Mass at Snowmass.” Science 293 (5530):582.