The U.S. Department of Energy (DOE) and its predecessor agencies have been at the center of many of the great achievements in science and engineering in this century. They have built many of the world's best research facilities, funded excellent academic research, and developed and maintained the nuclear arsenal. DOE's diverse missions are supported by hundreds of projects, ranging from hazardous waste cleanups at individual sites of the weapons complex to the construction of scientific facilities. Many are complex one-of-a-kind projects that rely on technologies that are unproven at field scale. DOE currently spends billions of dollars annually and plans to construct more than $20 billion worth of defense, energy research, environmental management, fissile materials disposition, and other projects in the next five years. In recent years DOE's budget has been dominated by the monumental task of environmental restoration and waste management to repair damage caused by the production of nuclear weapons in the past.
Documentation shows that DOE's construction and environmental remediation projects take much longer and cost about 50 percent more than comparable projects by other federal agencies or projects in the private sector. Moreover, DOE projects commonly overrun their budgets and schedules, leading to pressures for cutbacks that have resulted in facilities that do not function as intended, projects that are abandoned before they are completed, or facilities that have been so long delayed that, upon completion, they no longer serve any purpose. In short, DOE's record calls into question the credibility of its procedures for developing designs and cost estimates and managing projects.
To identify the root causes of these problems, this committee reviewed
DOE's performance on projects and assessed the organizations and procedures by which they are carried out. The objectives of this study, which were established in the 105th Committee of Conference Report on Energy and Water Development are to (1) review and assess the processes used by DOE and its contractors to identify project requirements, develop scopes of work, execute and manage designs, prepare cost estimates, select contract types, and execute and manage environmental restoration, waste management, and construction projects; (2) assess the level of oversight and experience of personnel in field offices and at DOE headquarters; (3) analyze the effectiveness of current DOE practices and recommend improvements; and (4) recommend guidelines for management and contracting that would help DOE establish an overall departmental process with more control of projects to reduce cost and schedule overruns. In the collective opinion of the committee, if DOE acts upon the findings and recommendations of this report, project performance would be significantly improved.
The fundamental deficiency is DOE's organization and culture, which do not provide a focus for project management. As a result, the processes used by field offices, operations offices, and their contractors for planning and executing projects are inconsistent; lessons learned about cost estimating techniques, project review processes, change control mechanisms, and performance metrics are not transferred from one project to another; and there is no systematic program for recruiting and training professional project managers and no career path for project management. Related fundamental problems are a general lack of accountability and unclear lines of authority. The areas of authority of field and operations offices, contractors (who perform most of DOE's work), and DOE headquarters programs are complex and overlapping. Problems vary by location, reflecting the influence of local management and local contractors.
Although DOE has attempted to address these problems, in the view of many people inside and outside of DOE (including this committee), it has not succeeded. The creation of the Office of Field Management might have become the basis for an effective DOE project management organization, but the office took on only an advisory and oversight function, essentially as an advocate for the field offices with DOE headquarters. The Contract Reform Initiative (intended to increase competition and accountability in DOE project acquisition through new approaches to project financing and contracting) was a positive step that has some successes to its credit, but it has not achieved substantive or consistent results. The adoption of the recommendations from the National Research Council Phase I report to implement independent external project reviews, and the recent establishment of internal project reviews in the major program offices could become an important tool for continued improvements.
The committee compared DOE's project management practices with the standard practices used by private industry and other government agencies and found that DOE falls far short of best practices in a number of areas:
- organization-wide project management policy
- clear definitions of responsibility and accountability
- control of changes in the scope, cost, and definition of projects
- state-of-the-art project management systems
- identification, dissemination, and implementation of lessons learned
- pre-project and preconstruction planning
- scope definition at the project baseline stage
- assessing and managing project risk
- setting contingency allowances based on risk
- cost estimation and scheduling
- objective performance-based incentives
- performance measurement and progress reports
- DOE-wide financial reporting systems
- cost and performance databases and information systems
- selection, training, and qualification of project managers
- project management core competency and organization
Findings and Recommendations
DOE's portfolio of projects is large, complex, and sophisticated. Many projects are one of a kind, involving unique systems, processes, and technical challenges. Delivering projects of this magnitude that meet baseline costs and schedules is a constant challenge that requires excellent management. The findings and recommendations that follow provide guidelines for lifting DOE's project management to a level commensurate with other agencies and private industry. No single change will raise DOE's project management to the level required for such vital and expensive projects, because the problems are pervasive and cultural, and resolving them will require more than a quick fix. DOE must undertake a broad program of reform for the entire project management process.
This program of reform is set out in the recommendations, culminating in the recommendation that an office of project management be established to implement these reforms and drive cultural changes in DOE. To be effective, the proposed project management office must include the staff necessary to support the project managers and must provide consistent methods and systems for cost estimation, risk analysis, contracting, incentives, change control, progress reporting, and earned value management. The reform will require full and continuing support of the Secretary of Energy to ensure the support of program offices, field offices, and the entire DOE project management organization.
Policies, Procedures, Documentation, and Reporting
Finding. DOE does not have adequate policies and procedures for managing
projects. No single authority is responsible for enforcing or ensuring that project management tools are used.
Finding. DOE has developed comprehensive practice guidelines for the design and construction phases of projects but has not developed comparable guidelines for the early conceptual and preconceptual phases, when the potential for substantial savings is high.
Finding. Many DOE projects do not have comprehensive project management plans to define project organization, lines of authority, and the responsibilities of all parties.
Finding. DOE does not effectively use value engineering to achieve project savings, even though federal agencies are required to do so.
Finding. DOE project documentation is not up to the standards of the private sector and other government agencies.
Finding. DOE does not have a consistent system for controlling changes in project baselines.
Finding. DOE does not effectively use available tools, such as earned value management, to track the progress of projects with respect to budget and schedule.
Finding. ISO 9000 provides a certification process by which an organization can measure itself against its stated goals, but DOE has not obtained certification. The certification process would help DOE remake the entrenched operating procedures and standards that have accumulated over the past 50 years.
Recommendation. As a part of its project management system, DOE should issue fundamental policies, procedures, models, tools, techniques, and standards; train project staff in their use; and require their use on DOE projects. DOE should develop and support the use of a comprehensive project management system that includes a requirement for a comprehensive project management plan document with a standard format that includes a statement of the project organization covering all participating parties and a description of the specific roles and responsibilities of each party.
Recommendation. DOE should update the project performance studies to document progress in these areas and extend the benchmarking baseline to include all major DOE construction projects. The study results should then be used to improve project procurement and management practices.
Recommendation. DOE should mandate a reporting system that provides the necessary data for each level of management to track and communicate the cost, schedule, and scope of a project.
Recommendation. DOE should establish a system for managing change that provides traceability and visibility for all baseline changes. Change control requirements should apply to the contractor, the field elements, and headquarters.
Recommendation. DOE should establish minimum requirements for a cost-effective earned-value performance measurement system that integrates information on the work scope (technical baseline), cost, and schedule of each project. These requirements should be included in the request for proposals.
Recommendation. DOE, as an organization, should obtain and maintain ISO 9000 certification for all of its project management activities. To accomplish this, DOE should name one office and one individual to be responsible for acquiring and maintaining ISO 9000 certification for the whole department and should require that consultants and contractors involved in the engineering, design, and construction of projects also be ISO 9000 certified.
Recommendation. DOE should establish an organization-wide value-engineering program to analyze the functions of systems, equipment, facilities, services, and supplies for determining and maintaining essential functions at the lowest life-cycle cost consistent with required levels of performance, reliability, availability, quality, and safety. Value engineering should be done early in most projects, and project managers should take the resulting recommendations under serious consideration.
Project Planning and Controls
Finding. DOE preconstruction planning is inadequate and ineffective, even though preconstruction planning is one of the most important factors in achieving project success.
Finding. DOE often sets project baselines too early, usually at the 2- to 3-percent design stage, sometimes even lower. (An agreement between Congress and DOE's chief financial officer for establishing baselines at the 20- to 30-percent design stage is scheduled to be implemented in fiscal year 2001.)
Finding. DOE often sets project contingencies too low because they are often based on the total estimated cost of a project rather than on the risk of performing the project.
Finding. DOE does not always use proven techniques for assessing risks of major projects in terms of costs, schedules, and scopes.
Recommendation. DOE should require that strategic plans, integrated project plans, integrated regulatory plans, and detailed project execution plans be completed prior to the establishment of project baselines. To ensure facility user and program involvement in the preconstruction planning process, DOE should require written commitments to project requirements from the ultimate users.
Recommendation. DOE should significantly increase the percentage of design completed prior to establishing baselines. Depending on the complexity of the project, the point at which project baselines are established should be between the completion of conceptual design and the completion of the preliminary design. which should fall between 10 and 30 percent of total design. The committee supports continuing efforts by Congress and the DOE to develop project baselines at a point of adequate definition beginning with fiscal year 2001.
Recommendation. Baseline validation should be assigned specifically to the project management office recommended in this report. The Military Construction Program of the U.S. Department of Defense, which requests planning and design funds for all projects in the preliminary design stage on the basis of total program size, is a potential model for DOE.
Recommendation. DOE should establish contingency levels for each project based on acceptable risk, degree of uncertainty, and confidence levels for meeting baseline requirements. The authority and responsibility for managing contingencies should be assigned to the project manager responsible for doing the work. In the process of evaluating potential projects, DOE should apply risk assessment and probabilistic estimating techniques, as required by the Office of Management and Budget.
Skills, Selection, and Training of Personnel
Finding. DOE's failure to develop project management skills in its personnel is a fundamental cause of poor project performance. DOE has shown little commitment to developing project management skills, as indicated by the lack of training opportunities and the absence of a project management career path. Successful organizations recognize that project management skills are an essential core competency that requires continuous training.
Recommendation. DOE should establish a department-wide training program for project managers. To ensure that this program is realistic, practical, and state of the art, DOE should enlist the assistance of an engineer/construction
organization with a successful record of training project managers. DOE should establish criteria and standards for selecting and assigning project managers, including documentation of training, and should require that all project managers be trained and certified. DOE should also require that all contractors' project managers be experienced, trained, and qualified in project management appropriate to the project.
Finding. Independent project reviews are essential tools for assessing the quality of project management and transferring lessons learned from project to project.
Finding. External independent reviews of 26 major projects are under way to assess their technical scope, costs, and schedules. The reviews so far have documented notable deficiencies in project performance verifying the committee's conclusion that DOE's project management has not improved and that its problems are ongoing. However, DOE has yet to formalize and institutionalize a process to ensure that the recommendations from these reviews are implemented.
Finding. Various DOE program offices are also developing the capability of conducting internal independent project reviews.
Recommendation. DOE should formalize and institutionalize procedures for continuing independent, nonadvocate reviews, as recommended in the Phase I report of the National Research Council to ensure that the findings and recommendations of those reviews are implemented. DOE should ensure that reviewers are truly independent and have no conflicts of interest.
Recommendation. All programs that have projects with total estimated costs of more than $20 million should conduct internal reviews, provided that the value of the reviews would be equal to or greater than the costs of conducting them. Deciding if an internal review is justified for a given project should be the joint responsibility of program management and the project management organization. The decision should be based on past experience with similar projects, the estimated cost of the project, and the uncertainty associated with the project. Internal reviews are expensive and take up the time of valuable people, so they should not be undertaken lightly. However, under the present circumstances, the committee believes that more internal reviews would be justified. The project management organization should manage these reviews for the director or assistant secretary of the cognizant program office. The results of these reviews should be taken by the program office to the Energy Secretary's Acquisition Advisory Board (ESAAB), and used as a basis for the decision whether to continue the project.
Acquisition and Contracting
Finding. Traditional DOE contracting mechanisms, such as cost-plus-award-fee and manage-and-operate (M&O) arrangements, are not always optimal for DOE's complex mission. These approaches are being replaced with more effective approaches based on objective performance incentives, but change has been slow.
Finding. DOE's long history of hiring contractors to manage and operate its sites on the basis of cost-plus-award-fee contracts has created a culture in which neither DOE nor its contractors is sufficiently accountable for cost and schedule performance.
Finding. DOE does not use effective performance-based incentives and does not have standard methods for measuring project performance.
Finding. DOE does not effectively match project requirements and contracting methods. Mismatching often results in cost and schedule overruns.
Finding. The numbers of bidders on major DOE contracts has been declining and in some cases have not elicited truly competitive bids. This may indicate that projects are not being appropriately defined and packaged and that the disincentives to bid often outweigh the incentives.
Recommendation. DOE should strengthen its commitment to contract reform focusing on the assessment and quantification of project uncertainties, the selection of the appropriate contract type and scope for each job, and increased use of performance-based incentive fees rather than award fees to meet defined project cost and schedule goals. A comprehensive risk analysis should be conducted before deciding whether to issue fixed-price contracts for work that involves a high level of uncertainty (such as new technology or incomplete characterization). Specific contract scopes and terms should be negotiated to define both DOE and contractor responsibilities to prevent cost overruns. Clear, written roles, authorities, and responsibilities should be established for DOE headquarters, field elements, contractors, and subcontractors for each contract. Guidelines should be provided for the appropriate times in the project for the selection of contractors.
Recommendation. DOE should develop written guidelines for structuring and administering performance-based contracts. The guidelines should address, but need not be limited to, the following topics: the development of the statement of work; the allocation of risks to whomever would be most effective at controlling the risks (either DOE or the contractor); the development of performance measures and incentives; the selection of the contracting mechanism; the selection of the contractor; the administration of the contract; and the implications of federal
and DOE acquisition regulations. DOE should train its employees in the roles and responsibilities of a performance-based culture and then hold both employees and contractors accountable for meeting these requirements.
Recommendation. DOE should provide financial rewards for outstanding contractor performance to attract bids from the best contractors. A DOE-wide policy should be developed that provides fiscal rewards for contractors who meet or exceed schedule, cost, and scope performance targets. Contractor fees should be based on contractor performance.
Recommendation. DOE employees and contractor employees essential to projects should be trained in acquisition and contract reform. The training of source selection officials and members of source evaluation boards should be expedited; a minimum level of training should be a prerequisite.
Organizational Structure, Responsibility, and Accountability
Finding. DOE's organizational structure makes it much more difficult to carry out projects than in comparable private and public sector organizations. Successful corporations and agencies responsible for major projects arrange their organizations to provide focused and consistent management attention to projects.
Finding. Too many people in DOE act as if they were project managers for the same project, and too many organizations and individuals outside the official project organizations and lines of accountability can affect project performance.
Finding. Compliance with DOE's policy requiring the establishment of performance agreements and self-assessments from the field has been limited and slow.
Recommendation. To improve its project management performance, DOE should establish an office of project management on a level equal to or higher than the level of the offices of assistant secretaries. Department-wide project management functions should be assigned to the project management office, and the director of this office should have the authority and the resources to set and enforce reporting requirements for all projects. Other responsibilities, such as property and asset management, should be assigned to existing DOE headquarters offices. To be successful, the office of project management must have the full and continuing support of the secretary, the under secretary, the deputy secretary, and of all of the program offices and field offices as a top-down management initiative.