Progress in Improving Project Management at the Department of Energy: 2001 Assessment (2001)

DOE’s portfolio of projects is large, complex, and sophisticated. Many projects are one of a kind, involving unique systems, processes, and technical challenges. Delivering projects of this magnitude that meet baseline costs and schedules is a constant challenge that requires excellent management. The findings and recommendations that follow provide guidelines for lifting DOE’s project management to a level commensurate with other agencies and private industry. No single change will raise DOE’s project management to the level required for such vital and expensive projects, because the problems are pervasive and cultural, and resolving them will require more than a quick fix. DOE must undertake a broad program of reform for the entire project management process.

This program of reform is set out in the recommendations, culminating in the recommendation that an office of project management be established to implement these reforms and drive cultural changes in DOE. To be effective, the proposed project management office must include the staff necessary to support the project managers and must provide consistent methods and systems for cost estimation, risk analysis, contracting, incentives, change control, progress reporting, and earned value management. The reform will require full and continuing support of the Secretary of Energy to ensure the support of program offices, field offices, and the entire DOE project management organization.

Finding. DOE does not have adequate policies and procedures for managing projects. No single authority is responsible for enforcing or ensuring that project management tools are used.

Finding. DOE has developed comprehensive practice guidelines for the design and construction phases of projects but has not developed comparable guidelines for the early conceptual and pre-conceptual phases, when the potential for substantial savings is high.

Finding. Many DOE projects do not have comprehensive project management plans to define project organization, lines of authority, and the responsibilities of all parties.

Finding. DOE does not effectively use value engineering to achieve project savings, even though federal agencies are required to do so.

Finding. DOE project documentation is not up to the standards of the private sector and other government agencies.

Finding. DOE does not have a consistent system for controlling changes in project baselines.

Finding. DOE does not effectively use available tools, such as earned value management, to track the progress of projects with respect to budget and schedule.

Finding. ISO 9000 provides a certification process by which an organization can measure itself against its stated goals, but DOE has not obtained certification. The certification process would help DOE remake the entrenched operating procedures and standards that have accumulated over the past 50 years.

Recommendation. As a part of its project management system, DOE should issue fundamental policies, procedures, models, tools, techniques, and standards; train project staff in their use; and require their use on DOE projects. DOE should develop and support the use of a comprehensive project management system that includes a requirement for a comprehensive project management plan document with a standard format that includes a statement of the project organization covering all participating parties and a description of the specific roles and responsibilities of each party.

Recommendation. DOE should update the project performance studies to document progress in these areas and extend the benchmarking baseline to include all major DOE construction projects. The study results should then be used to improve project procurement and management practices.

Recommendation. DOE should mandate a reporting system that provides the necessary data for each level of management to track and communicate the cost, schedule, and scope of a project.

Recommendation. DOE should establish a system for managing change that provides traceability and visibility for all baseline changes. Change control requirements should apply to the contractor, the field elements, and headquarters.

Recommendation. DOE should establish minimum requirements for a cost-effective earned-value performance measurement system that integrates information on the work scope (technical baseline), cost, and schedule of each project. These requirements should be included in the request for proposals.

Recommendation. DOE, as an organization, should obtain and maintain ISO 9000 certification for all of its project management activities. To accomplish this, DOE should name one office and one individual to be responsible for acquiring and maintaining ISO 9000 certification for the whole department and should require that consultants and contractors involved in the engineering, design, and construction of projects also be ISO 9000 certified.

Recommendation. DOE should establish an organization-wide value-engineering program to analyze the functions of systems, equipment, facilities, services, and supplies for determining and maintaining essential functions at the lowest life-cycle cost consistent with required levels of performance, reliability, availability, quality, and safety. Value engineering should be done early in most projects, and project managers should take the resulting recommendations under serious consideration.

Finding. DOE preconstruction planning is inadequate and ineffective, even though preconstruction planning is one of the most important factors in achieving project success.

Finding. DOE often sets project baselines too early, usually at the 2- to 3-percent design stage, sometimes even lower. (An agreement between Congress and DOE’s chief financial officer for establishing baselines at the 20- to 30-percent design stage is scheduled to be implemented in fiscal year 2001.)

Finding. DOE often sets project contingencies too low because they are often based on the total estimated cost of a project rather than on the risk of performing the project.

Finding. DOE does not always use proven techniques for assessing risks of major projects in terms of costs, schedules, and scopes.

Recommendation. DOE should require that strategic plans, integrated project plans, integrated regulatory plans, and detailed project execution plans be completed prior to the establishment of project baselines. To ensure facility user and program involvement in the pre-construction planning process, DOE should require written commitments to project requirements from the ultimate users.

Recommendation. DOE should significantly increase the percentage of design completed prior to establishing baselines. Depending on the complexity of the project, the point at which project baselines are established should be between the completion of conceptual design and the completion of the preliminary design, which should fall between 10 and 30 percent of total design. The committee supports continuing efforts by Congress and the DOE to develop project baselines at a point of adequate definition beginning with fiscal year 2001.

Recommendation. Baseline validation should be assigned specifically to the project management office recommended in this report. The Military Construction Program of the U.S. Department of Defense, which requests planning and design funds for all projects in the preliminary design stage on the basis of total program size, is a potential model for DOE.

Recommendation. DOE should establish contingency levels for each project based on acceptable risk, degree of uncertainty, and confidence levels for meeting baseline requirements. The authority and responsibility for managing contingencies should be assigned to the project manager responsible for doing the work. In the process of evaluating potential projects, DOE should apply risk assessment and probabilistic estimating techniques, as required by the Office of Management and Budget.

Finding. DOE’s failure to develop project management skills in its personnel is a fundamental cause of poor project performance. DOE has shown little commitment to developing project management skills, as indicated by the lack of training opportunities and the absence of a project management career path. Successful organizations recognize that project management skills are an essential core competency that requires continuous training.

Recommendation. DOE should establish a department-wide training program for project managers. To ensure that this program is realistic, practical, and state of the art, DOE should enlist the assistance of an engineer/construction organization with a successful record of training project managers. DOE should establish criteria and standards for selecting and assigning project managers, including documentation of training, and should require that all project managers be trained and certified. DOE should also require that all contractors’ project managers be experienced, trained, and qualified in project management appropriate to the project.

Finding. Independent project reviews are essential tools for assessing the quality of project management and transferring lessons learned from project to project.

Finding. External independent reviews of 26 major projects are under way to assess their technical scope, costs, and schedules. The reviews so far have documented notable deficiencies in project performance verifying the committee’s conclusion that DOE’s project management has not improved and that its problems are ongoing. However, DOE has yet to formalize and institutionalize a process to ensure that the recommendations from these reviews are implemented.

Finding. Various DOE program offices are also developing the capability of conducting internal independent project reviews.

Recommendation. DOE should formalize and institutionalize procedures for continuing independent, nonadvocate reviews, as recommended in the Phase I report of the National Research Council to ensure that the findings and recommendations of those reviews are implemented. DOE should ensure that reviewers are truly independent and have no conflicts of interest.

Recommendation. All programs that have projects with total estimated costs of more than $20 million should conduct internal reviews, provided that the value of the reviews would be equal to or greater than the costs of conducting them. Deciding if an internal review is justified for a given project should be the joint responsibility of program management and the project management organization. The decision should be based on past experience with similar projects, the estimated cost of the project, and the uncertainty associated with the project. Internal reviews are expensive and take up the time of valuable people, so they should not be undertaken lightly. However, under the present circumstances, the committee believes that more internal reviews would be justified. The project management organization should manage these reviews for the director or assistant secretary of the cognizant program office. The results of these reviews should be taken by

the program office to the Energy Secretary’s Acquisition Advisory Board (ESAAB), and used as a basis for the decision whether to continue the project.

Finding. Traditional DOE contracting mechanisms, such as cost-plus-award-fee and manage-and-operate (M&O) arrangements, are not always optimal for DOE’s complex mission. These approaches are being replaced with more effective approaches based on objective performance incentives, but change has been slow.

Finding. DOE’s long history of hiring contractors to manage and operate its sites on the basis of cost-plus-award-fee contracts has created a culture in which neither DOE nor its contractors is sufficiently accountable for cost and schedule performance.

Finding. DOE does not use effective performance-based incentives and does not have standard methods for measuring project performance.

Finding. DOE does not effectively match project requirements and contracting methods. Mismatching often results in cost and schedule overruns.

Finding. The numbers of bidders on major DOE contracts has been declining and in some cases have not elicited truly competitive bids. This may indicate that projects are not being appropriately defined and packaged and that the disincentives to bid often outweigh the incentives.

Recommendation. DOE should strengthen its commitment to contract reform focusing on the assessment and quantification of project uncertainties, the selection of the appropriate contract type and scope for each job, and increased use of performance-based incentive fees rather than award fees to meet defined project cost and schedule goals. A comprehensive risk analysis should be conducted before deciding whether to issue fixed-price contracts for work that involves a high level of uncertainty (such as new technology or incomplete characterization). Specific contract scopes and terms should be negotiated to define both DOE and contractor responsibilities to prevent cost overruns. Clear, written roles, authorities, and responsibilities should be established for DOE headquarters, field elements, contractors, and subcontractors for each contract. Guidelines should be provided for the appropriate times in the project for the selection of contractors.

Recommendation. DOE should develop written guidelines for structuring and administering performance-based contracts. The guidelines should address, but need not be limited to, the following topics: the development of the statement of work; the allocation of risks to whomever would be most effective at controlling

the risks (either DOE or the contractor); the development of performance measures and incentives; the selection of the contracting mechanism; the selection of the contractor; the administration of the contract; and the implications of federal and DOE acquisition regulations. DOE should train its employees in the roles and responsibilities of a performance-based culture and then hold both employees and contractors accountable for meeting these requirements.

Recommendation. DOE should provide financial rewards for outstanding contractor performance to attract bids from the best contractors. A DOE-wide policy should be developed that provides fiscal rewards for contractors who meet or exceed schedule, cost, and scope performance targets. Contractor fees should be based on contractor performance.

Recommendation. DOE employees and contractor employees essential to projects should be trained in acquisition and contract reform. The training of source selection officials and members of source evaluation boards should be expedited; a minimum level of training should be a prerequisite.

Finding. DOE’s organizational structure makes it much more difficult to carry out projects than in comparable private and public sector organizations. Successful corporations and agencies responsible for major projects arrange their organizations to provide focused and consistent management attention to projects.

Finding. Too many people in DOE act as if they were project managers for the same project, and too many organizations and individuals outside the official project organizations and lines of accountability can affect project performance.

Finding. Compliance with DOE’s policy requiring the establishment of performance agreements and self-assessments from the field has been limited and slow.

Recommendation. To improve its project management performance, DOE should establish an office of project management on a level equal to or higher than the level of the offices of assistant secretaries. Department-wide project management functions should be assigned to the project management office, and the director of this office should have the authority and the resources to set and enforce reporting requirements for all projects. Other responsibilities, such as property and asset management, should be assigned to existing DOE headquarters offices. To be successful, the office of project management must have the full and continuing support of the secretary, the under secretary, the deputy secretary, and of all of the program offices and field offices as a top-down management initiative.