The committee considers effective performance measurement and a benchmarking process to be essential activities of successful project management organizations. The committee notes that benchmarking is one of many tools that can be used for process improvement. In the case of project management, benchmarking is particularly useful because external benchmarks have already been developed by the Construction Industry Institute (CII) and others. Such external benchmarks can help develop an understanding of what constitutes good performance. As noted in Chapter 1, the NRC’s 2003 assessment report concluded that shortcomings in these areas persist in DOE (NRC, 2004a). In the present report, the committee provides guidance for establishing a system that will enable DOE to measure project management performance at both the project and program levels and to benchmark internally within and between program secretarial offices (PSOs) and externally with other federal agencies and the private sector. The corrective actions required for continued improvement of project management can be based on information gained in this way. The information provided in this report should be viewed not as the final word but rather as a first step toward development of a viable methodology to suit the needs and goals of DOE.
The establishment of an effective and sustainable performance measurement and benchmarking system requires the commitment of top management and of resources. While maintaining such a system is not without cost, its value has been demonstrated within the government and private-sector organizations (NPR, 1997).
This report does not contain new findings and recommendations. It is intended to provide guidance for implementing previous NRC recommendations on performance measures and benchmarking. Further, the committee believes specific recommendations on the implementation of performance measures and benchmarking would be inappropriate. For a system to be effective it should be crafted to fill multiple organizational needs and it should carry the imprimatur of the users and be accepted at all levels of the organization.
Measuring performance and benchmarking should be viewed as a routine, integral part of the project management process rather than a separate function. However, integration into the personnel appraisal system is to be avoided. Accountability for project outcomes continues to be important; however, benchmarking should emphasize the continuous and consistent application of the system and the resultant identification and implementation of improved project management procedures through lessons learned and feedback systems.
DEVELOPMENT OF A PERFORMANCE MEASUREMENT AND BENCHMARKING SYSTEM
The performance measures presented in Chapter 2 are fundamental and should be considered as a starting point for the development of a performance measurement and benchmarking system for DOE.
While additions and adjustments will be necessary to accommodate particular situations, consistency is essential for meaningful benchmarking. To assure department-wide consistency, it will be necessary to agree on the definition of the key performance measures, how they will be calculated, and a reporting format. Senior management will need to show its commitment and champion the process at all levels of the department. OECM should lead the process, but program offices and project directors and managers in the field need to play an active role as well.
Gathering data for calculating the various performance measures will largely be done by individuals at the project level. For this to happen with an acceptable level of certainty, the responsible individuals should be inculcated with the need for accuracy and the benefits of benchmarking and be committed to implementing the system. Above all, it must be made clear that no penalties or recriminations will be incurred for honest reporting. A high code of ethics must prevail. Although an outside contractor could help to establish the system and gather data in the early stages, the working input to the system must come from DOE personnel.
This report largely addresses the first four steps of the benchmarking roadmap presented in Figure 3.2:
Determine what to benchmark,
Define the measures,
Develop data collection methodology, and
The remaining five steps define the essence and purpose of benchmarking—that is, to continuously improve project management. For any benefits to be derived from the effort, it is essential that DOE establish a means for implementing the steps beyond the collection of data. This will require advocacy to be built into the system. The committee believes that corrective improvements to the project management process should be guided from within DOE, not from outside. As noted in the 2003 assessment report, this process should be led by a senior management champion with the full participation of OECM and the PSOs (NRC, 2004a).
Benchmarking data should not be used for individual performance reviews, but a project director’s application of the benchmarking process should be assessed. Performance measurement and benchmarking need to be part of DOE’s Project Manager Career Development Program beginning at the first level of certification. This will require specific training in the application of DOE-specific benchmarking procedures and performance measures. The training should be detailed enough to enable project directors to benchmark their projects as well as contribute to the development and improvement of the measures and process. The investment in training will also communicate the importance of benchmarking.
DATA COLLECTION AND VALIDATION
The committee believes that one of the first ways to make benchmarking meaningful and consistent is to ensure that the Earned Value Management System (EVMS) is being used consistently throughout the department. The DOE Project Analysis and Reporting System (PARS) should be expanded to become an integral part of the benchmarking data collection system. In order to serve this function PARS needs to be a robust, user-friendly, real-time, electronic system that accommodates the necessary data elements and produces the comparisons in tabular and graphic form. While the extant PARS may serve as a base for the benchmarking, it will need to be expanded before it can become a useful tool for benchmarking at all levels.
Any benchmarking system requires periodic evaluation to ensure that the basic definitions of the measures are understood, that the data being reported are consistent and accurate, and that the information
is being used to make corrections and improve performance. The generation and input of data as well as the validation of these data are inherently a function of the owner, in this case DOE. Owners may engage contractors to assist in the collection, validation, and analysis of the data, but the owner’s management team needs to be involved throughout the process.
For benchmarking to succeed, an organization needs to have the necessary technical, managerial, and cultural mechanisms in place (Keehly, 1997). A common set of performance measures needs to be developed that is consistent within the organization and understood by external benchmarking partners. The performance measures also need to be integrated into a common data collection system. As noted above, project directors need to be trained and a system for validating data needs to be established. The most difficult step is establishing an organizational culture that is ready to assess, compare, and analyze performance and to adopt best practices used by others when they are identified. This requires an organizational commitment to continuous improvement, acceptance of new ideas, and open communication and cooperation at all levels of the organization.
Organizational readiness for benchmarking can be developed incrementally, starting with a limited number of measures and internal benchmarking within a program, then expanding the effort to include more diverse measures and comparisons with other programs. The objective over time should be to develop a full set of measures and to benchmark both internally and externally, against organizations in other government agencies and private industry.
Keehly, Patricia, S.Medlin, S. MacBride, and L. Longmire, 1997. Benchmarking for Best Practices in the Public Sector. San Francisco, Calif.: Jossey-Bass.
NPR (National Partnership for Reinventing Government Reports, formerly the National Performance Review). 1997. Serving the American Public: Best Practices in Performance Measurement. Available online at http://govinfo.library.unt.edu/npr/library/papers/benchmrk/nprbook.html. Accessed March 14, 2005.
NRC (National Research Council). 2004. Progress in Improving Project Management at the Department of Energy, 2003 Assessment. Washington, D.C.: The National Academies Press.