National Academies Press: OpenBook
« Previous: Front Matter
Suggested Citation:"Summary." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Effectiveness and Efficiency of Defense Environmental Cleanup Activities of the Department of Energy's Office of Environmental Management: First Report. Washington, DC: The National Academies Press. doi: 10.17226/26000.
×
Page 1
Suggested Citation:"Summary." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Effectiveness and Efficiency of Defense Environmental Cleanup Activities of the Department of Energy's Office of Environmental Management: First Report. Washington, DC: The National Academies Press. doi: 10.17226/26000.
×
Page 2
Suggested Citation:"Summary." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Effectiveness and Efficiency of Defense Environmental Cleanup Activities of the Department of Energy's Office of Environmental Management: First Report. Washington, DC: The National Academies Press. doi: 10.17226/26000.
×
Page 3
Suggested Citation:"Summary." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Effectiveness and Efficiency of Defense Environmental Cleanup Activities of the Department of Energy's Office of Environmental Management: First Report. Washington, DC: The National Academies Press. doi: 10.17226/26000.
×
Page 4
Suggested Citation:"Summary." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Effectiveness and Efficiency of Defense Environmental Cleanup Activities of the Department of Energy's Office of Environmental Management: First Report. Washington, DC: The National Academies Press. doi: 10.17226/26000.
×
Page 5
Suggested Citation:"Summary." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Effectiveness and Efficiency of Defense Environmental Cleanup Activities of the Department of Energy's Office of Environmental Management: First Report. Washington, DC: The National Academies Press. doi: 10.17226/26000.
×
Page 6
Suggested Citation:"Summary." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Effectiveness and Efficiency of Defense Environmental Cleanup Activities of the Department of Energy's Office of Environmental Management: First Report. Washington, DC: The National Academies Press. doi: 10.17226/26000.
×
Page 7
Suggested Citation:"Summary." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Effectiveness and Efficiency of Defense Environmental Cleanup Activities of the Department of Energy's Office of Environmental Management: First Report. Washington, DC: The National Academies Press. doi: 10.17226/26000.
×
Page 8
Suggested Citation:"Summary." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Effectiveness and Efficiency of Defense Environmental Cleanup Activities of the Department of Energy's Office of Environmental Management: First Report. Washington, DC: The National Academies Press. doi: 10.17226/26000.
×
Page 9

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Summary The U.S. Department of Energy (DOE) and its predecessor agencies have conducted activities to develop atomic energy for civilian and defense purposes since the initiation of the World War II Manhattan Project in 1942. These activities took place at large federal land reservations of hundreds of square miles involving industrial-scale operations, but also at many smaller federal and non-federal sites such as uranium mines, materials processing and manufacturing facilities. At its peak, this nuclear complex encompassed 134 distinct sites in 31 states and one territory, with a total area of more than 2 million acres. The nuclear weapons and energy production activities at these facilities produced large quantities of radioactive and hazardous wastes and resulted in widespread groundwater and soil contamination at these sites. DOE initiated a concerted effort to clean up these sites beginning in the 1980s. Many of these sites have been remediated and are in long-term caretaker status, closed or repurposed for other uses. There are currently seventeen sites undergoing major cleanup and disposal activities. These activities are managed by the DOE Office of Environmental Management (EM), which, in fiscal year 2020, had budget authority of over $7 billion for cleanups and site services, performed by contractors. The present study is responsive to a request in the National Defense Authorization Act of Fiscal Year 2019 (P.L. 115-232) to conduct a review of the effectiveness and efficiency of the management of the various EM projects. Congress asked the National Academies of Sciences, Engineering, and Medicine to consider the following: (1) project management practices, (2) project outcomes, and (3) the appropriateness of the level of engagement and oversight by the DOE-EM organization. The committee entered into an agreement with EM that divided the work into two phases, with the first phase of study being focused on the execution of projects, the appropriateness and effectiveness of the controls and oversight applied to these projects, and the effectiveness with which these projects are realized through contracts.1 This summary provides background information on the sites currently assigned to EM undergoing cleanup; discusses current practices for management and oversight of the cleanups; offers findings and recommendations on such practices and how progress is measured against them; and considers the contracts under which the cleanups proceed and how these have been and can be structured to include incentives for improved cost and schedule performance. The seventeen sites (FIGURE S.1) currently in the EM portfolio include sixteen contaminated from civilian nuclear fuel cycle, naval propulsion, or nuclear weapons development activities. A seventeenth site in Carlsbad, New Mexico, carries out disposal operations. Eleven of these sites are co- located with currently operating DOE facilities; the other six are inactive other than for cleanup activities. 1 The second phase will address how EM manages and measures progress on cleanups both at the site level and those of programs that cut across more than one site (e.g., for Portsmouth and Paducah). The committee will also look at how these pieces are rolled-up into an EM-wide portfolio. The second phase will also consider how the policies and directives described by EM headquarters during the work on this first report are realized in projects at the sites. It will also consider further issues that obtain when considering the larger suite of EM activities, such as the cleanup and disposal liabilities ascribed to EM’s (currently 17) sites. PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION S-1

FIGURE S.1 Sites in Office of Environmental Management inventory as of 2020. SOURCE: Adapted from Todd Shrader, Principle Deputy Assistant Secretary (EM-2), Office of Environmental Management, Department of Energy, “EM Program History and Overview,” presentation to the committee, February 24, 2020. The EM contracting model has evolved over time to meet changing requirements. Initially, Management & Operating (M&O) contracts prevailed, embodying a unique relationship between the Government and contractor where the contractor was expected to apply its management expertise to implement the full suite of activities at a particular site within a general work scope established by the government. These were cost-type contracts with fees paid either on a fixed fee schedule or incentive basis. Later EM used cost-type contracts that had more specific work scope and performance based awards and fees.2 In 2000 DOE implemented two “closure contracts” at the Rocky Flats Plant in Colorado and the Fernald site in Ohio directed toward a defined end-state with large monetary incentives for the contractor to achieve that end-state in the most efficient and expeditious manner. During the study, DOE explained that in the future large contracts will be implemented under a new end-state contracting model (ESCM). The “End State” is defined as the specified situation, including accomplishment of completion criteria, for an environmental cleanup activity at the end of the Task Order period of performance. This end state concept will be implemented using single-award contracts of a certain type—Indefinite Delivery, Indefinite Quantity (IDIQ)—with a 10-year draw period during which task orders with very specific work scope and 5-years’ duration may be issued as either firm fixed price or cost reimbursable. Three IDIQs have been awarded to date, two at the Hanford site, and one at the Nevada National Security Site; a fourth ESCM IDIQ procurement process is underway at the Savannah River site. The EM project management and control systems also have evolved. Concerns regarding the effectiveness of and efficiency of project management, not only within EM but also department-wide, prompted a series of studies and actions beginning in the 1990s. These include several prior studies by the National Academies; a series of investigations and reports by the Government Accountability Office; and internal DOE-initiated and led reviews as well. These activities have led to the establishment and 2 Norbert Doyle, Deputy Assistant Secretary, Office of Acquisition & Project Management (EM-5.2), “Contract Overview,” presentation to the Committee, Washington, D.C., February 24, 2020. PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION S-2

updating of Department-wide program and project management guidelines, currently codified in DOE Order 413.3B, Program and Project Management for the Acquisition of Capital Assets: Change 5. The Order sets out procedures for project development and management, with the attendant controls and oversight, including review by the Energy Systems Acquisition Advisory Board (ESAAB); a hierarchy of project approval authority based on size of project; and on-going tracking of project performance via management information systems owned by a separate Office of Project Management (PM) within DOE. EM applies Order 413.3B to certain line-item construction projects, which, as of February 2020, numbered 14 projects and with an estimated $21.6 billion total project cost (TPC) in obligations (taking into account funding across multiple years) and comprised roughly one-quarter of EM’s annual (one-year) budget authority. The EM program has made substantial progress over the past several decades, primarily evidenced by the fact that it has reduced the footprint of contaminated sites from 134 to 17. EM also has been responsive to the various external and DOE internal management reviews by adopting a number of management improvements over time. EM, however, is currently at an inflection point, an appropriate time for further review and recalibration. Completion of cleanup activities at the 17 sites remaining in the EM portfolio will take many decades (and 11 of the EM sites are co-located with operating DOE facilities that will not be closing in the foreseeable future), and so project completions and site closures will no longer suffice as the principal program performance metric. Moreover, estimates of financial liability for cleanup of the remaining sites has outpaced the rate of expenditure for cleanup, with total cleanup liabilities currently estimated at over $400 billion, about 60 times the current annual EM budget. While EM has adopted many management reforms in recent years, challenges remain, with further opportunities to improve the effectiveness and efficiency of program performance. It is in this spirit that the committee has taken on this task. The committee met several times to hear testimony from the principals involved in the above- described efforts, supplemented this information with written queries to EM, and deliberated on its own. During public meetings, the committee heard presentations chiefly from EM but also from other elements of the department that oversee or execute large projects. The committee made roughly 60 written queries of DOE to gather further information. The committee read and considered prior and ongoing reviews of project management at DOE including those conducted by the department, the Government Accountability Office (GAO)—who also briefed the committee during the public meetings—and the National Academies. The committee’s findings and recommendations are included below along with the context for each as given in the chapters. All the recommendations appear in this summary. The committee observes that these recommendations will have more impact if implemented in a coordinated fashion rather than piecemeal, and we urge EM to strive for that coordination. PROJECT MANAGEMENT The committee assessment of EM project management proceeded on two tracks: assessing the extent to which Order 413.3B represents best practice for project management and assessing how EM applied Order 413.3B to its portfolio of projects. The committee compared the requirements and procedures of Order 413.3B with other leading international protocols for project management, including the Project Management Institute Best Practices, the Construction Industry Institute Best Practices and the U.K. Government Functional Standard GovS 002. The committee found that DOE Order 413.3B generally compared favorably with these other benchmarks but did identify several areas where the Order could be further enhanced. PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION S-3

For example, DOE interprets Order 413.3B such that it “applies ONLY to construction projects, major items of equipment (MIE’s) and (currently) environmental cleanup projects.”3 This interpretation appears relatively narrow compared to the Order’s stated purpose to implement “new requirements and leading practices for project and acquisition management” such as those derived from Office of Management and Budget (OMB) Circulars. In particular, OMB Circular A-11 in Appendix 1 of the supplement, Capital Programming Guide states, “Capital assets include the environmental remediation of land to make it useful.” EM however, based on the narrow interpretation of Order 413.3B, does not apply the Order to groundwater remediation projects that clearly have the intent of remediating land to make it useful. In addition, Order 413.3B is only applicable to projects with estimated total project cost (TPC) of $50 million or higher, which excludes a number of EM activities. EM-funded projects below $50 million TPC are not currently subject to the controls and oversights noted for the larger projects. EM has addressed this issue through a memorandum4 requiring that projects below the $50 million threshold follow the principles of project management outlined in Appendix C of Order 413.3B. The committee, however, has not found evidence of the tracking accorded such projects. In the future, even more EM cleanup projects could be excluded under the proposed end state contracting model where more of the work will be broken down into smaller task orders. Extending the applicability threshold down to $20 million would have the beneficial effect of: (1) applying a consistent set of principles across all such small projects; (2) invoking the controls and oversights of 413.3B such as ESAAB, (3) tracking by the Office of Project Management’s management information systems, and so forth. The small projects (between $20 million and $50 million TPC) could have a more streamlined application of Order 413.3B commensurate with their size and risk. Order 413.3B Section 3b requires the Contractor Requirements Document (CRD). The CRD is inserted into the Contract, and the list of requirements for the CRD are found in Attachment 1 of the Order. Application of Order 413.3B to specific projects or project types is best carried out through effective use of the project execution plans (PEPs), as successfully demonstrated in the National Nuclear Security Administration (NNSA), another division of DOE that makes substantially more outlays on contracting than the EM program. More generally, the committee found applying the requirements of Order 413.3B to be beneficial and that the Order compared well against recognized benchmarks. The majority of activities however do not fall under the Order, and such activities thus do not benefit from the full range of processes and requirements. The Demolition Protocol, for example, applies to projects in a certain category but does not include all such processes and requirements. The Demolition Protocol appears to exclude roles for the Project Management Risk Committee (PMRC) and the Energy Systems Acquisition Advisory Board (ESAAB), which would apply in Order 413.3B in certain instances. It also appears that certain independent reviews called for per Order 413.3B (e.g., Independent Project Reviews (IPRs) and External Independent Reviews (EIRs)), have been replaced with Independent Field Office and Headquarters Assessments. In addition to the issue of the $50 million threshold, the committee also reviewed several other major instances where Order 413.3 was not applied within the EM project portfolio. As noted earlier, EM applies Order 413.3B only to large line-item construction projects, which, as of February 2020, numbered 14 projects and totaled $21.6 billion total project cost (TPC) in obligations (taking into account funding across multiple years) and comprised roughly one-quarter of EM’s annual (one-year) budget authority. The remaining three quarters of them include activities to which EM is not applying Order 413.3B EM activities that are implemented outside the Order include site services, demolition of buildings, and waste disposal operations, as well as environmental remediation under $50 million noted earlier. The committee 3 Paul Bosco, Office of Project Management, DOE, “Project Management (PM) Governance, Systems and Training,” presentation to the Committee, May 6, 2020. 4 DOE (2018b) reiterates that “all projects equal to or less than $50 million shall follow the Project Management Principles as established in Appendix C of DOE Order 413.3B.” PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION S-4

found that the narrow interpretation of the applicability of Order 413.3B, relative to the OMB Capital Programming Guide for capital asset projects, is a major factor contributing to this situation. EM does not appear to meet the criteria that would otherwise exempt it from Order 413.3B. Finally, the committee reviewed the recent effort by EM to establish new project management process for demolition projects. DOE has large numbers of buildings and facilities that are no longer in use and require demolition. Within DOE, both the NNSA and EM have “ownership” of buildings to be demolished. NNSA currently implements demolition projects under the same guidelines and procedures as apply to new construction. EM, on the other hand, believes that its demolition activities, driven as they often are by regulatory requirements are not optimal for oversight under Order 413.3B guidelines and procedures. In July 2020, the Under Secretary (S3) approved and issued a demolition protocol “for operations-funded projects demolishing excess decontaminated facilities”; 5 such activities are not subject to 413.3B. The committee review did not find a strong rationale for EM demolition projects to be managed under different procedures than NNSA demolition projects, which do follow 413.3B. EM’s protocol for demolition projects states, “Disaggregation of site program work into smaller, discrete work activities is encouraged as it provides better project definition and clarity, is more manageable, reduces time horizons and risks, and is consistent with the project management best practices found in DOE O Order 413.3B.” A multiplicity of projects transfers a greater burden for program and project management to EM; increases responsibilities with respect to interface management; creates a growing level of risk in the “white space” between individual projects (i.e., omissions); partitions risks which were demonstrated to be best aggregated on both the Rocky Flats Plant and the Fernald site; and limits the scope for innovations in project delivery and the opportunity for accruing meaningful incentives by the contractor. The committee recommends that DOE confirm, clarify, and expand DOE Order 413.3B to establish its applicability to all capital asset projects (not just construction and Major Instruments and Equipment and certain cleanup projects) and all EM projects, whether major systems projects or work carried out by a Management and Operating (M&O) contractor. The committee makes the following specific recommendations regarding the Order as well: 1. Pending the outcome of the NNSA pilot project, reduce the threshold value for applicability of Order 413.3B from $50 million to $20 million; 2. Continue applying the requirements of Order 413.3B to M&O contract work on capital asset projects—the latter including construction projects, major items of equipment and cleanup projects; (Recommendation 4-1, first half) DOE should clarify Order 413.3B to incorporate best practices with respect to dispute prevention and resolution, which will be of growing significance as EM implements the “end-state” contracting approach. Sources for such best practices include the Construction Industry Institute. (Recommendation 4-2) EM should apply the requirements for Project Execution Plans equivalent to those in Order 413.3B to those projects that are not formally managed under Order 413.3B. (Recommendation 4-3) 5 Mark W. Menezes, Undersecretary of Energy, July 13, 2020, “Memorandum for Heads of Department Elements; SUBJECT: Demolition Projects.” PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION S-5

PROJECT MANAGEMENT METRICS EM, with the help of DOE’s Office of Project Management, has developed detailed processes and methods for tracking project-level outcomes and success measures. The committee reviewed five primary performance measurement approaches used by EM, including Earned Value Management System (EVMS), Project Dashboards, Project Evaluation and Measurement Plans (PEMPs), contract performance metrics and progress reports to Congress. After reviewing EM’s success measures and how they use them to guide decisions and report progress, the committee identified several issues for further action. In general, Earned Value Management System (EVMS) represents a principal system for an organization to monitor project management through an integrated set of work scopes, schedules and budgets. EVMS should provide a transparent and reliable process and approaches that explicitly, consistently, and clearly highlight the projects’ temporal status. Previous studies have advised EM to use such a system.6,7 A key element of EVMS is a Schedule Performance Index (SPI), defined by the Project Management Body of Knowledge as a metric that is used to measure schedule efficiency. EM currently includes a measure of SPI in its EVMS system that is based on dollars expended, not time. Because it is the key measure of schedule performance, it is important to calculate SPI based on time, not dollars, using the ratio of Scheduled Time of Work Performed (STWP) over Actual time of Work Performed (ATWP). The difference between calculating SPI using dollars versus time can be dramatic. For example, SPI based on dollars will not flag a project as behind schedule at the completion as long as the project completes within Budgeted Cost of Work Scheduled. Hence, a calculation based on cost does not always reliably convey possible schedule delays at the project completion and can lead to wrong conclusions about how successful it was. In contrast, SPI based on time will always reflect how delayed a project is regardless of the actual cost of the project. DOE EM should implement a modification to its EVMS system that captures the project’s temporal status more clearly and explicitly. Specifically, EM should immediately require that a revised Schedule Performance Index, SPI(t), which is the ratio of Scheduled Time of Work Performed (STWP) and Actual Time of Work Performed (ATWP), be reported to accurately track schedule performance. (Recommendation 5-2) EM’s portfolio of projects (work that is subject to following 413.3B) is approximately 25 percent of its annual budget. The percentage of actively tracked projects using certified EVM systems is even smaller (required for capital investment projects greater than $100 million). EM could similarly track a larger majority of activities, but does not. It should be recognized that EM project management issues are not unique; environmental cleanup at the Department of Defense (DoD) Base Realignment and Closure (BRAC) and Formerly Used Defense Sites (FUDS) provide similar examples of the challenges faced by EM (EPA, 2017). Both programs use a variety of contract forms, and the procurement processes vary to fit the project need. DoD manages them as decentralized projects and are closer in size and term (five to ten years) to EM’s new end state contracting model (ESCM). For BRAC, program management and program management oversight (PMO) are typically performed internally, e.g., Naval Facilities Engineering Systems Command (NAVFAC) for Department of Navy (DON) BRAC. The committee recommends that as EM increases its project management (PM) and Office of Project Management (OPM) responsibilities using Indefinite Delivery/Indefinite Delivery (IDIQ) contracts, it should share and compare best project management (PM) practices 6 Daniel B. Poneman, Deputy Secretary of Energy, September 9, 2011, “Secretarial Review of Environmental Management Programs and Projects.” 7 National Research Council. 2003. Progress in Improving PM in the DOE: 2003 Assessment. Washington, D.C.: National Academy Press. PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION S-6

with others across the U.S. government. To implement this, EM should form a “Joint Task Force” or less formal cooperative structure with Naval Facilities Engineering Systems Command (NAVFAC) and other Base Realignment and Closure (BRAC) and Formerly Used Defense Sites (FUDS) program management organizations. (Recommendation 5-1) The other key measure of project performance is cost management. The committee found two areas where information on cost management was not being reported with sufficient transparency. The first issue involves the calculation of cost performance for the EM project portfolio. Performance is based on number of projects as opposed to aggregate cost performance of the portfolio of projects considered. In the IDIQ approach, comprised of task orders, EM would disproportionately weigh many small projects towards their overall performance. Estimates at completion based on the cost-performance index are a floor to actual final cost given that program cost performance rarely improves as the program proceeds to its completion. The Department of Energy should 3. Clarify the definition related to project performance found at Section 3c(4), point 3 to calculate performance on aggregate value and not number of projects; and 4. Shift eligibility for project overruns, currently 10% per project, to be applied instead based on the aggregate value. (Recommendation 4-1, second half) A second area for improvement is the reporting of cost information in the project dashboards and project success reports. Currently, EM integrates all cost overruns into binary success metrics of Yes/No, which does not provide information on the magnitude of a cost overrun or underrun. Including the percentage (%) of cost over- or under-run, compared to the baseline (i.e., Original Critical Decision (CD)- 2 Total Project Cost (TPC)) in the Project Success metrics would provide more clarity. Some projects have significant cost overruns (and some of which EM has still not completed have more than doubled their original estimated cost) and others have lower cost overruns (reference Project Success Spreadsheet, 2020). There are also some projects that finished exactly at the estimated cost. DOE EM should explicitly include the percentage (%) of cost over (under) run in their Project Success metrics dashboard, rather than the current “green/yellow/red” metric to bring more transparency to cost performance. (Recommendation 5-3) CONTRACTING The committee reviewed the rationale for new end state contracting model (ESCM), reviewed the advantages and disadvantages of different types of contract types, and then compared the previous clean- up contract models for Fernald and Rocky Flats with the end-state contracting model in use today by EM. EM has focused increased attention on the need to be on a trajectory toward end-states. Creating and motivating a culture of completion is important to EM’s mission success. In its own management analysis, EM has identified important ongoing efforts including “Defining requirements in measurable outcomes” and “Using objective performance measures focusing on outcomes to balance considerations of cost control, schedule achievement, and technical performance”. The committee concurs with the imperative of outcomes-based completion contracting and agrees with the need to build on past, successful initiatives such as Rocky Flats and Fernald completion contracts. EM has advanced the ESCM as a new and improved vehicle for achieving outcomes-based completion contracting. The committee has carefully reviewed the ESCM model and compared it to the attributes of the completion contracts successfully deployed at the Fernald and Rocky Flats site. The PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION S-7

committee found that many of the features of the completion contracts that made Fernald and Rocky Flats successful are not present in the current ESCM. In short, the committee found that the ESCM is neither outcomes-based nor completion-focused. Rather, ESCM is focused on delivery of a set of discrete outputs that are not clearly mapped by contract to achievement of either a clearly defined intermediate or final end-state. This significant deficiency deprives EM and the IDIQ contractor of the benefits of having a completion-oriented contract fully integrated throughout the supply chain and the fostering of innovation at the scale the program requires. Finally, the ESCM approach, as defined, focuses on narrowly defined performance criteria and increases risks associated with incomplete statements of work. These concerns and deficiencies were largely successfully addressed in Rocky Flats and Fernald. Under the ESCM, EM awards a single IDIQ8 contract for a draw period of up to 10 years. Within the IDIQ contract umbrella, EM will establish a series of smaller, shorter-term task orders within the IDIQ umbrella, using a combination of Firm Fixed Price (FFP) and Cost Reimbursement (CR) Task Orders.9 DOE sees the benefits of this end state concept to include: quicker evaluations of proposals; less risk of protest loss; freeing up of contractor key personnel; and less proposal cost to industry. The committee notes that the IDIQ task order structure will create a significant number of task orders, triggering a pro rata increase in the project management burden to EM. The anticipated size of the task orders in the IDIQ clean-up contracts, averaging about $100 million, will result in EM having to manage potentially 100 task orders over the life of one clean-up contract. This process carries greater risk for EM, requiring the possible management of an unwieldy number of task orders and a significant amount of DOE oversight. Discrete task orders also could limit benefits that might come from contractor innovation that contributed to the success at Fernald and Rocky Flats. Breaking up the scope of work into a large number of discrete tasks will diminish the focus on project outcomes and overall project optimization within an outcomes-based framework. The committee believes that the current contract procurement process can be adapted by awarding larger task orders that define one or more intermediate end states, thereby reducing residual risk to EM. Larger task orders could increase the opportunity for contractor innovation and provide for focused oversight at a higher level within EM. DOE’s reliance on “discrete tangible progress” through individual task orders under an IDIQ contract, without identifying an overall strategy or program management plan is not, in the committee’s view, outcomes-based contracting. The ESCM concept does not define what “end-states” (or reasonable subsets thereof) truly are. The committee supports true “end-state” or outcomes-type contracts but has not seen the requisite strong links between the management of portfolio, program, and projects that are a core element of moving toward “end state” completion. Finally, the committee notes that the ESCM single award IDIQ contracts may not achieve the desired streamlining in the procurement process. The protest of the Hanford central plateau End State contract, which ultimately was decided in DOE’s favor, necessitated significant and lengthy corrective actions. Contractor key personnel had to be maintained from date of award (December 2019) until notice to proceed in mid-September 2020. EM should establish well-defined, outcomes-based intermediate end-states in its ten-year clean-up contracts. Any intermediate outcomes should have clear, measurable metrics to assess site-based (versus task-based) achievement of the defined end-states. EM should report progress on these metrics across the portfolio of end-state programs on a quarterly basis and such reports should represent a key EM performance measure. (Recommendation 6-1) 8 Norbert Doyle, Deputy Assistant Secretary, Office of Acquisition & Project Management (EM-5.2), “Contract Overview,” presentation to the Committee, Washington, D.C., February 24, 2020. 9 Ibid. PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION S-8

EM should structure task orders on a scale that is appropriate for defining intermediate outcomes, award fewer individual tasks. EM should apply to such task orders the same management oversight as currently required for a Major Systems Projects (MSP) exceeding the $750 million in total cost. (Recommendation 6-2) CONTRACT EXECUTION: FEES AND INCENTIVES EM seeks to obtain the maximum return from its contractors by offering a balanced mix of integrated, fair, and challenging incentives. This requires that contractor fees be directly tied to contractor performance. In establishing appropriate incentives for contractors, fees should be reasonable, reflecting effort (noting the complexity of the work and the resources required for contract completion), cost risk (the cost responsibility and associated risk the contractor assumes under the contract type and the reliability of the cost estimates in relation to the complexity of the task), and other factors (for example, support of federal socioeconomic programs, investment in capital, and independent development). EM contracts typically provide a two-part fee structure consisting of a base fee and a performance fee. The performance fee generally includes both objective and subjective fee components and must relate to clearly defined performance objectives and measures. Where feasible, these objectives and measures should be expressed as desired results or outcomes. As noted by GAO10 there appear to be few guidelines to distinguish between “objective” and “subjective” award fee criteria. Using subjective fee components is less desirable than using objective fee components because the link between performance and reward is less clear. Only when it is not feasible to use objective measures of performance should subjective fee components be used. For example, although it might be feasible, it is difficult to specify performance metrics for “Environmental Stewardship and Compliance.” If there are well-specified subjective criteria, they should be tied to identifiable interim outcomes, discrete events, or milestones. The committee examined subjective and objective performance assessment summaries and resulting fees as presented in “Scorecards” posted on applicable DOE field office websites; in particular, for contracts awarded at the Hanford site. After reviewing the evaluation of performance with Hanford cleanup contracts, DOE EM’s rating of contractor performance does not appear to be consistent through years for a specific contract or across contracts in a specific year. Performance ratings sometimes appear to not correspond to comments by the contract evaluator. To increase transparency in contractor performance evaluation the committee recommends that DOE-EM should ensure that the contracts it issues for cleanup work (1) create a consolidated set of unambiguous “subjective” criteria for similar types of cleanup activities, and (2) use these criteria in the evaluation of all contract performance across its portfolio. (Recommendation 7-1) 10 DOE “has a different process for determining incentive and award fees, depending on whether the fee is tied to objective or subjective performance criteria.” In Government Accountability Office (GAO). 2019. Department of Energy Performance Evaluations Could Better Assess Management and Operating Costs, GAO-19-5. Washington, D.C.: GAO. PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION S-9

Next: 1 Introduction »
Review of Effectiveness and Efficiency of Defense Environmental Cleanup Activities of the Department of Energy's Office of Environmental Management: First Report Get This Book
×
Buy Paperback | $50.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The U.S. Department of Energy (DOE) and its predecessor agencies have conducted activities to develop atomic energy for civilian and defense purposes since the initiation of the World War II Manhattan Project in 1942. These activities took place at large federal land reservations of hundreds of square miles involving industrial-scale operations, but also at many smaller federal and non-federal sites such as uranium mines, materials processing and manufacturing facilities. The nuclear weapons and energy production activities at these facilities produced large quantities of radioactive and hazardous wastes and resulted in widespread groundwater and soil contamination at these sites. DOE initiated a concerted effort to clean up these sites beginning in the 1980s. Many of these sites have been remediated and are in long-term caretaker status, closed or repurposed for other uses.

Review of the Effectiveness and Efficiency of Defense Environmental Cleanup Activities of the Department of Energy's Office of Environmental Management: First Report provides background information on the sites currently assigned to the DOE's Office of Environmental Management that are undergoing cleanup; discusses current practices for management and oversight of the cleanups; offers findings and recommendations on such practices and how progress is measured against them; and considers the contracts under which the cleanups proceed and how these have been and can be structured to include incentives for improved cost and schedule performance.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!