National Academies Press: OpenBook
« Previous: 3 Contracting and Project Management in the Office of Environmental Management
Suggested Citation:"4 Project Management Policies, Processes, and Procedures." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Effectiveness and Efficiency of Defense Environmental Cleanup Activities of the Department of Energy's Office of Environmental Management: First Report. Washington, DC: The National Academies Press. doi: 10.17226/26000.
×
Page 30
Suggested Citation:"4 Project Management Policies, Processes, and Procedures." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Effectiveness and Efficiency of Defense Environmental Cleanup Activities of the Department of Energy's Office of Environmental Management: First Report. Washington, DC: The National Academies Press. doi: 10.17226/26000.
×
Page 31
Suggested Citation:"4 Project Management Policies, Processes, and Procedures." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Effectiveness and Efficiency of Defense Environmental Cleanup Activities of the Department of Energy's Office of Environmental Management: First Report. Washington, DC: The National Academies Press. doi: 10.17226/26000.
×
Page 32
Suggested Citation:"4 Project Management Policies, Processes, and Procedures." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Effectiveness and Efficiency of Defense Environmental Cleanup Activities of the Department of Energy's Office of Environmental Management: First Report. Washington, DC: The National Academies Press. doi: 10.17226/26000.
×
Page 33
Suggested Citation:"4 Project Management Policies, Processes, and Procedures." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Effectiveness and Efficiency of Defense Environmental Cleanup Activities of the Department of Energy's Office of Environmental Management: First Report. Washington, DC: The National Academies Press. doi: 10.17226/26000.
×
Page 34
Suggested Citation:"4 Project Management Policies, Processes, and Procedures." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Effectiveness and Efficiency of Defense Environmental Cleanup Activities of the Department of Energy's Office of Environmental Management: First Report. Washington, DC: The National Academies Press. doi: 10.17226/26000.
×
Page 35
Suggested Citation:"4 Project Management Policies, Processes, and Procedures." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Effectiveness and Efficiency of Defense Environmental Cleanup Activities of the Department of Energy's Office of Environmental Management: First Report. Washington, DC: The National Academies Press. doi: 10.17226/26000.
×
Page 36
Suggested Citation:"4 Project Management Policies, Processes, and Procedures." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Effectiveness and Efficiency of Defense Environmental Cleanup Activities of the Department of Energy's Office of Environmental Management: First Report. Washington, DC: The National Academies Press. doi: 10.17226/26000.
×
Page 37
Suggested Citation:"4 Project Management Policies, Processes, and Procedures." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Effectiveness and Efficiency of Defense Environmental Cleanup Activities of the Department of Energy's Office of Environmental Management: First Report. Washington, DC: The National Academies Press. doi: 10.17226/26000.
×
Page 38
Suggested Citation:"4 Project Management Policies, Processes, and Procedures." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Effectiveness and Efficiency of Defense Environmental Cleanup Activities of the Department of Energy's Office of Environmental Management: First Report. Washington, DC: The National Academies Press. doi: 10.17226/26000.
×
Page 39
Suggested Citation:"4 Project Management Policies, Processes, and Procedures." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Effectiveness and Efficiency of Defense Environmental Cleanup Activities of the Department of Energy's Office of Environmental Management: First Report. Washington, DC: The National Academies Press. doi: 10.17226/26000.
×
Page 40
Suggested Citation:"4 Project Management Policies, Processes, and Procedures." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Effectiveness and Efficiency of Defense Environmental Cleanup Activities of the Department of Energy's Office of Environmental Management: First Report. Washington, DC: The National Academies Press. doi: 10.17226/26000.
×
Page 41
Suggested Citation:"4 Project Management Policies, Processes, and Procedures." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Effectiveness and Efficiency of Defense Environmental Cleanup Activities of the Department of Energy's Office of Environmental Management: First Report. Washington, DC: The National Academies Press. doi: 10.17226/26000.
×
Page 42
Suggested Citation:"4 Project Management Policies, Processes, and Procedures." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Effectiveness and Efficiency of Defense Environmental Cleanup Activities of the Department of Energy's Office of Environmental Management: First Report. Washington, DC: The National Academies Press. doi: 10.17226/26000.
×
Page 43
Suggested Citation:"4 Project Management Policies, Processes, and Procedures." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Effectiveness and Efficiency of Defense Environmental Cleanup Activities of the Department of Energy's Office of Environmental Management: First Report. Washington, DC: The National Academies Press. doi: 10.17226/26000.
×
Page 44
Suggested Citation:"4 Project Management Policies, Processes, and Procedures." National Academies of Sciences, Engineering, and Medicine. 2021. Review of Effectiveness and Efficiency of Defense Environmental Cleanup Activities of the Department of Energy's Office of Environmental Management: First Report. Washington, DC: The National Academies Press. doi: 10.17226/26000.
×
Page 45

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

4 Project Management Policies, Processes, and Procedures DESCRIPTION OF CURRENT EM PROGRAM AND PROJECT STRUCTURES AND TYPES This chapter describes and assesses DOE Order 413.3B, Program and Project Management for the Acquisition of Capital Assets, (hereafter “Order 413.3B”) (DOE, 2018a). The chapter assesses Order 413.3B by comparing it against recognized industry standards and looking at its current application within the Office of Environmental Management’s (EM’s) cleanup program. The committee deferred the review of program management to the second phase of work and has emphasized project management for this first phase. The two levels of management—program and project—are linked, with one enabling the success of the other. The chapter provides specific findings regarding Order 413.3B compared to “best practice” status for significant elements of Order 413.3B together with actionable recommendations. In meetings with the committee, EM outlined an approach to end-state contracting, discussed in greater detail in Chapter 3. This approach will utilize Indefinite Delivery, Indefinite Quantity (IDIQ) contracts with 5-year task orders issued over a 10-year draw period. Under this concept, EM plans to utilize such contracts but with a single award.47 In one sense, DOE-EM does not thoroughly define successful outcomes or “end-states.” The use of IDIQ, typically employed when an agency has not defined the work except in broad terms, underscores this. Rather, ESCM is focused on delivery of a set of discrete outputs that are not clearly mapped by contract to achievement of either a clearly defined intermediate or final end state. This significant deficiency deprives EM and the IDIQ contractor of the benefits of having a completion-oriented contract fully integrated throughout the supply chain and the fostering of innovation at the scale the program requires. Finally, the ESCM approach, as defined, focuses on narrowly defined performance criteria and increases risks associated with incomplete statements of work. These concerns and deficiencies were largely successfully addressed in Rocky Flats and Fernald. EM appears to be at an inflection point where outcomes-based contracting for an entire site is not feasible given the scale of the challenges at the individual site. However, the committee believes that subgroupings of cleanup activities exist which lend themselves to end state approaches similar to what was achieved on a site-wide basis at Rocky Flats and Fernald. This chapter considers the lessons learned from these two sites and their impact on DOE’s path forward and choice of the IDIQ task order approach and the degree to which this approach can be linked with meaningful end-states. OMB Circular A-11 and PMIAA Applicability This section discusses three areas where additional clarity and expansion would be beneficial, particularly in applying Order 413.3B to the Environmental Management (EM) cleanup program. (The section following this, Assessment of Order 413.3B compared to other project management standards, discusses Order 413.3B vis-à-vis best practices for project management.) These include the following: 47 “Responses to NAS Questions” sent to NASEM staff by Rodney Lehman, EM-5.22, June 30, 2020. PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION 4-1

 Applicability of Order 413.3B: There is an inconsistency between the statements by DOE and what is stated in Order 413.3B itself. The Office of Project Management states that Order 413.3B “Applies ONLY to construction projects, major items of equipment (MIE’s) and (currently) environmental cleanup projects – Over $50 (M) Million Dollars.”48 In EM currently, the active (i.e., Critical Decision 3 [CD-3]) construction projects include 14 Projects with a combined $21.6B Total Project Cost (TPC). The “recorded” cleanup projects include four projects with combined TPC of $717 million.49 DOE Order 413.3B itself, however, describes its purpose as being: To provide the Department of Energy (DOE) Elements, including the National Nuclear Security Administration (NNSA), with program and project management direction for the acquisition of capital assets with the goal of delivering projects within the original performance baseline (PB), cost and schedule, and fully capable of meeting mission performance, safeguards and security, and environmental, safety, and health requirements unless impacted by a directed change. To implement Office of Management and Budget (OMB) Circulars to include: A-11, and its supplement, Capital Programming Guide, which prescribes new requirements and leading practices for project and acquisition management . . . (DOE, 2018a, p. 1) To a literal reading of the above, Order 413.3B would apply to all projects meeting the definition in Appendix 1 of Capital Programming Guide V 3.0: Supplement to Office of Management and Budget Circular A–11, Preparation, Submission and Execution of the Budget concerning capital assets (OMB, 2016).  Overly narrow interpretation of OMB Circular A-11, Appendix 1 - Capital Programming Guide: The Capital Programming Guide of A-11, Appendix 1 notes that “capital assets include the environmental remediation of land to make it useful…” It continues “Examples of capital assets include the following, but are not limited to them…. Environmental restoration (decontamination and decommissioning efforts).” Further it states “The cost of a capital asset is its full life-cycle costs, including all direct and indirect costs for planning, procurement (purchase price and all other costs incurred to bring it to a form and location suitable for its intended use), operations and maintenance (including service contracts), and disposal (underline added).” Additional clarity in A-11 Appendix 1 Capital Programming Guide and DOE Order 413.3B, to clearly establish that all environmental restoration that derives from the prior construction and operation of a capital asset is to be included in the definition of a capital asset as provided for in OMB Circular A-11, would be beneficial. A third area may be added to the above observations on applicability, one which the committee will further examine in the Phase 2 report when discussing the links from portfolio-to-program-to-project, as follows:  Portfolio and program management - The Program Management Improvement Accountability Act (PMIAA) requires portfolio reviews50 as part of the agency’s annual strategic review process. 48 Paul Bosco, Office of Project Management, DOE, “Project Management (PM) Governance, Systems and Training,” presentation to the Committee, May 6, 2020. 49 Rodney Lehman, Director, EM Office of Project Management (EM-5.22), “Overview of DOE O 413.3B and EM Project Management Protocol for Demolition Projects,” presentation to the Committee, February 24, 2020, Washington, D.C. 50 OMB defines program as the functions or activities which agencies are authorized and funded by statute to administer and enforce. Programs typically involve broad objectives. OMB views projects as temporary efforts with defined scopes to create products or services to improve the efficient and effective implementation of programs. PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION 4-2

PMIAA established a new position, the Program Management Improvement Officer (PMIO) responsible for implementing program management policies established under respective agencies (DOE). Currently, Order 413.3B represents itself as encompassing program management.   FINDING: The applicability of Order 413.3B is to “capital asset projects” which DOE describes as being construction projects, major items of equipment (MIE’s) and certain environmental cleanup projects. Projects less than $50 million total project cost are exempted. Order 413.3B applies to a small subset of EM’s budget, chiefly to construction projects and to “recorded” cleanup projects. Demolition Protocol The Department released a new Demolition Protocol on July 13, 2020. This is expressly for the “demolition of excess decontaminated buildings.”51 Formally called, Office of Environmental Management Cleanup Project Management Protocol and Implementation Standard for Demolition Projects, the Protocol has the stated purpose “to establish tailored project management requirements that are applicable to Office of Environmental Management (EM) demolition projects and consistent with Department of Energy (DOE) Order (O) 413.3B.” (DOE, 2020, p. 1). EM further elaborated that the Demolition Protocol is suited for activities that take place in a regulatory framework that governs the cleanup. It employs the same terminology as Order 413.3B.52 EM described the new Protocol in comparison to Order 413.3B in terms of the Critical Decision points in a high-level process map comparison.53 The largest differences are found at CD-0/CD-1, in which the Protocol requires a memorandum describing mission need and the framework under which the project will proceed. At those same CD points, Order 413.3B has a number of reviews and documentation steps. As of March 2020, EM intended to include five cleanup projects under the Demolition Protocol that were previously under Order 413.3B. The Total Project Cost of these was $1.084 billion. The sections of the report that follow include discuss Demolition Protocol in terms of its applicability and its relation to other DOE requirements. FINDING: EM has created a new Demolition Protocol that applies to selected cleanup projects that were previously proceeding under Order 413.3B. DOE’s objective in creating this Protocol is to provide a set of requirements more suited for activities subject to regulatory frameworks. The Protocol is streamlined compared to Order 413.3B particularly at the CD-0 and CD-1 stages. Because programs are comprised of projects, programs inherently address the projects subsumed within them. Finally, OMB defines portfolios as organized groupings of programs whose coordination in implementation enables agencies to achieve their objectives. (Office of Management and Budget, Improving the Management of Federal Programs and Projects through Implementing the Program Management Improvement Accountability Act (PMIAA), OMB Memorandum M-18-19 (Washington, D.C.: June 25, 2018).) 51 Mark W. Menezes, Undersecretary of Energy, July 13, 2020, “Memorandum for Heads of Department Elements; SUBJECT: Demolition Projects.” 52 Rodney Lehman, Director, EM Office of Project Management (EM-5.22), “Overview of DOE O 413.3B and EM Project Management Protocol for Demolition Projects,” presentation to the Committee, February 24, 2020, Washington, D.C. 53 Rodney Lehman, Director, EM Office of Project Management (EM-5.22), “Overview of DOE O 413.3B and EM Project Management Protocol for Demolition Projects,” presentation to the Committee, February 24, 2020, Washington, D.C. PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION 4-3

ASSESSMENT OF ORDER 413.3B COMPARED TO OTHER PROJECT MANAGEMENT STANDARDS The committee assessed DOE Order 413.3B against three reference systems for program and project management. These included:  Project Management Institute (PMI) Best Practices (9 Elements)54  Construction Industry Institute (CII) Best Practices55  UK Government Functional Standard GovS 002: Project delivery – portfolio, programme and project management56    The three program and project management systems selected for benchmarking of DOE EM’s project and program management practices include the Project Management Institute’s Best Practices which encompasses nine elements of success. Founded in 1969, the Project Management Institute (PMI) is a professional extension of the project management trend that emerged from the 1960’s explosion of project management in the defense industries. Today, PMI has over 600,000 global members. PMI best practices were also considered in GAO’s report entitled, “NUCLEAR WASTE CLEANUP: DOE Could Improve Program and Project Management by Better Classifying Work and Following Leading Practices” (GAO, 2019). Also considered in benchmarking was the Construction Industry Institute’s (CII) Best Practices. CII is a consortium of more than 140 leading owner, engineering-contractor, and supplier firms from both the public and private arenas and includes the U.S. Department of Energy among its members. The final system considered in benchmarking is UK Government Functional Standard GovS 002: “Project delivery – portfolio, programme and project management” (July 2017). This standard looks across portfolios, programs and projects and was specifically development for government project use. Its program management standards will serve as a benchmarking basis for this committee’s second report focused more on programs. In this section and in Table 4.1, the committee will assess the extent to which Order 413.3B represents best practice for project management and will note areas for improvement for portfolio and program management, with “portfolio” not indicated as a coverage area for Order 413.3B (program and project). Table 4.1 is organized by major topical areas that cut across the various benchmarking standards. Within the topical areas the project lifecycle as described in DOE Order 413.3B is used as an organizing principle for comparison.  54 See, for example: Salapatas, J. N. (2000). Best practices—the nine elements to success. Paper presented at Project Management Institute Annual Seminars & Symposium, Houston, TX. Newtown Square, PA: Project Management Institute. 55 Further information available at < https://www.construction-institute.org/resources/knowledgebase/best- practices>. 56 Available at <https://www.gov.uk/government/publications/project-delivery-functional-standard>. PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION 4-4

Project Management TABLE 4.1 Evaluation of Order 413.3Ba versus Three Benchmarking Standards: Project Management Institute (referred to as “PMI” in the Table); Construction Industry Institute (“CII”); and UK Government Functional Standard GovS 002: Project Delivery – Portfolio, Programme and Project Management (“GovS 002”). Topic Assessment Growth Areas Governance Frameworks  Overall Governance framework for projects  Areas requiring further attention include portfolio and program outlined in Order 413.3B is largely consistent management, which DOE does not address in Order 413.3B; CII with GovS 002 Best Practices on Alignment and Partnering are not addressed in Order 413.3B; assurance frameworks of Order 413.3B do not address the role/value of internal/project audit; Order 413.3B does not highlight a data/information-centric approach essential for managing a program of the scale of EM’s cleanup; coverage of configuration management requires improvement (the parallel approach to CD-1,2,3 in EM was judged to be reasonable given the nature of the program and the required environmental and regulatory processes it is subject to). Critical Decisions (CD)  The critical decision process laid out in  The parallel execution of CD 1, 2, and 3 in EM is not consistent Order 413.3B provides for the phases, with PMI Best Practices (1) and (5)iii but the committee has deliverables, key milestones and sufficiency judged it to be reasonable. This is now codified in the Demolition criteria envisioned in Best Practice (1) of Protocolb which the committee regards as a Project Execution Plan PMI (PEP) within the context of Order 413.3B and consistent with the PEP approach used by NNSA. The appropriate use of PEPs for EM projects could address this concern across all EM project types. See DOE G 413.3-15,c Project Execution Plans.d  DOE construction management plans and processes are not well developed in Order 413.3B or associated guides.  Approvals – Order 413.3B demonstrates PMI Best Practice (9) Work Authorization and Change Control  Cost estimates –This will be looked at in  Undersecretary Mark Menezes July 13, 2020 memorandum on conjunction with the second report Demolition Projects states in section IV that: “Disaggregation of site program work into smaller discrete work activities is encouraged as it provides better project PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION 4-5

Topic Assessment Growth Areas definition and clarity, is more manageable, reduces time horizons and risks, and is consistent with the project management best practices found in DOE O Order 413.3B.”e While improved project definition and clarity is a desired outcome, what is gained in definition and clarity for small individual tasks may come at the expense of delivering on EM’s overall cleanup objectives for a particular site and EM’s broader mission. The disaggregation of work introduces risks between each of the projects comprising the program and has the effect of limiting opportunities (negative risks) while increasing complexity from a program perspective.  Management of multiple task order projects requires multiple scope development activities and negotiations. Scope development must be clearly linked to overall site outcomes in a manner that assures that missing scope is not just picked up in a subsequent task. Scope completeness responsibilities weigh much heavier on DOE than the contractor.  Segregation of resources, funds and accounting by task requires DOE to assure that funds paid for one task are not being applied to another task by a sole source contractor. This is important when incentives are task based versus overall outcome based and when both cost-reimbursable and fixed price work are being carried out simultaneously.  The committee is concerned that completion schedules may be extended as more tasks introduce more schedule precedences into the overall program. As expressed in this report the absence of strong schedule performance indexes underscores this concern and time is clearly a significant cost adder.  Final program cost is closer to the estimates at completion based on the schedule-cost index (product of the cost-performance index and the schedule-performance index), which suggests a ceiling to the final cost. Estimates at completion based on the cost- performance index are a floor to actual final cost given that program cost performance rarely improves as the program proceeds to its completion. PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION 4-6

Topic Assessment Growth Areas  The committee in its next report will look at DOE’s ability to deliver the cleanup mission utilizing its current approach to project and program delivery and likely levels of funding. At current levels of funding overall EM cleanup is extended by 15 years if escalation in cleanup costs exceeds general inflation, a surrogate for growth in the federal budget, by 1%. With a 2% differential similar to that anticipated in nuclear plant decommissioning, current funding levels do not support the cleanup mission being completed. The committee will examine whether the task order approach further exacerbates this situation.  Finally, with respect to risk reduction the committee is concerned that the focus on tasks can lead to disaggregation at the expense of overall portfolio risk and optimization. The committee views high end risks as outweighing low end uncertainties resulting in the sum of the likely values of the individual tasks being significantly less than likely program and portfolio costs. We believe that a high degree of statistical correlation exists between the various tasks in a program and to a degree the overall EM portfolio.  Total program cost grows with both the level of correlation between projects and the number of projects. Correlation in a program or portfolio is driven by common resources, project execution methods, management practices, common regulatory drivers and outcomes and schedule (precedence) interdependencies. These are all present. CD-0  Establishment of requirements is consistent  Areas requiring further attention include: with PMI Best Practice (2)  Expanded Basis of Design established at CD-0, addressing  Program requirements document defining technical, construction and O&M considerations ultimate goals project must satisfy, currently used by NNSA, would improve EM linkage  DOE should require project risk review by PMRC for all projects between program and project aiding the >$100 million at this stage, considering their involvement at CD – approach to “end-state” contracting. 1. CD-1  System engineering methods and other  The combined CD-1, 2, and 3 approach by EM raises concerns on requirements are consistent with PMI Best the timing of completion of risk identification and analysis Practices (4), (5)iv, (5)vi. PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION 4-7

Topic Assessment Growth Areas  Areas that require further attention include ensuring that the safety design strategy requirements ensures that non-hazardous and nuclear safety through design principles and practices extend to all projects CD-2  Order 413.3B meets PMI Best Practices (2)  EM needs to provide better evidence of assessing and controlling and (9). In EM, the stated PMI requirements schedule baselines to demonstrate meeting PMI Best Practice (6). are not met until CD-3 given the combined approach to CD-1,2,3.  EM should fully establish key performance indicators (KPIs) for assessing project performance at this stage.  The covered scope includes both scope of facilities and scope of services.  Requirements related to preliminary / final design need to clarify the facility lifetime EM is considering in sustainability  Configuration control for facilities not covered by the defined Hazard Categories (HC-1,2,3) is required  EM needs to clarify how they accumulate life-cycle costs from the Value Management process into the broader EM program and portfolio. CD-3  EM does not clearly identify detailed submissions and construction planning documents for external independent reviews to confirm construction and execution readiness (corresponds to CII Best Practices). CD-4  EM should emphasize the importance of validation and  Order 413.3B meets PMI Best Practice verification in complex programs and recognize that this emphasis (5)vii. cascades into projects comprising the program.  EM must begin start-up (shutdown) and commissioning (decommissioning) planning at a much earlier stage than outlined in DOE Order 413.3B Control Documents  DOE Order 413.3B meets PMI Best Practice  Benchmarking at CD-0, 1, and 2 across EM projects is not (5)v; (5)vi; (9) apparent and not addressed in Order 413.3B.  It appears EM partially addresses PMI Best  Properly implementing Order 413.3B requires incorporating more Practice (8) Escalation and Issue leading indicator type metrics and predictive performance tools. Management through the quarterly project  EM would benefit from more frequent deep-dive project reviews reviews, but effectiveness and timeliness with greater project coverage. requires further assessment. If EM defines the Energy Systems Acquisition Advisory PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION 4-8

Topic Assessment Growth Areas Board (ESAAB) meetings as meeting these  EM should thoroughly reconcile Final Project Data Sheet and PMI Best Practices, they appear not to have funding documents with performance baseline and systemic the level of granularity that effective project lessons learned identified. reviews require.  EM should consider final peer reviews for the lessons learned stage. DOE Order 413.3B could be improved to require implementing DOE Order 210.2A related to reviewing, vetting, and sharing lessons learned. EM should distribute lessons learned to all DOE Federal Project Directors (FPD). Project life-cycle control  Concerning organization roles and  Changes to the baseline schedule would be better controlled if responsibilities, Order 413.3B meets PMI they required Chief Executive for Project Management approval. Best Practice (3) and NNSA has Currently, only technical and cost require approval. implemented its guidance. The committee  EM should define change control boards consistently across the has not yet reviewed EM’s systems. Given PEPs. This is especially significant given the new Demolition the nature of the EM mission it might be Protocol.a expected the responsibilities of the Office of Project Management (EM 5.22) within EM’s  Order 413.3B does not address roles and responsibilities with office of Corporate Services would fall at a respect to portfolio and program, and these should be considered higher organizational level. in light of the comment in the previous column on EM 5.22. NOTES: a Order 413.3B refers to U.S. Department of Energy (DOE), 2018, Program and Project Management for the Acquisition of Capital Assets: Change 5. DOE O 413.3B: Change 5. Washington, D.C., April 12. b DOE Office of Environmental Management, 2020, Office of Environmental Management Cleanup Project Management Protocol and Implementation Standard for Demolition Projects, EM Protocol, Final June 8, 2020, Washington, D.C. c DOE G 413.3-15 refers to DOE Office of Project Management, 2018, Project Execution Plans: DOE G 413.3-15A, 9-14-2018, Washington, D.C. d The Project Execution Plan, described in DOE G 413.3 and referenced throughout Order 413.3B, is interwoven with other requirements beginning at the CD-1 stage, including the Tailoring Strategy (Order 413.3B, p. A-6), the Integrated Project Team (p. A-6), the Risk Management Plan (p. A-7), the Funding Profile (p. A-9) and so forth. e Mark W. Menezes, Undersecretary of Energy, “Memorandum for Heads of Department Elements; SUBJECT: Demolition Projects,” July 13, 2020. . PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION 4-9

FINDING: DOE Order 413.3B generally compared favorably with benchmarks for project management practices including those of the Project Management Institute (PMI), the Construction Industry Institute (CII) and the UK Government. FINDING: DOE Order 413.3B does not incorporate Construction Industry Institute Best Practices for Advanced Work Packaging, Materials management, Planning for Modularization, or Disputes Prevention & Resolution.   Program Management  Portfolio and program not adequately addressed in Order 413.3B  o Order 413.3B does not address the challenges, opportunities and processes that affect project to project interfaces and efficient “end-state” oriented program delivery  o Order 413.3B does not address the risks associated with the totality of a program and more broadly the EM program   Order 413.3B does not specifically address commercial and financial considerations, which are more typically found at the portfolio and program levels from a management perspective. Other governance documents will become important in evaluating the proposed use of “end-state” contracts using IDIQ’s  Conclusion CONCLUSION: Overall, DOE Order 413.3B represents best practice for project management, but there are opportunities for improvement for portfolio and program management, for example, by expanding its applicability to include “portfolio,” whereas currently the Order covers program and project.  CURRENT COVERAGE OF ORDER 413.3B The scope and applicability of Order 413.3B to portfolios and programs is discussed above. Two overly-constrained interpretations of Order 413.3B were described previously. The first, noted above in the section “OMB Circular A-11 and PMIAA Applicability,” is DOE’s interpretation that Order 413.3B “applies ONLY to construction projects, major items of equipment (MIE’s) and (currently) environmental cleanup projects.”57 This appears to be inconsistent with the purpose as stated in the introduction to Order 413.3B. This interpretation of the scope of 413.3B in large part appears to have driven EM’s development of a protocol for demolition projects, Office of Environmental Management Cleanup Project Management Protocol and Implementation Standard for Demolition Projects (hereafter “Demolition Protocol”) (DOE, 2020) (discussed in detail below in the section, The committee considered the possibility of increasing the applicability of Order 413.3B. Table 4.2 includes the committee’s analysis of what would be the effect of applying Order 413.3B to a greater number of projects in EM and also what would be the effect of adding provisions addressing certain issues. The latter include for example dispute prevention provisions that, in addition to being discussed in the table, are discussed further in Chapter 6.  57 Paul Bosco, Office of Project Management, DOE, “Project Management (PM) Governance, Systems and Training,” presentation to the Committee, May 6, 2020. PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION 4-10

The second instance of EM’s narrow interpretation and application of the DOE Order 413.3B is the exclusion for example of groundwater remediation. Here EM’s position could be seen as inconsistent with OMB Circular A-11, Preparation, Submission and Execution of the Budget—specifically the supplement, “Capital Programming Guide”—which states, “Capital assets include the environmental remediation of land to make it useful” (OMB, 2016, p. 55). This latter concern is somewhat mitigated by the issuance of the Demolition Protocol (DOE, 2020). DOE further described its application of Order 413.3B to site-based contracts and projects. Where applicable (e.g., if the threshold requirements are met), Order 413.3B Section 3b requires a Contractor Requirements Document (CRD), and Attachment 1 of the Order elaborates a list of requirements applicable to the contract into which the CRD is inserted. As noted much of the work under contracts awarded by EM is not subject to 413.3B. M&O contractors have a stewardship role at two of EM’s sites (Savannah River Site and WIPP). In certain instances, M&Os are tasked with performing cleanups, such as at Savannah River, 58 and capital asset projects (CAPs), such as at WIPP.59 To the extent M&Os are, or will be, conducting or overseeing cleanup contracts and CAPs, the committee believes EM should ensure the requirements of the Order are applied at the project level. The incorporation of the Demolition Protocol into Order 413.3B was either planned or in progress during the Committee’s study, and the specifics of how the incorporation was to be accomplished were not known to the Committee. Were this to be accomplished by invoking one of the exemptions in Order 413.3B, the committee notes that section 3c(4) requires approval by the Deputy Secretary (S2) for any exemption to meeting the requirements of Order 413.3B and the meeting of all three requirements of this section. EM does not appear to meet the first requirement of an established Project Management Support Office (PMSO) with “adequate project management requirements, processes and procedures defined to enable project success,” given their high percentage of Capital Asset projects that they do not perform per DOE Order 413.3B. The definition found in Section 3c(4) of Order 413.3B, third bullet point, defines eligibility for exemption as: Completed 90% of projects across a three-year rolling average, not to exceed by more than 10% of the original cost baseline for the original approved scope at CD-2 for all capital asset projects with TPC [Total Project Cost] greater than $50 million. (DOE, 2018a, p. 4) This definition implies a number of possible issues:  Acceptance of a 10% overrun  Performance is based on number of projects as opposed to aggregate cost performance of the portfolio of projects considered. In the proposed IDIQ approach EM disproportionately weighs many small projects towards their overall performance. A simple example to illustrate the point would be a group of 10 projects, 9 with TPC at $50 million and one with TPC of $1 billion. The 9 projects are each completed for $55 million (10% overrun and deemed acceptable); the 10th project experiences a 20% overrun and deemed unacceptable). 90% of the projects have been completed within 10% but the aggregate of the group on 10 projects is an overrun of nearly 17%.  The Demolition Protocol (DOE, 2020) objectives could also be accomplished as a Project Execution Plan (PEP) for a class of projects within the context of Order 413.3B and is consistent with the PEP approach taken by NNSA. The formal incorporation of the Demolition Protocol into 58 For example, the Savannah River Nuclear Solutions LLC contract was expanded to include cleanups. SOURCE: Fluor, 2020, “U.S. DOE Savannah River Site Management & Operations” Available at < https://www.fluor.com/projects/savannah-river-nuclear-management-operations>. 59 “NAS 3133 Response to Request for Additional Information #1 dated 03062020 (Item 5),” sent by Catherine Bohan, Office of Environmental Management, April 6, 2020. PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION 4-11

Order 413.3B was still either planned or in progress during the study and therefore not known to the committee. FINDING: Adoption of Order 413.3B to specific projects or project types is best carried-out through effective use of the PEPs as successfully demonstrated in NNSA.   The committee considered the possibility of increasing the applicability of Order 413.3B. Table 4.2 includes the committee’s analysis of what would be the effect of applying Order 413.3B to a greater number of projects in EM and also what would be the effect of adding provisions addressing certain issues. The latter include for example dispute prevention provisions that, in addition to being discussed in the table, are discussed further in Chapter 6.  TABLE 4.2 Assessment of Benefits and Challenges of Broader Application of Order 413.3B to EM Projects Coverage of Order 413.3B Benefits Challenges Increase Order 413.3B coverage Order 413.3B generally represents Increases the need for FPD and best practice for project management associated PM staff and should be broadly applied within Requires a broad consistent EM to improve the overall quality of PM culture to be built across project execution and the results the sites obtained Requires a resource sharing Acts to strengthen the culture of culture to be built across sites project management required to to meet evolving EM priorities deliver EM’s mission Requires strengthening of EM project support capabilities and potential elevation of these within the organization when program management needs are considered in the second part of our report Address relationship to PMIAA Strengthens links between portfolio, and expand coverage to address program and project relationship of programs and Create defined linkage between projects to portfolios portfolio outcomes, program end- states (and defined portions thereof) and projects including those awarded under a task order approach, if appropriate Clarify that scope of coverage Ensures Order 413.3B coverage of includes capital assets as defined EM projects of all types within in A-11, not just construction established size thresholds projects, including the full life- cycle, through environmental restoration, of projects related to a capital asset PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION 4-12

Coverage of Order 413.3B Benefits Challenges Explicitly recognize the Maintains EM as consistent with circumstances that allow for a Order 413.3B and can be combined CD-1,2,3 approach accomplished through the PEP process as contrasted with the inclusion of the protocol as an Appendix to Order 413.3B Life-cycle cost accumulation and Strengthens links between portfolio, links to portfolio and program program and project and provided goals needs to be expanded increased transparency of overall EM progress towards ultimate cleanup outcomes Links to DOE Order 210.2A The committee found the lessons related to lessons learned should learned process to be an opportunity be strengthened for improvement Changes to the baseline schedule Increased focus on program and should require Chief Executive for project schedule performance Project Management approval Dispute prevention and resolution Growing reliance on single-award should be added to Order 413.3B IDIQs for “end-state” contracting require strengthening of dispute prevention Retain Order 413.3B coverage for NNSA has demonstrated how to EM projects without the proposed comply with Order 413.3B through addition of the issued protocol as appropriate use of the PEPs. This an appendix to Order 413.3B approach is viewed as appropriate for EM and retains Order 413.3B best practices Modify the definition of eligibility Reinforces a higher standard of for exemption found at Section performance, which is increasingly 3c(4), point 3 as described important given increased use of previously IDIQ Reduce threshold for Order PEP for small projects can adapt 413.3B applicability to $20 Order 413.3B to improve project million consistent with current management systems and practices pilot project in NNSA and build the broader project management culture that task order contracting requires. This would foster greater cross-site consistency of implementation of Appendix D PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION 4-13

COMPARISON OF ORDER 413.3B AND NEW PROPOSED CLEANUP PROTOCOL The Department presented its rationale for why the Demolition Protocol was needed and why the protocol is envisaged to be separate from Order 413.3B: EM often demolishes and disposes of facilities where the design elements common to construction may not be applicable. Further, demolition projects are often conducted against the requirements of a regulatory framework, court orders, consent decrees, or site-specific cleanup agreements that are legally binding and may govern their processes, schedules, alternative selections, technical approaches, scope, end states, decision points and required approvals. The work is frequently covered by a Record of Decision (ROD) or Action Memorandum. The draft Protocol establishes a standard tailored approach to comply with project management requirements specifically related to demolition projects within the framework of DOE O 413.3B, by allowing substitution of equivalent processes, and consolidating Critical Decision (CD) phases.60 The rationale given above for the need for a separate Demolition Protocol however still appears to take a narrower view of the applicability of Order 413.3B, as follows:  The OMB Circular A-11 supplement, “Capital Programming Guide,” includes in its scope environmental restoration (i.e., “decontamination and decommissioning efforts”) in its Appendix on Definition of Capital Assets (OMB, 2016, p. 55); and  There is limited overall coverage of EM outlays (i.e., spending) by Order 413.3B. In particular it is the committee’s understanding that projects below $50 million are not covered61,62 although equivalency is encouraged through a DOE (2018b) policy (issued by memorandum in August 31, 2018).63 Deactivation projects were described as not covered by Order 413.3B. The committee’s concerns are underscored by the committee’s finding that Order 413.3B is generally a best practice with respect to project management. The Demolition Protocol appears to exclude roles for the Project Management Risk Committee (PMRC) and the Energy Systems Acquisition Advisory Board (ESAAB), which were in Order 413.3B. Given the scale of the demolition challenge it is conceivable that specific projects may exceed the $750 million threshold laid out for Major Systems Projects that would otherwise require Deputy Secretary approval, whereas, in the Demolition Protocol, approval has been delegated to a lower organizational level (S4). In addition, although the Protocol’s development was heavily influenced by the regulatory processes that are often present, the demolition projects that would be covered include those for which such regulatory frameworks are not present. Such projects would have been amply covered by Order 413.3B and its PEP process. It also appears that certain independent reviews called for per Order 413.3B (e.g., Independent Project Reviews (IPRs) and External Independent Reviews (EIRs), have been replaced with Independent Field Office and Headquarters Assessments. Many of the features that contribute to Order 413.3B representing a “Best Practice” for project management have been diluted by or not included in the Protocol. 60 “Responses to NAS Questions” sent to National Academies’ staff by Rodney Lehman, EM-5.22, June 30, 2020. 61 Paul Bosco, Office of Project Management, DOE, “Project Management (PM) Governance, Systems and Training,” presentation to the Committee, May 6, 2020. 62 The National Nuclear Security Administration (NNSA) has a pilot project setting this threshold at $20 million for four projects. SOURCE: Bob Raines, NNSA, “NNSA and DOE O 413.3B,” presentation to the Committee, May 6, 2020. 63 DOE (2018b) reiterates that “all projects equal to or less than $50 million shall follow the Project Management Principles as established in Appendix C of DOE O[rder] 413.3B.” PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION 4-14

The new Demolition Protocol discusses “end-states” as they relate to specific projects. The protocol further advises “Federal teams should consider how best to package their site program and/or projects within a task order or series of task orders, if possible.” As noted previously, EM has expressed an intent to utilize an IDIQ form of contract.64 Some observations about this approach include the following:  End-state or outcomes-type contracts are desirable but EM’s interpretation of end-state is not aligned with past examples. Historically, Fernald and Rocky Flats (from year 2000 onward, when the closure contract took over from the cleanup contract [see Table 6.1]) represented DOE best practices towards “end-state” contracts, and EM can further improve upon these examples in current projects by incorporating the lessons learned from each project and defining end-states that represent significant integral portions of remaining portions of EM’s mission.  The shortcomings EM finds in Order 413.3B are fully addressed within the context of Order 413.3B through effective use of the PEPs. The committee noted both deficiencies in maintaining the PEPs as well as the successful approach adopted by NNSA.  EM’s application of the IDIQ contract form does not follow traditional procurement practices. IDIQ contracts place less burden on agencies to be precise about the scope of work, which is instead defined when task orders are issued. The contractor can heavily influence the subsequent task order statements of work, and the negotiations for scope of work and cost. The schedule goals in a single-award IDIQ are determined on a sole-source basis, further adding to the risk. Multiple award contracts, such as is the case with EM’s multiple contractor award for a national deactivation contract at the Paducah Gaseous Diffusion Plant, maintain effective competition throughout the acquisition.  EM’s Demolition Protocol for demolition projects states, “Disaggregation of site program work into smaller, discrete work activities is encouraged as it provides better project definition and clarity, is more manageable, reduces time horizons and risks, and is consistent with the project management best practices found in DOE O[rder] 413.3B.” (DOE, 2020, p. 3) The committee does not agree with this assessment. Specifically, a multiplicity of projects transfers a greater burden for project management to the DOE from a selected contractor; increases responsibilities with respect to interface management; creates a growing level of risk in the “white space” between individual projects; partitions risks which were demonstrated to be best aggregated on both Rocky Flats and Fernald; and limits the scope for innovations in project delivery and the opportunity for accruing meaningful incentives by the contractor. Industry best practices on large complex programs have sought to maximize risk aggregation consistent with industry appetite and capabilities and provide sufficient scope for innovation. The committee was unable to identify a clear reference to this as a best practice in Order 413.3B.65 FINDING: M&O contractors have a stewardship role at two of EM’s sites (Savannah River Site and the Waste Isolation Pilot Plant). In certain instances, such as the Savanah River Nuclear Solutions contract, M&Os are tasked with performing cleanups. 64 Norbert Doyle, Deputy Assistant Secretary, Office of Acquisition & Project Management (EM-5.2), “Contracting Overview,” presentation to the committee, February 24, 2020, Washington, D.C. 65 This will be further considered when the committee turns its attention to program management, as risk and “end state” must be considered from a programmatic perspective recognizing that the sum of project risks is less than the programmatic risk in large complex programs. PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION 4-15

CONCLUSIONS AND RECOMMENDATIONS CONCLUSION: The rationale for creating a new set of requirements outside of Order 413.3B that applies to projects proceeding under the Demolition Protocol is not apparent. There is a risk that these projects will have insufficient oversight. CONCLUSION: The current focus of Order 413.3B is on project management. The Order lacks attention to program management issues that are included in its stated purpose. RECOMMENDATION 4-1: The committee recommends that DOE confirm, clarify, and expand DOE Order 413.3B to establish its applicability to all capital asset projects (not just construction and Major Instruments and Equipment and certain cleanup projects) and all EM projects, whether major systems projects or work carried out by a Management and Operating (M&O) contractor. The committee makes the following specific recommendations regarding the Order as well: 1. Pending the outcome of the NNSA pilot project, reduce the threshold value for applicability of Order 413.3B from $50 million to $20 million; 2. Continue applying the requirements of Order 413.3B to M&O contract work on capital asset projects—the latter including construction projects, major items of equipment and cleanup projects; 3. Clarify the definition related to project performance found at Section 3c(4), point 3 to calculate performance on aggregate value and not number of projects; and 4. Shift eligibility for project overruns, currently 10% per project, to be applied instead based on the aggregate value. RECOMMENDATION 4-2: DOE should clarify Order 413.3B to incorporate best practices with respect to dispute prevention and resolution, which will be of growing significance as EM implements the “end- state” contracting approach. Sources for such best practices include the Construction Industry Institute. RECOMMENDATION 4-3: EM should evaluate the benefits of applying the requirements for Project Execution Plans equivalent to those in Order 413.3B to those projects that are not formally managed under Order 413.3B. REFERENCES U.S. Department of Energy (DOE). 2018a. Program and Project Management for the Acquisition of Capital Assets: Change 5. DOE O 413.3B. Washington, D.C.: DOE. April 12. DOE. 2018b. Office of Environmental Management Policy for Management of Capital Asset Projects with Total Project Cost Equal to or Less than $50 Million. EM Policy. April. DOE. 2020. Office of Environmental Management Cleanup Project Management Protocol and Implementation Standard for Demolition Projects. EM Protocol; Final: 06/08/2020. Washington, D.C.: DOE. GAO. 2019. Nuclear Waste Cleanup: DOE Could Improve Program and Project Management by Better Classifying Work and Following Leading Practices. GAO-19-223. Washington, D.C.: GAO. Office of Management and Budget. 2016. Capital Programming Guide V 3.0: Supplement to Office of Management and Budget Circular A–11, Preparation, Submission and Execution of the Budget. Washington, D.C.: Executive Office of the President. July. PREPUBLICATION COPY – SUBJECT TO FURTHER EDITORIAL CORRECTION 4-16

Next: 5 Project Management Metrics »
Review of Effectiveness and Efficiency of Defense Environmental Cleanup Activities of the Department of Energy's Office of Environmental Management: First Report Get This Book
×
Buy Paperback | $50.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The U.S. Department of Energy (DOE) and its predecessor agencies have conducted activities to develop atomic energy for civilian and defense purposes since the initiation of the World War II Manhattan Project in 1942. These activities took place at large federal land reservations of hundreds of square miles involving industrial-scale operations, but also at many smaller federal and non-federal sites such as uranium mines, materials processing and manufacturing facilities. The nuclear weapons and energy production activities at these facilities produced large quantities of radioactive and hazardous wastes and resulted in widespread groundwater and soil contamination at these sites. DOE initiated a concerted effort to clean up these sites beginning in the 1980s. Many of these sites have been remediated and are in long-term caretaker status, closed or repurposed for other uses.

Review of the Effectiveness and Efficiency of Defense Environmental Cleanup Activities of the Department of Energy's Office of Environmental Management: First Report provides background information on the sites currently assigned to the DOE's Office of Environmental Management that are undergoing cleanup; discusses current practices for management and oversight of the cleanups; offers findings and recommendations on such practices and how progress is measured against them; and considers the contracts under which the cleanups proceed and how these have been and can be structured to include incentives for improved cost and schedule performance.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!