Discussion of Hanford Systems Engineering and Principal Findings

The Systems Requirements Review for the Hanford TWRS (TWRS SRR) (U.S. Department of Energy, 1995) was initiated in 1994 by then Secretary of Energy Hazel O'Leary in response to an earlier review of TWRS conducted by the Defense Nuclear Facilities Safety Board (DNFSB), and the Board's Recommendation 92-4 (Defense Nuclear Facilities Safety Board, 1992). The TWRS SRR, conducted by a high-level DOE headquarters team, found many deficiencies with the practice of systems engineering at the Hanford Site. The review has led to substantial changes in the TWRS management approach as described in the TWRS SRR Action Plan (U.S. Department of Energy, 1996b). These changes continue to be implemented as this report is being completed (U.S. Department of Energy, 1997a).

TWRS itself developed as a by product of the review and renegotiation of the 1989 Hanford Tri-Party Agreement (TPA) (Washington State Department of Ecology, U.S. Environmental Protection Agency, and U.S. Department of Energy, 1996—current version) that took place in 1992-93. What was then termed the "New Technical Strategy" involved combined and expedited retrieval, treatment, and disposal of both single-shell and double-shell tank wastes at Hanford. TWRS was established to deal with the wastes in the tanks, with the intention that it would use the methodology of systems engineering as the program's technical systems were developed and implemented.

The TWRS SRR team was unable to validate either the enabling assumptions or cost estimates of the TWRS conceptual architecture (i.e., the initial pre-design concept of the system to be developed). It found the reliance TWRS had placed on numerous first-of-a-kind architectures posed high programmatic risks (defined by DOE as risks with respect to cost, schedule, and technical performance that result from uncontrollable events, unforeseen circumstances, or unverified assumptions). The risks identified included the possibilities that remediation costs would be considerably higher than estimated, that schedules could not be met as planned, and that neither the high-nor low-level waste forms proposed would meet the relevant waste acceptance criteria.

The TWRS SRR team found that (U.S. Department of Energy, 1995, pp. Iv-v):

  • Systems engineering of TWRS has not yet reached the level of maturity needed to provide low risk "design-to" specifications.
  • Systems engineering is not the driving force for current TWRS programs.
  • The functional structure developed for TWRS is unnecessarily complicated and not well integrated with all site activities.
  • Quantitative performance requirements have not been established for most of the functions.
  • Processes for retrieval, pretreatment, and immobilization of waste often have been based on unverified assumptions rather than being selected from the results of defensible analyses of viable alternatives.
  • Satisfactory performance of key processes has been assumed in the absence of substantive data.
  • Key testing programs to obtain performance data do not follow proven engineering practice; they are focused on preferred processes with negligible attention being given to alternatives that might be needed if performance assumptions are not met.
  • Cost constraints are not included as requirements, cost estimates for the TWRS system cannot be verified, and these cost estimates appear to be very optimistic for "first of its kind" systems.

Similar issues had been recognized by the committee prior to issuance of the TWRS SRR report. In effect, the benefits of the systems engineering process are slow in being realized for the TWRS program.

In response to these identified deficiencies, DOE and Westinghouse Hanford Company (WHC) undertook the development of the TWRS SRR Action Plan (U.S. Department of Energy, 1996b). The committee was briefed on this plan in March 1996. The plan acknowledges the lack of an effective decision-making process for TWRS, exemplified by an



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 6
--> Discussion of Hanford Systems Engineering and Principal Findings The Systems Requirements Review for the Hanford TWRS (TWRS SRR) (U.S. Department of Energy, 1995) was initiated in 1994 by then Secretary of Energy Hazel O'Leary in response to an earlier review of TWRS conducted by the Defense Nuclear Facilities Safety Board (DNFSB), and the Board's Recommendation 92-4 (Defense Nuclear Facilities Safety Board, 1992). The TWRS SRR, conducted by a high-level DOE headquarters team, found many deficiencies with the practice of systems engineering at the Hanford Site. The review has led to substantial changes in the TWRS management approach as described in the TWRS SRR Action Plan (U.S. Department of Energy, 1996b). These changes continue to be implemented as this report is being completed (U.S. Department of Energy, 1997a). TWRS itself developed as a by product of the review and renegotiation of the 1989 Hanford Tri-Party Agreement (TPA) (Washington State Department of Ecology, U.S. Environmental Protection Agency, and U.S. Department of Energy, 1996—current version) that took place in 1992-93. What was then termed the "New Technical Strategy" involved combined and expedited retrieval, treatment, and disposal of both single-shell and double-shell tank wastes at Hanford. TWRS was established to deal with the wastes in the tanks, with the intention that it would use the methodology of systems engineering as the program's technical systems were developed and implemented. The TWRS SRR team was unable to validate either the enabling assumptions or cost estimates of the TWRS conceptual architecture (i.e., the initial pre-design concept of the system to be developed). It found the reliance TWRS had placed on numerous first-of-a-kind architectures posed high programmatic risks (defined by DOE as risks with respect to cost, schedule, and technical performance that result from uncontrollable events, unforeseen circumstances, or unverified assumptions). The risks identified included the possibilities that remediation costs would be considerably higher than estimated, that schedules could not be met as planned, and that neither the high-nor low-level waste forms proposed would meet the relevant waste acceptance criteria. The TWRS SRR team found that (U.S. Department of Energy, 1995, pp. Iv-v): Systems engineering of TWRS has not yet reached the level of maturity needed to provide low risk "design-to" specifications. Systems engineering is not the driving force for current TWRS programs. The functional structure developed for TWRS is unnecessarily complicated and not well integrated with all site activities. Quantitative performance requirements have not been established for most of the functions. Processes for retrieval, pretreatment, and immobilization of waste often have been based on unverified assumptions rather than being selected from the results of defensible analyses of viable alternatives. Satisfactory performance of key processes has been assumed in the absence of substantive data. Key testing programs to obtain performance data do not follow proven engineering practice; they are focused on preferred processes with negligible attention being given to alternatives that might be needed if performance assumptions are not met. Cost constraints are not included as requirements, cost estimates for the TWRS system cannot be verified, and these cost estimates appear to be very optimistic for "first of its kind" systems. Similar issues had been recognized by the committee prior to issuance of the TWRS SRR report. In effect, the benefits of the systems engineering process are slow in being realized for the TWRS program. In response to these identified deficiencies, DOE and Westinghouse Hanford Company (WHC) undertook the development of the TWRS SRR Action Plan (U.S. Department of Energy, 1996b). The committee was briefed on this plan in March 1996. The plan acknowledges the lack of an effective decision-making process for TWRS, exemplified by an

OCR for page 6
--> absence of traceability in the consideration or justification of key decisions on system architecture. A root-cause analysis conducted by a joint DOE-WHC review team identified numerous problems in planning, communication, and implementation (U.S. Department of Energy, 1996b). As a remedy, the TWRS SRR Action Plan recommends instituting a coordinated decision management process, with emphasis on the generation of alternative system configurations and their analysis via "trade studies" (comparisons of the relative advantages and disadvantages of alternatives). These trade studies would be conducted with specific reference to questions of programmatic risk. With respect to the TWRS systems engineering program itself, numerous important components either have been completed or are now in the process of being developed under the TWRS SRR Action Plan. These include the TWRS Mission Analysis (Acree, 1998), the TWRS Systems Engineering Management Plan (Westinghouse Hanford Company, 1996a), and development of the privatization baseline (Lockheed Martin Hanford Company, 1997; Pacific Northwest National Laboratory, 1996, 1997), intended to ensure compatibility between the TWRS baseline and the efforts of the private contractors used by DOE to begin treatment of double-shell tank wastes. The systems to be designed, constructed, and operated under the privatization contract are considered to be within the TWRS project boundary (Acree, 1998). FINDING: The TWRS systems engineering effort has not reached the level of maturity where a defensible conceptual architecture capable of achieving the TWRS mission has emerged. This point was acknowledged by DOE in its 1995 TWRS Systems Requirements Review (SRR). Implementation of the 1996 TWRS SRR Action Plan, which is now under way, should help remedy many of the deficiencies that DOE and this committee have identified in earlier reviews. Integration of TWRS with Broader Site Concerns The committee's 1994 report (National Research Council, 1994) noted the programmatic separation within DOE among remediation efforts for tank wastes, the tanks themselves, and contaminated soils. Since that report, a memorandum of agreement between DOE/EM Offices of Waste Management and Environmental Restoration has placed programmatic responsibility for all of these remediation efforts under the DOE/EM Waste Management Office (DOE/EM-30) (Person, 1995). This is a positive step forward in the integration of these efforts. However, the original memorandum also indicated that there would be ". . . no funding allocated for the remediation of single-shell tanks contaminated soil and ancillary equipment after fiscal year 1994. [T]his transfer will have no further impact on cost or schedule for the Environmental Restoration Program." Although the responsibility has been formally transferred, this statement suggests that no activity for integrating the tanks and surrounding soils into TWRS is planned. Recently, the memorandum of agreement was revised to give TWRS responsibilities for developing and mapping a vadose zone program plan, indicating additional steps toward an integrated program to clean up contaminated soils adjacent to the tanks (Kinzer, 1997). The TWRS Mission Analysis report (Acree, 1998) states that the TWRS program now includes contaminated soil sites. However, the report gives no details as to how the soil will be characterized or remediated. It does state that tanks will be closed with small quantities of residual waste that cannot practically be retrieved and that a surface barrier will be constructed over the tanks to limit infiltration of water. DOE has placed a high priority on the development of a site-wide strategy to address the impacts of Hanford tank contaminants in the surrounding unsaturated soils (vadose zone) and the groundwater beneath the Hanford site (U.S. Department of Energy, 1998). A technology development effort, the Hanford Tanks Initiative, is underway to demonstrate the methods and requirements to retrieve difficult-to-remove waste from Hanford single-shell tanks. Included in this initiative is characterization of the soil contaminated by a tank leak and evaluation of the risk of the residual waste left in the tank and leaked into the soil. DOE now acknowledges that TWRS must be viewed as a subsystem of the larger Hanford Environmental Management mission and must be fully integrated within it (Acree, 1998). However, this acknowledgment currently stops short of specifically defining the key interfaces between the TWRS project and the external factors integral to TWRS (Figure 2). The present lack of integration in the approach to remediation of the tank contents, tanks, and surrounding soils is exemplified by the TWRS Environmental Impact Statement (EIS) (U.S. Department of Energy and Washington State Department of Ecology, 1996), in which the preferred alternative is one in which all single-shell and double-shell tanks would be subject ultimately to the same waste removal goal, 99 percent. Consideration of remediation alternatives for nearby soils contaminated by tank leakage and past waste management is deferred to a future EIS, as is disposition of the tanks themselves and residual contents. Thus the contents of some tanks might be subject to extensive removal and treatment without regard to the ultimate disposition of the contaminants that have already leaked to the surrounding soils (National Research Council, 1996; Conaway et al., 1997). DOE's approach to preparing the TWRS EIS reflects the commitments made in the Hanford TPA; however, enlarging the "system" under analysis to include contaminated soils may put DOE in the position of appearing to hedge on its

OCR for page 6
--> Figure 2 TWRS Project Boundary Diagram. From Acree, 1998 (Figure 2, p. 10). TPA commitments. Nevertheless, organizing the systems engineering so that disposition of the contents of all tanks in the same way is taken as a "given" blurs a potentially useful distinction between externally imposed constraints (i.e., results of negotiation of the Hanford TPA and responses to other stated public values) and those derived from system requirements. Treating the problem posed by the tank contents in isolation from the problem posed by wastes already leaked from the tanks foregoes the opportunity to ask whether, for some tanks or tank farms, engineered systems proposed to slow the migration of leaked wastes might not also provide adequate mitigation of the risks associated with the wastes still in the tanks. The TWRS SRR team recognized this lack of integration in remediation planning for the tank contents, tanks, and soils. A letter report by the committee's predecessor panels (National Research Council, 1991) commented on the Notice of Intent (Federal Register, October 22, 1990, vol. 55, no. 204, pp. 42633-42638) of DOE to prepare a programmatic environmental impact statement (PEIS) on the Department's proposed integrated environmental restoration and waste management program throughout the Defense Nuclear Weapons Complex. In the report the panels "applaud the intent of this PEIS. They have long been supportive of the need to develop such an integrated program to insure that decisions concerning waste site restoration and management practices are made systematically, with due consideration of the effect of each action on the overall situation. Critical elements for such decisions include knowledge of the character of both the waste materials and the sites, plans for future use of the sites, and informed public support. A nation-wide integrated program, in the Panels' opinion, would provide a great improvement over the present piecemeal application of limited funds at particular sites." The PEIS that has recently been issued (U.S. Department of Energy, 1997b), however, has eliminated from its scope the analysis of environmental restoration alternatives. FINDING: Despite the formal transfer within DOE of responsibility for the tanks themselves and the soils contaminated by tank leakage to TWRS, the "system" being analyzed by TWRS is largely confined to the wastes within the tanks. The result is a decoupling of strategies for reducing the risks posed by the tank wastes from consideration of the risks posed by residual contamination left in the tanks and surrounding soils once the tank contents have been removed. Progress Toward Greater Reliance on Systems Engineering The Hanford Site Systems Engineering Plan (Westinghouse Hanford Company, 1994) was initiated in response to DNFSB's Recommendation 92-4, which criticized the adequacy of integration of systems engineering efforts at DOE/RL. However, the scope of the plan is somewhat more limited than its name implies. Recognizing that

OCR for page 6
--> some remediation activity, including physical systems acquisition, is already under way at Hanford, the plan reflects a "valued added" philosophy. Termed "situational systems engineering" by DOE, the goal is to manage the interfaces between projects and programs, particularly where common physical facilities or services are required. A Site Integration Group has been formed to serve as the primary technical integration forum for the site. A Hanford Site Systems Engineering Implementing Directive states that, "a detailed composite total site systems engineering network is not required'' (U.S. Department of Energy, 1996c, Attachment A, p. 4). The Hanford Site Systems Engineering Plan thus comprises a hybrid top-down, bottom-up approach (bottom-up in the sense that some system elements that would normally be developed through systems engineering already exist), in contrast to the classical "top-down" systems engineering model described earlier in this report. The goals of the plan include the production of a site-wide systems engineering database and a consistent set of integrated technical documents for site projects and programs. In addition to the formation of the Site Integration Group, Interface Control Working Groups have been formed to resolve inter-project or inter-site issues identified through systems engineering analyses and/or from interactions at the Site Integration Group. The DOE/RL representatives note that the current Hanford Site contractors also appear to be committed to high-level managerial oversight of systems engineering. A recent letter to the committee from the current contractor, Lockheed Martin Hanford Corporation, states that "we have made substantial progress employing systems engineering principles to develop a technically defensible and integrated baseline for TWRS" (Boston, 1998). The Hanford Site Systems Engineering Implementing Directive provides a broad systems-level view of the Hanford site-wide remediation problem (Figure 3). It includes the concepts of both initial and desired end states of the site, trade studies (alternative solutions), and definition of the overall system architecture. The Implementing Directive states in part that, "Sitewide Systems Engineering will define and manage requirements, issues, assumptions, and interfaces for sitewide activities requiring physical facilities and the boundary inputs/outputs requirements for projects. All operable units are considered physical facilities." (U.S. Department of Energy, 1996c, Attachment A, p. 3). The total Hanford cleanup system envisioned by the Site Systems Engineering Plan is illustrated in Figure 4. Detailed interface control documents (which define all interfaces between subsystems and between the system being developed and the external systems with which develop the system must inter-operate and assure compatibility [Eisner, 1997]) are being developed, together with the coordinating groups noted above. This should help assure that the work of multiple entities within DOE/RL having resources that functionally and physically connect is compatible and coordinated with the same basic mission and milestones. The committee recently learned from DOE/RL representatives that an integrated technical baseline for all site remediation activities (the Integrated Site Baseline) is being developed by the Site Integration Group under the Project Hanford Management Contract (PHMC). The committee agrees with a recent communications of the Defense Nuclear Facilities Safety Board that acknowledges progress in implementing Board Recommendation 92-4 (Arcaro, 1997; Conway, 1997). The Board found, however, that "the systems engineering process at Hanford is not yet institutionalized to the point where it is clearly directed, proceduralized, implemented, and repeatable." Systems engineering is now being applied in a "demonstration" way by DOE/RL in the Double-Shell Tank Retrieval Project that is part of the Hanford Tanks Initiative. FINDING: Progress has been made in infusing the systems engineering concept into remediation programs across the Hanford Site through institution of the Hanford Site Systems Engineering Plan. This plan has significant potential to improve site-wide integration if implemented effectively. Documentation The primary purpose of systems engineering is to organize information and knowledge so that it assists those who manage, direct, and control the planning, development, production, and operation of the systems necessary to accomplish a given mission (Sage, 1992). But this purpose can be compromised or defeated if information production and organization becomes an end unto itself. The primary products of the system engineering process should be a set of traceable design requirements that are used in design and procurement and in system verification and validation, a set of documented interfaces, a baseline description of the physical system, and a baseline description of the operational concept. These are the implementation-level products. Functional analysis (determination of the necessary functional requirements) and trade studies represent necessary preparatory work that result in background or backup documentation. Although design and procurement decisions are now being made (e.g., waste vitrification plants), the requirements necessary for implementing such system functions have not been documented yet. A first step, the detailed development of logic diagrams, is under way. Systems engineering within the privatized facilities, such as evaluation and selection of processes to be used, is the responsibility of the privatization contractors. The baseline system description that illustrates how all program elements at the site will fit together is still under development. The TWRS SRR found that significant interface discrepancies existed, particularly with respect to TPA commitments. Plans to address 44 of the 179 identified findings from the TWRS SRR are documented in the TWRS

OCR for page 6
--> Figure 3 Project Hanford Systems Engineering Approach. From U.S. Department of Energy, 1996a (Figure 1). SRR Implementation Plan (U.S. Department of Energy, 1997a). Until a clear description of the operational concept behind all significant remediation efforts planned for the site is produced, a credible site-wide assessment of the risk reduction that will be achieved through remediation is not possible. An early version of what was then referred to as the Hanford Site Systems Engineering Management Plan (Westinghouse Hanford Company, 1994) contains the document hierarchy description shown in Figure 5. Several concerns exist regarding this approach to site-wide hierarchy description. The site-wide document hierarchy contains an undifferentiated mix of program directives and policies, work-breakdown structure, cost and schedule, and system technical descriptions. In addition, this hierarchy merely clones itself for site, program, and project levels, resulting in unnecessary repetition and over-complication. As such, the plan appears to emphasize the production of documents rather than to act as a vehicle that effectively organizes necessary products for successful site remediation. The extraneous complexity of the figure obscures its real message—that report production is supposed to support logical evolution of the program. Another early version of the TWRS Systems Engineering Management Plan document hierarchy is shown in Figure 6. It is clearer than the Site Systems Engineering Management Plan figure (Figure 5) in that it delineates program policies, cost/schedule baselines, and technical documentation. How-

OCR for page 6
--> Figure 4 Hanford System Cleanup Hierarchy. From M. Grygiel, Westinghouse Hanford Company, March 1996 (viewgraphs entitled "Site Wide Systems Engineering Status Meeting").

OCR for page 6
--> Figure 5 Hanford Site Information and Document Hierarchy. From Westinghouse Hanford Company, 1994 (Figure 1-1).

OCR for page 6
--> Figure 6 Systems Engineering Document Hierarchy. From U.S. Department of Energy, 1994b (Annex 2, Figure 3-2, page 1-4).

OCR for page 6
--> ever, it lacks a clear description of the system and the system operational concept. This problem appears to have been resolved in the updated TWRS Systems Engineering Management Plan (Westinghouse Hanford Company, 1996a). The updated hierarchy is shown in Figure 7. This hierarchy more clearly shows the system description documentation and the existence of background/support documentation. FINDING: In the Hanford Site Systems Engineering Plan there generally has been an overemphasis on detailed analysis of systems functions, and, to date, an underemphasis on concrete system definition and evaluation of system alternatives. As a consequence, the primary purpose of systems engineering has been misconstrued as one of providing information as opposed to supporting the development and implementation of engineering solutions. This has led to an excessive emphasis on document production as the product of systems engineering, rather than implementation of activities at the site as the product. Remediation Alternatives Retrofitting systems engineering to existing projects and programs requires that a mix of top-down and bottom-up systems analytic strategies be employed, unlike the classical, top-down approach to systems engineering. Where top-down analysis leads to derived requirements that can be further analyzed through trade studies, the bottom-up analysis that is being done by DOE in its ''situational systems engineering" is flowing in part from requirements that are the products of legal and political constraints and are thus less amenable to trade-offs as the system architecture is defined. In both of the systems engineering programs that were reviewed (TWRS and the Hanford site-wide), derived requirements of the physical system are not clearly distinguished from constraints presented in the Hanford TPA and applicable regulations. Examples include the low-level waste vitrification plant and the goal to achieve 99 percent removal of single-shell tank waste on a tank-by-tank basis, both specified in the TPA. Given the numerous technical uncertainties associated with these and other TPA commitments, trade studies still need to be conducted in ways that do not preclude the consideration of reasonable alternatives to the TPA-determined path. The schedules imposed by the TPA may also operate to constrain trade studies. Milestone M-33 in the Hanford TPA, for example, makes the negotiated dates for TWRS retrieval, treatment, and final closure into major TWRS milestones. As such, they are not tradable against issues raised by leaked wastes and the potential risk reductions that may be associated with alternatives for treating the tanks. The Multi-Function Waste Tank Facility (MWTF) was the TWRS program's first major physical system acquisition. Conceived of by TWRS as six new double-shell tanks to be used for processing the wastes to be removed from the existing tanks at Hanford, the MWTF project drew the attention of the Defense Nuclear Facilities Safety Board (DNFSB). DNFSB Recommendation 92-4 expressed concern for the lack of a comprehensive systems engineering standard to guide the MWTF effort. A Technical Team created to advise the Hanford Advisory Board also raised questions concerning the need for the new tanks (Paulson et al., 1995). The project subsequently was scaled back to just two tanks, then cancelled. The fact that the TPA was subsequently re-negotiated to drop the MWTF requirement on the basis of new analysis leaves the committee hopeful that appropriate technical studies can lead to redefinition of the path that regulators and DOE program developers take through on-going TPA negotiation. FINDING: The need to "retrofit" systems engineering to programs and projects already under way, in a context in which major commitments are negotiated through the Hanford Tri-Party Agreement, is making it difficult for DOE to use systems engineering methods to generate and evaluate alternatives to the preferred system. This complicates the task of ensuring that the chosen remediation strategy is the most appropriate from a sound risk and fiscal basis. Programmatic Risk A previous NRC report (National Research Council, 1992) addressing systems engineering efforts at Hanford has noted the lack of adequate attention to uncertainties about technical assumptions and external influences that could affect adversely the ability to implement the preferred baseline program. This report presented a review of a draft systems engineering study for closure of the single-shell tanks and urged caution in placing confidence in technologies that have not been tested at the pilot-plant stage or that have not been applied previously at full scale under similar conditions to those for which they are proposed. The review recommended bringing promising options through the laboratory and pilot stages to ensure they would be ready for plant and field operation. In its 1994 report to Thomas Grumbly, this committee noted the lack of iterative evaluation of program contingencies, risks, resources, and other external and internal factors that could affect program implementation. The report concluded that the planning basis for tank remediation seemed to have substantial and unnecessary technical risks that created programmatic uncertainties related to possible schedule delays and budget overruns. It called for systematic contingency planning, including, in some cases, work on alternative technologies at levels sufficient to permit their fuller development if needed.

OCR for page 6
--> Figure 7 TWRS Integrated Baseline Document Hierarchy. From Westinghouse Hanford Company, 1996a (Figure 2.1, page 2-2).

OCR for page 6
--> The TWRS SRR team expressed similar concerns. It noted that plans for mitigating risks and validating key assumptions lacked sufficient detail, and it called for identifying important cost and schedule risks and development of mitigation plans. In response, the TWRS SRR Action Plan (U.S. Department of Energy, 1996b) called for preparation of a TWRS Programmatic Risk Management Plan (issued by Westinghouse Hanford Company, 1996b). The TWRS Systems Engineering Management Plan (Westinghouse Hanford Company, 1996a) includes an integrated and systematic risk management program approach involving three functions: risk assessment (identification of programmatic cost, schedule, and technical performance risks); risk analysis (quantification of the likelihood of an undesirable event and its impact should it occur); and risk handling (identification of appropriate actions, planning, implementation, and status tracking). The Hanford Site Systems Engineering Implementing Directive (U.S. Department of Energy, 1996c, Attachment B) also includes a risk management plan. Because the site-wide systems engineering effort focuses on integration of the wide range of projects and services being conducted on the Hanford Site, its risk management approach should address issues having a major impact beyond any single project or service. Its focus on resolving conflicts or discontinuities between projects/services, omissions in project/service baselines, interface compatibility issues, and conflicts in requirements or planning assumptions should provide a mechanism for addressing and resolving such issues at the appropriate level. More recently, the so-called "Contractor Integration Report" prepared for Assistant Secretary Alm in support of the 2006 Accelerating Cleanup Plan used a prescriptive systems engineering approach in its complex-wide assessment of opportunities for cost savings in the EM program (Complex-Wide EM Integration Team, 1997, p.3). FINDING: The TWRS systems engineering program is to be commended for the emphasis that it now places on explicit consideration and mitigation of programmatic risks—i.e., significant uncertainties about cost, schedule, or technology performance. Programmatic risk management procedures contained in the Hanford TWRS and Site Systems Engineering Management Plans, if implemented as proposed, and if implemented effectively, should remedy weaknesses that have been identified by earlier NRC reviews and by the TWRS Systems Requirements Review.