Click for next page ( 2


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 1
Executive Summary The U.S. DeparDnent of Energy (DOE) Environmental Management (EM) program is responsible for radioactive waste management and environmental remediation at DOE sites. Within the DOE-EM program, the Office of Science and Technology (OST) has a mission to develop technology to facilitate these activities by reducing their technical risks and/or costs. OST funds research, development, and demonstration (RD&D) projects, and those that result in viable technologies are available for use by site contractors who are under the direct management of the other DOE-EM offices. SCOPE AND CONTENT OF THIS REPORT This National Research Council (NRC) report examines the prioritization and decision-making processes of OST. The major recommendations developed by the committee conducting this study are summarized and organized below around the four decision process issues raised in the committee's statement of task (see Box ESPY. Since the committee found that OST's decision processes are inextricably linked with DOE-EM organizational structure, institutional procedures, and program management, these related program elements are also discussed in this report as necessary. The broader institutional environment within which OST operates imposes constraints on the OST . ~ ~ ,~ . .~ .~ ~ ~ ~% ~ decision process. . Two features of this environment are the other 190~ ;M program offices and the federal budget process, both of which impact the decisions of OST program managers. Other significant influences on OST have included changes in top-level DOE-EM priorities and changes in the technical baselines (for example, changing the in-tank precipitation process for cesium removal at the Savannah River plant) that the other EM offices pursue in their site cleanup and waste management activities. In response to such influences, the programs, prioritization and planning methods, and decision- making processes used by OST have evolved over time. Consequently, the committee sought to evaluate He OST decision-making process in the context of its history and continuing evolution, thereby taking into account changes, improvements, and present directions. The committee concluded its information-~atherin~ activities in April 1998. Since then, OST has made changes to improve its operations. Nonetheless, this report addresses important characteristic and systemic issues in the RD&D program within DOE-EM, arid makes recommendations to farther improve OST's operations and the results they achieve. Some of these sites were part of the former nuclear weapons complex, with no future on-site mission except for the remediation of waste inventories, radiologically contaminated structures, and environmental contamination.

OCR for page 1
2 Decision Making in the DOE-OST BOX ES.l Statement of Task This report responds to the task statement reproduced below: A pane! of the CEMTa will be appointed to evaluate the effectiveness of the OST decision-making process and make specific recommendations to improve it, if appropriate. In particular, the pane} will address the following: the appropriateness and effectiveness of decision-making processes currently in use by OST to select, prioritize, and fiend RD&D activities, both at sites and at headquarters; _ & 1 _ ~ _ _ 1 1 ~ ~ . . ~ ~ . the technical factors appropriate to consider in the decision-making process for selection, prioritization, and the development of cleanup technologies, and the adequacy with which these factors can be measured: recommendations, if appropriate, for improving the decision-making process; and the role and importance of effective peer reviews in the decision-making process. For the latter topic the pane] will coordinate its work with the CEMT pane! on peer review.b The chair of the NRC appointed a committee of nine scientists and engineers to perform this study (see Appendix K). This committee held information-gathering meetings to obtain input from OST program units and site management personnel. Interviews with individuals and invited speakers at committee meetings provided information from corporations and institutions having technology development problems and activities in some ways similar to those of OST. aThe Committee on Environmental Management Technologies (CEMT), the parent committee of the decision- making panel, became inactive August 1997, after which time the Board on Radioactive Waste Management provided coordination and oversight of the study panels examining separate facets of the OST program. At that time the decision-making panel was renamed the Committee on Prioritization and Decision Making in the Department of Energy Office of Science and Technology. bThis panel, also renamed as a stand-alone committee, was chartered to examine OST's peer review process. APPROPRIATENESS AND EFFECTIVENESS OF OST DECISION-MAKING PROCESSES The first issue of the statement of task is "the appropriateness and effectiveness of decision- making processes currently in use by OST to select, prioritize, and fund R&D activities, both at sites and at headquarters." To fulfill! its technology development mission in a way that is responsive to DOE-EM site technology needs, OST's major decision process elements can be summarized as (1) identification of high-priority DOE-EM site technology needs, (2) program planning, (3) assessments of technology development proposals and projects, and (4) headquarters oversight. The committee believes that these process elements are appropriate but not fully effective as currently implemented. The recommendations that follow are offered to improve the effectiveness of each of these process elements. ~- - - -a ~ ~r -A ~ Process Element 1: Identification of High-Priority DOE-EM Site Technology Needs OST has formed Site Technology Coordination Groups (STCGs) at each major DOE-EM site to interact with local contractor personnel and others to obtain that site's technology needs. The STCGs

OCR for page 1
Executive Summary 3 form statements of these needs that they evaluate and prioritize (i.e., rank or rate) according to a set of STCG-generated criteria. The committee found weaknesses in the STCG structuring of criteria and in the STCG evaluative and prioritization methods. The criteria were different at each site and at some sites were not rigorously constructed. Although improvements to these criteria and methods are possible, the committee instead proposes the following twofold approach for effective identification of high-priority needs. ., . , . ,, , in, .. . .. ~. . ~. The first recommendation applies to user-requested and relatively near-term needs. Recommendation: OST should use the best available information on DOE-EM site technology needs as a guide for tailoring program goals and RD&D projects. As one way to acquire this information, OST should establish (or increase) its direct contact with site personnel at the problem-solving and decision- making levels. The committee believes that {onger-term needs should come from OST's consideration of the functional flowsheets2 for site remediation that the technology user organizations (i.e., other DOE-EM offices) already develop, use in their planning, and subject to reviews. However, these reviews should ideally include consideration of not only the current "baseline" version of the functional flowsheet, but also alternative fi~nctional flowsheets that are designed to deal with uncertainties for process steps with significant risk of technical failure or of underperformance in attaining a cleanup goal. Such alternatives would serve a backup role and produce additional technology needs. At present, OST has no direct role in establishing or reviewing these flowsheets, which are activities conducted by other EM organizations and contractors at the site level. Recommendation: In conjunction with the other DOE-EM offices responsible for site cleanups, OST should participate to the extent possible (e.g., by establishing a role for its contractors) in a review of site remediation functional flowsheets. OST's technology development projects should be responsive to technology needs identified from baseline remediation plans and from alternatives. This involvement of OST should assist in the derivation of technology needs for remediation plans. Process Element 2: Program Planning Each major OST program unit (e.g., a Focus Area or a Crosscutting Program; see Table l.l for a complete list) conducts planning activities in order to identify (~) the technical needs of greatest priority for that program and (2) a suitable suite of technology development projects to address these needs. These planning activities are important in directing the allocation of resources and are conducted in different ways; for example, no two Focus Areas have the same approach. This planning occurs in a complex, ever-changing, and politicized environment in which OST has more than one "customer" to satisfy. Other EM offices responsible for site waste management and environmental remediation activities are obvious end users of OST-developed technologies, but the U.S. Congress must also be satisfied that a reasonable fraction of OST products are useful and worm their cost. Furthermore, parts of OST expenditures are congressionally mandated (U.S. Congress, 1994; 199Sa-b). Consequently, the type and quality of the information provided to the U.S. Congress (and to other interested review groups) are critically important to OST. Recommendation: To aid program planning in this institutional environment, a decision methodology should be employed that is structured using quantifiable attributes wherever 2The functional flowsheet for site remediation plans is the planned sequence of process steps that comprise the waste treatment process from the initial waste configuration to the final waste end state.

OCR for page 1
4 Decision Making in the DOE-OST applicable, but that also allows for managerial flexibility. For decisions involving Me allocation of significant resources. OST should institute a decision-makin~ structure wherein ~rniects and/or nrono~nl.~ ~ C7 ~ - t--J--~ r--r~ are evaluated against consistently Retuned cr~tena such as project cost, probability of technical success, probability of implementation in field applications, potential cost savings, and human health risk reduction. This structure should be applied broadly throughout the organization, with each OST program unit evaluating projects against the same criteria, which should be quantifiable where possible. An important criterion would be the project's relevance to site activities; that is, all projects should have one or more specific objectives related to the cleanup and waste management goals at sites. For an ongoing OST project, its history of past performance in meeting appropriate developmental objectives would be useful information to gauge its likely future success. The quantification of project benefits and probabilities of success is necessarily uncertain, but the committee believes that a rigorous attempt to use quantifiable estimates and other available data is preferable to the less structured methods that OST has used in the past. The recognition and evaluation of uncertainties is itself a key part of a decision process. Any such decision support too! has limitations, particularly if it fails to account for all significant factors;3 therefore, managerial flexibility to "override the numbers" in occasional situations is necessary. There should also be sufficient managerial flexibility to enable responses to late-breaking needs Mat are important technology development opportunities. Recommendation: Important funding decisions and their rationales should be documented and made publicly available. The criteria and the decision-making process should be recorded and summarized to serve as a basis for learning from experience and for defending decisions. Records should be kept of the reasoning used in reaching decisions, including methodist used in evaluations, especially when managenal pnvilege to "override the numbers" is invoked. This structured methodology and documentation, in which the goals, factors, and criteria influencing the decision are clearly specified, is a useful way to organize information for the OST managers who make decisions; moreover, it creates a clear and defensible record. The committee's judgment is that the discipline imposed by this methodology would improve the effectiveness of the OST decision-making process by defining how projects relate to top-level goals and projected benefits. Part of the structure to the recommended decision process is that of estimating cost savings from the use of technologies other than current baseline approaches. Although OST has attempted to obtain a uniform and credible cost-estimating methodology by employing the U.S. Army Corps of Engineers, further steps are needed to establish the credibility of this methodology. Recommendation: OST should do "cost avoidance" (or return on investment) calculations on its more expensive technologies in a more credible manner than was done in past efforts and should communicate the results to potential technology users in the most effective way possible. Initial estimates of costs and benefits should be developed at the inception of large RD&D projects. Refinements of the estimates should be a part of the project as it progresses, and followed up by a comparison of the estimates with the actual incurred costs. 3 The optimal decision is not always that of selecting the candidate with the highest score from a quantitative evaluation against a set of criteria, particularly if the criteria and scoring system do not adequately represent the complexity of the most relevant considerations. In the absence of a more rigorous method to treat these considerations' the committee suggests that managers be allowed to select candidates that are not necessarily those with the highest scores when there are justifiable and documented reasons to do so.

OCR for page 1
Executive Summary Process Element 3: Assessments of Technology Development Proposals and Projects s Each OST program unit manages new and ongoing projects. Ongoing projects are continued subject to their relevance to current site technology needs and to their successful review in the "stage-and-gate" system that OST programs use to monitor the status of work in progress. New technology development projects are solicited currently based on statements of high-priority site needs. Principal investigators (PIs) at national laboratories, DOE-EM sites, universities, and industries respond to these solicitations with proposals that are evaluated and scored against a set of criteria, typically by independent review teams. Program managers use these technical inputs, programmatic considerations (e.g., relevance to areas of greatest need and to program goals), and available budget to select the PI proposals to be funded as new activities. The committee believes that the "stage-and-gate" system to monitor ongoing projects has more development stages, separated by decision points (gates), than necessary for effective management of OST projects. Furthermore, the current use of the "stage-and-gate" system seems to merely track projects. It does not seem to assist effectively in decision making either in (~) the initial decision to select a project for funding or in (2) the decision to terminate a project. Recommendation: OST should use the minimum number of stages and gates needed to track a project and should use peer reviews (NRC, 1997b; 199Xb) at key decision points (gates), especially in the selection of a new project. The independent review teams, if appropriately constituted, provide this initial peer review, but not all OST program units use such review groups. Before a technology need statement is translated into a solicitation for new technology development projects, the decision process should include a "buy-versus-make" procurement decision of whether the need can be met adequately with technologies already developed and available from the private sector. The present approach to technology procurement-wherein several OST organizational units (specifically, the Industry Program, the Large-Scale Demonstration Projects tESDPs], and the Accelerated Site Technology Deployment [ASTD] Program) perform some aspects of technology selection and procurement from industry is cumbersome and duplicative, impairing OST's effectiveness. Recommendation: A better-coordinated, less duplicative, and less cumbersome system should be established for integration of technology procurement activities. Since decisions to develop technologies should be made only if warranted following a "make-or-buy" review, the ability to assess available technology is crucial. These assessments should be done through up-to-date surveys of commercially available technologies that are coordinated across OST organizational units. Another component of the decision-making process is the methodology used to evaluate PI proposals received in response to an OST solicitation. For selecting new projects from among many proposals, OST program units typically use simple multi-attribute utility analysis scoring systems. In these constructions, criteria (e.g., safety benefit and technical performance superior to baseline methods) are provided to reviewers, who score a proposal on a numerical scale. These scores are combined with weighting factors to alTive at a composite score for the proposal. In the committee's judgment, these scoring systems are adequate, although improvements are possible. Process Element 4: Headquarters Oversight In an organization with several layers of management and program responsibilities, decisions on technology development cut across many boundaries and, to be fully successful, must be accepted by

OCR for page 1
6 Decision Making in the DOE-OST other offices. For OST, these other offices are (~) the DOE-EM technology user programs at DOE sites, which are responsible for both the planning that generates technology needs and the deployment of RD&D results, and (2) the hierarchy of institutions outside the DOE-EM program that are involved in the process that establishes the OST budget. in this institutional environment, OST headquarters oversight has important roles in coordinating the alignment of OST's goals with those of other DOE-EM offices (i.e., the community of technology users), managing tensions that may result from a conflict of goals with another program office or institution, and interacting with other government bodies (e.g., the Office of Management and Budget and the U.S. Congress) to establish the annual budgets of each of the individual OST program units. These considerations lead to the necessity for clear and consistent top-level goals to provide adequate direction to OST's efforts. The decision-making process within other DOE-EM offices is a "bottom-up" one (DOE, 1997j) relying on site-specific resolutions of cleanup priorities. However, a bottom-up approach does not preclude the use of top-level goals, which have been formulated in the past for these EM offices and for OST and continue to be refined. Recommendation: OST managers, in conjunction with other top- leve! EM managers,4 should develop strategic goals and plans that define explicitly the technical problems that OST program units will and will not address. These strategic goals should be communicated throughout OST and be specific enough to guide decisions. Any such strategic goals for OST should be guided by priorities established within over DOE-EM offices. For example, an area of current importance is the privatization of cleanup efforts at many DOE sites. Since privatization may impact DOE-EM's RD&D activities, top DOE-EM management should establish goals for OST's role in pr~vatized cleanups. APPROPRIATE TECHNICAL FACTORS AND THE ADEQUACY WITH WHICH THEY CAN BE MEASURED The second issue ofthe committee's statement of task is "the technical factors appropriate to consider in the decision-making process for selection, prioritization, and the development of cleanup technologies, and the adequacy with which these factors can be measured." The committee identified three major technical factors: (~) the degree of maturity of a technology, (2) the ease of integration of a technology into a total system as described by functional flowsheets, and (3) the impact of a technology in reducing costs, schedule delays, environmental and health risks, and/or the technical risks of system failure or of inadequate cleanup performance. The degree of maturity of a technology is represented by a developing technology's remaining assumptions and uncertainties that require testing, and can be measured in terms of the estimated time required to achieve a sufficiently large-scale (e.g., pilot-scale) demonstration. Recommendation: The aforementioned gate review of an ongoing project that is ready to progress into a different stage of development should include not just a technical peer review, but also an assessment of the maturity and likelihood of future success of that technology projects More mature projects have a shorter time to completion and therefore to potential deployment than concepts requiring more research and consequently more time. However, an unbalanced RD&D program having only relatively mature projects would fad! to deliver the benefits that can come Tom investments in projects of a less mature nature that 4Input from other EM offices is recommended, since OST's goals should be derived from user plans and needs. 5The likelihood of success is the combined likelihood of achieving the expected technical performance and achieving at least one field application.

OCR for page 1
Executive Summary 7 address longer-term needs. Therefore, the decision process should lead to a balanced portfolio of projects of varying degrees of maturity. The ease of integration of a technology into a total cleanup system, as represented by a site remediation plan in functional flowsheet form, depends on the flexibility of the flowsheet in permitting a change in the technical approach to one of the individual remediation steps. l:n proposing a technology change, a new technology must be more than simply an adequate substitute for the baseline technology it is proposed to replace. The new technology must have significantly better technical performance, less technical risk, and/or lower cost. However, for an OST technology to be integrated into a site remediation plan to meet a specific cleanup need, nontechnical factors also must be addressed. One nontechnical issue is that some remediation approaches are constrained by existing regulatory agreements and cannot be changed without renegotiation. The earlier recommendation for OST to participate in a review of the site remediation functional flowsheets will provide OST with information germane to both the technical and the nontechnical issues identified here. The intended impact of a replacement technology is to reduce costs, schedules, and/or risks of various kinds, and/or to enable a difficult or intractable task to proceed. Recommendation: The gate reviews of the stage-and-gate tracking system should also assess estimations of cost, risk, and schedule reductions, for situations in which this information is available. OST should use the resulting information to terminate projects with insufficient potential to achieve significant impact. One way to estimate these cost, risk, and schedule impacts is to identify the number of functional flowsheets needing a technology, the number of needs in each flowsheet, and the importance of each flowsheet need. The technology user community is the source of the information necessary to make quantitative assessments in this way. Such assessed impacts could enable OST to make informed decisions about which site needs to address with technology development projects. OST's current criteria for funding these projects include consideration of a technology's potential applicability at more than one DOE-EM site. Although this guideline is useful, the committee believes that it should be balanced against the opportunity for a technology to be primarily beneficial for a single application or single site. The quantification of technology impacts would enable these (and other) opportunities to be readily identified and would lessen the need for OST to rely upon guidelines. RECOMMENDATIONS FOR IMPROVEMENT The Bird issue of the committee's statement of task is "recommendations, if appropriate, for improving the decision-making process." In addition to the recommendations shown previously, OST's decision malting can be enhanced by learning Tom the practices of the private sector. Application of Practices in Private-Sector RD&D Decision Making Industries engaging in environmental decision making and the utility consortia of the Gas Research Institute (GRI) and the Electric Power Research Institute (EPRI) apply some of the practices listed below in settings analogous in some ways to OST's. Therefore, the committee thinks that these practices are relevant to DOE's technology development decisions. Recommendation: OST should adopt, where applicable and appropriate in the OST environment and to the extent practicable, basic principles of private- sector formal decision making and follow-up practices. The committee recommends that DOE-OST focus on Me following major practices:

OCR for page 1
8 Decision Making in the DOE-OST Understand, focus on, and monitor changes in customer needs and requirements. Agree on clear and measurable goals. Use a formal (i.e., common, consistent, structured, and rational) technology development decision making process and apply it uniformly. Think strategically (i.e., long-term and high impact). Measure and evaluate to guide resource allocation. Communicate across organizational boundaries (i.e., with technology users). Continually improve the research and development (R&D) management process. Hire the best people possible and maintain expertise. . . . These descriptors of practices are fashioned to capture one or more related ideas taken from already published work and are offered here to describe the attributes of a successful R&D management environment. Since decision processes exist in the context of an organizational structure, some of the practices apply to RD&D management, whereas others apply to the decision process itself. OST management already uses many of these practices to some extent, but the committee recommends that OST re-examine the degree and effectiveness of their use. Chapter 5 describes possible applications of these practices to OST. However, other applications are possible, and the committee prefers to leave to OST management the task of translating how these practices could best be adapted and Implemented. For example, "hire the best people possible and maintain expertise" would refer in private industry to the cadre of researchers within the company; for OST, which is a collection of program units run by program managers who contract out the technology development work to PTs, the application of this concept might be best translated as "select the most competent contractors." ROLE AND n~PORTANCEOFEFFECTWE REVIEWS The fours and final explicit issue of the statement of task is "the role and importance of effective peer reviews in the decision-making process." Peer reviews that is, reviews by technical experts who are independent of and external to the program of work being reviewe~are a vital part of a credible decision-making process. Recent NRC reports (1997b, 1998b) offer guidance with respect to the peer review of individual technology projects as they progress through development stages. As noted previously, the committee recommends the use of peer review in the selection of a new project for filming and at "gate" reviews that serve as later decision points to determine whether to continue funding that project's development. In addition, the committee considered other key decision points where independent, external review could be valuable, and identified two such opportunities: (~) the point at which OST headquarters managers develop budget targets for major OST program units, and (2) the point at which OST program unit managers establish their statements of technology needs. Recommendation on 9~: OST should have an in~lependent, external body review the bases for the annual decisions, made at the OST upper management level, that establish budget targets for OST program units. An existing organization such as the Environmental Management Advisory Board (EMAB), composed of representatives of interested and affected parties, could be structured to carry out such a review. The recommendation on (2) is, as discussed previously, for OST to have a role (e.g., through its contractors) in reviewing site remediation functional flowsheets. These two types of proposed reviews can be structured to have some, but not all, of the requisite attributes of a peer review (i.e., to be conducted by independent,

OCR for page 1
Executive Summary 9 external, technical reviewers); hence they might be described as "program" or "policy" reviews (for issue (~) or "quasi-external" reviews (for issue (2~. PROGRAM CHALLENGES AND MEASURES OF SUCCESS This section discusses the way in which the committee's recommendations address current issues, such as how to measure success, that challenge the OST program. OST's chief program goal is the use (or deployment) of OST-developed technologies in DOE-EM applications, either to reduce costs, technical risks, and/or schedules of waste management or cleanup activities or to perform a technically difficult or intractable task. Hence, the OST processes (or programs) could be measured by their outputs, as represented in measures such as the following: the sum of the cost impacts of each deployed technology; the estimates of risk and/or schedule reductions achieved by the use of OST technologies; and the enabling nature of OST-developed technologies to assist in difficult andJor intractable tasks. Other guidance on the generation of appropriate measures and the evaluation of a program against them can be found in NRC (1999b). Deployment of OST Technologies Thus far, OST has had only a modest percentage (approximately 20 percent) of its technology projects deployed at DOE sites, in part because of conditions, or ''barriers,'' outside OST's control. For example, DOE site managers are under no obligation to use the results of OST technology development activities. However, OST's deployment record is also due in part to the way it operated in the past, when technologies were often developed without due consideration to input from the site problem owners. in more recent times, these site inputs have been obtained through the STCGs. {Jnner-leve! OST management has decided to address this "denIo~rment barrier" explicitly with _ ~ . . ~ ~ ~ . ~ . ~ ~ , . ~ , , . . , programs such as the AS 1 D program, in Welch Ob 1 provlues lunOmg for me first on-sue aeploymem OI a fully demonstrated technology to a site that can show the potential for multiple uses of the technology in DOE-EM and an associated cost benefit. In 1997, OST received many more site proposals under these terms than could be funded, as an indication that DOE-EM sites can be given sufficient financial incentive to introduce fully demonstrated technology into their plans to expedite cleanups and accomplish remediation goals. However, the deployment of OST-developed technologies is not in general subject to these same site incentives, particularly if an OST-developed technology lacks full approval and wide acceptance as a proven technique, or if it cannot be integrated into the site baseline approach. The degree to which sites are unwilling and/or unable to assume some risk, cost, and/or delay in (~) substituting an OST-developed technology for a more familiar baseline technology or in (2) changing the baseline flowsheet approach are two examples of additional disincentives that are probably significant but that the ASTD program does not measure. The conclusion is that the ASTD program does not address all the factors regarding the adoption of new technology that contribute to the OST deployment barrier. In the committee's analysis, OST's deployment battier is a symptom of the challenges imposed by organizational structure. Both the inputs (technology needs) and the outputs (deployments) of OST's

OCR for page 1
10 Decision Making in the DOE-OST decision process are controlled by the other DOE-EM offices that are the users of the technology; hence deployment success requires coordination across program boundaries. The OST approach of performing RD&D in a centralized program office (e.g., OST) has the potential advantage of avoiding duplicative developments at multiple sites and coordinating development work across all of DOE-EM. However, for centralized RD&D to be successful within DOE-EM, as determined by deployment of OST-developed technologies or other measures, OST must (~) have good and current information on user technology needs, (2) develop technology products of interest to the user community, and (3) cultivate a customer within DOE-EM for these products that are ready for demonstration and deployment. All of these steps require interaction and cooperation between OST and other EM program offices. Two previous recommendations-on the identification of needs and top-level strategic goal~propose efforts that both OST and the other DOE-EM offices should undertake to promote technology deployments. As noted above, OST should identify technology needs based on (~) close and continuing interaction with knowledgeable site personnel and (2) vuinerabilities (i.e., process steps having significant probability of technical failure or underperformance) in baseline remediation functional flowsheets. Deployment will continue to be a challenging battier if the different DOE-EM program offices have different goals and incentives related to technology development. As noted above, OST should establish top-leve' strategic goals in order to guide internal RD&D efforts to be aligned with the priorities of other DOE-EM offices. CONCLUDING PERSPECTIVE With current uncertainties in costs and technical risks in the DOE-EM program,6 the RD&D function is necessary. The committee makes no finding of other possible organizational structures (i.e., different from the current DOE-EM program boundanes and delineation of responsibilities) Mat could be devised to accomplish Be RD&D needed in DOE-EM. Statements in this report are thus specific to the context and associated challenges provided by OST's RD&D activities undertaken to expedite the site cleanups conducted by He other DOE-EM offices. 6DOE-EM cleanup activities are estimated to represent in excess of $100 billion dollars over many decades. OST accounts for a relatively small part of this total DOE-EM budget; OST funding from FY 1991 to FY 1998 was approximately $2.8 billion, or 6 percent of the DOE-EM $45 billion budget during that period.