Click for next page ( 39


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 38
4 Decision Making in Research, Development, and Demonstration for the DOE-EM Program The key decisions for a DOE-EM technology development program are nominally those to identify and prioritize technology needs and to solicit and fiend projects in support of these needs, with the aim of helping to achieve EM cleanup goals. The way this is done within a federal program such as DOE-EM is governed by the framework (e.g., the federal budget planning process) and constraints (e.g., no direct control over the use of technology) of the institutional environment in which a central RD&D program office such as OST operates. These considerations are treated in this chapter with a discussion of the major decisions that must be made at a functional level, followed by a description of OST's institutional context. This context provides constraints, both internal and external to DOE-EM, that have some influence on whatever procedures are developed to perform the requisite functions. These procedures are a sequence of process steps that are institutionalized and conducted by the RD&D program office (i.e., OST) to make decisions. A decision-making process is presented below in a general form and then discussed in the context of current OST program structures. This chapter describes how the various process steps were conducted within the current DOE institutional structure; evaluative comments are provided in Chapter 5. MAJOR DECISIONS AT A FUNCTIONAL LEVEL Table 4.! shows the major technology development decision-making activities, which are conducted at various hierarchical levels in the federal organization. The first and last organizational levels conducted are outside of OST. The decisions made at those levels represent important influences outside the direct control of the RD&D organization. RD&D PROGRAM ENVIRONMENT The institutional environment, jurisdictional boundaries, and factors that impact the RD&D program within DOE-EM strongly influence the development of any decision process. OST decision making occurs in an environment with the following important features: Many players are involved, including Congress, the White House (via the OMB), DOE-EM technology users, stakeholders,~ and OST. iAs used here, a "stakeholder" is anyone outside the DOE organization who is interested in or affected by the activities of the EM program. Important stalceholder groups whose input is solicited by the EM program are local 38

OCR for page 38
DOE-KM RD&D Decision Making TABLE 4. ~ Decision-Making Activities for DOE-EM RD&D Organizational Level Decision-Making Activity Within the Office of Management and Budget (OMB) and Congress At DOE-EM headquarters, within the RD&D program 39 Within RD&D program units Within the site remediation organizations (i.e., the other DOE-EM offices that have remediation and waste management responsibilities at DOE sites) Establishing budgets for DOE-EM RD&D Establishing strategic objectives Proposing budget, and corresponding responsibilities and goals, among separate technical areas (e.g. Focus Areas and Crosscutting Programs) Determining what technology development work needs to be done (this requires interactions with the technology users and generation of a prioritized list of development needs across the EM sites, to be used as a basis for drafting requests for proposals) Evaluating proposals to select the technologies to be funded Evaluating and monitoring progress of funded projects Developing a baseline functional flowsheeta, and alternatives for each waste stream or remediation problem Determining the technology needs required to implement the baseline functional flowsheet (and alternatives to it), and forwarding these needs to the RD&D organization Determining whether the RD&D organization's technologies (i.e., those that have been or will be developed and demonstrated) will meet all the functional flowsheet requirements Deciding whether to use these RD&D technology products (i.e., their deployment) aThe baseline functional flowsheet is the presently preferred sequence of process steps that comprise the waste treatment sequence from the initial waste configuration to the final waste end state. It defines the technology needs in a general sense. A baseline functional flowsheet represents major commitments by the site operators to a technology path in which a great deal of planning and engineering commitment has been made and, very often, large amounts of money have been spent. In addition, the site operators have numerous nontechnical constraints, both regulatory and political, that discourage technology changes. Nonetheless, significant improvements to baseline functional flowsheets are possible. The federal budget process requires monitoring the current year budget and planning at least the one-year-out and two-year-out budgets. Therefore, three budgets have to be managed at any one time, in varying degrees of detail and certainty. OST is neither subordinate nor superior to other EM offices, which have cleanup responsibilities for the sites. In practice, this means that OST has no way to force the use of technologies it develops. However, these technologies must be used at DOE sites for OST to justify spending funds to develop them. Remediation "functional flowsheet" decisions are made by DOE sites funded by these other EM offices in response to regulatory and other drivers. Top-leve} EM goals2 and planning exercises can influence the activities of OST and other EM offices. The user-supplied needs that OST responds to are frequently Riven by EM goals (arid other influences) that, when changed, result in changed technology needs. advisory boards, comprised of citizens living near major DOE-EM sites; Native American tribal nations whose land is affected by DOE-EM activities; and groups such as the Community Leaders Network and the Western Governors Association. The taxpayer at large, as represented by Congress, also has an interest in reducing the costs associated with remediation activities. Headquarters priorities, such as risk and mortgage reduction, are examples of how top levels of management can dramatically change the timing of technology needs through changing goals. The use of risk reduction as a priority

OCR for page 38
40 Decision Making in the DOE-OST Congress makes a decision annually to provide OST's fielding based on broad perceptions of OST's benefit to the nation. Thus, OST has Congress as a "customer" as well as the technology users. On occasion, Congress provides directed funding to particular projects or organizations (U.S. Congress, 1992; 1994; 199Sa-b). MODEL PROCESS STEPS FOR DECISION MAKING IN A CENTRALIZED FEDERAL RD&D PROGRAM The mode! in Figure 4. ~ describes the managerial functions of an RD&D organization in the federal government. This figure is a generalization of the processes described in Appendixes B-E, constructed from site visits, documentation, and committee members' collective experience. It shows two important features of OST's current decision-making process: external constraints and functional steps, discussed separately below. External Constraints Figure 4.l diagrams the way OST interacts with the chief external organizations that provide important jurisdictional boundaries, namely, other DOE-EM offices and funding authorities, as depicted on the left-hand side and in the top section, respectively. The processes represented within these sections are outside OST's direct control but have impact on its decisions. Functional Steps budget (Box 8~; The boxes on the bottom right-hand side represent functions that must be performed in a decision- making approach of the kind OST has adopted, in which the RD&D program is based on site technology needs. The specific functions of this process are depicted as boxes representing the following steps: site technology needs are acquired from the user community (Box 6~; these needs are prioritized (Box 71; a plan is developed for technical activities based on these prioritized needs and the available Projects are solicited, and responses are evaluated to select those to be funded (Box 9~; Me principal investigator (Pl) of each fimded project conducts the work (Box ~ 0~; and setting measure governing EM work was championed during 1994-1996, corresponding with Thomas Grumbly's tenure as the DOE-EM Assistant Secretary. Beginning in 1996 win Alvin Alm's tenure as the DOE-EM Assistant Secretary, mortgage reduction~uying-down near-term costs by putting facilities in low-cost maintenance conditions was given highest priority. Either idea is readily incorporated in the selection of technology development projects and may be emphasized by increasing its relative weight compared to all other criteria against which projects are scored. It is clear that both risk reduction and mortgage reduction are worthwhile and necessary goals with comparable importance. However, the goals may be in conflict when compliance agreements are excessively rigid, for example, when remediation is required that results in negligibly small reductions in risk. This is a critical area of policy to be resolved by DOE and Congress.

OCR for page 38
! _ _ _ J _ Z 6 O cD Jo cn J LL <: r~ ~1 Z CO: m ,~ - 0 cn Z m o - c~ ~ UJ C) ~ O O. CD z O Z ~ C] _ CD ~: o cn ce ~: a: CD o CE U] cn C~ cn o cn LL z o Y o IIJ llJ 11 . ce ~ <: Z ~ ~- LL ~ cn o ~z J Q tL Ir e ~ ~ [C ZO > 0 ~ ~n . ~0 1- e llJ C ~ O O LL UJ l o 1~ C] ~ ~ O O ~ Z CL ~ ~ C) ~o 4< OCR for page 38
42 Decision Making in the DOE-OST . OST monitors and evaluates (Box ~ ~ ~ progress until funding ceases. A more detailed account of these functional steps, including a description of how they are currently done, is provided in the rest of this chapter. Allocation of the Budget Among User and Provider Program Units All DOE-EM program offices develop plans as a part of the federal budget cycle and receive their budgets from congressional appropriations. At the headquarters level, DOE upper management proposes the allocation of program funds among several subordinate program units. This allocation is then approved by Congress, often after modification. Occasionally, as in the case of the Environmental Management Science Program, Congress directs the appropriation of specified budgets to selected programs or projects. The steps in this part of the mode! decision-making process are depicted in Boxes i-4 of Figure 4.1. Boxes ~ and 3 show congressional decisions on budget allocations to two different types of EM program offices. Box ~ shows allocations directly to a technology user program, so-called because its RD&D- related role is that of the user of any RD&D results. Examples of user programs are DOE-EM offices such as EM-30 and EM-40 that bear responsibilities for site cleanup and waste management activities. Box 3 shows allocations to another type of program office, a technology provider program such as OST, which funds technology development activities and in so doing must interact with and be supportive of the user programs if it is to be effective and demonstrably beneficial. Boxes 2 and 4 show the role of DOE upper management in deciding what program funding for separate subordinate program units to propose in the budget submitted to Congress. For a technology user such as EM-30 or EM-40, these subordinate program units would be DOE field offices serving EM sites; for a technology provider such as EM-50, these program units would be Focus Areas or Crosscutting Programs. Planning Activity Involving Technology Decisions The EM offices plan during any one fiscal year for program work that may extend into the future over many fiscal years. Based on its own planning activities (Box 5 of Figure 4.~), the user identifies technology needs (Box 6) to support its site remediation functional flowsheet plans. These needs are sent to the provider. Because a typical technology development project takes at least two to three years, the provider must receive these needs sufficiently far in advance to allow time for development work. In practice, this means that the user needs for technologies in the future affect provider program development for more immediate budget cycles. Process Steps Internal to the Provider Program After receiving technology needs as inputs from a user program and a budget from upper-level management, the rest of the provider process steps are managed by the provider program. The provider manages the processes to

OCR for page 38
DOE-KM RD&D Decision Making 43 collect and prioritize user technology needs3 (Box 7), decide which technology needs to develop into funding solicitations (Box 8), which includes as substeps the identification of technical solutions to these needs statements and recognition of those technologies that may already be available outside DOE so that "make-or-buy" determinations can be performed (it is at this step that care must be taken to avoid unnecessanly duplicative technology development projects); prepare and tender solicitations for technology development proposals and evaluate the responses to select the ones to fund (Box 9~; initiate technology development projects by providing funds to the PI of each proposal selected (Box 10~; and monitor these technology development projects (Box ~1) to evaluate their progress over time. Deployment A very important aspect of the user-provider program interactions (Figure 4.~) is that deployment (Box 12) is controlled by the user. For those projects that are not terminated during the RD&D process for reasons such as technical difficulties that prove to be intractable, budget shortfalls, or changing needs, the end point is deployment of the technology to address DOE-EM's problems. This activity normally occurs when a DOE-EM problem owner determines that sufficient technical knowledge exists to design, build, and operate the new technology; an adequate budget is available for deployment; and the problem owner's need for such new technology has persisted. The problem owner (i.e., a technology user) can determine that the technology is adequate based on information received from documentation provided by the technology provider and from the user program office's participation (e.g., by contending the final stage of technology development) or the technology's demonstration in a field environment or on real or simulated waste, on a large- enough scale to foster confidence in its real-world application. Necessary Interactions and Feedback The arrow from Box ~ ~ to Box 5 of Figure 4. ~ represents, for each funded technology development project, the ideal outcome: demonstration of a technology that the user community then adopts. The arrow from Box 9 to Box 4 shows the role of each provider program unit (e.g., each Focus Area or Crosscutting Program) in presenting and defending, at the headquarters management level, information such as the program's Finding choices (i.e., the suite of technology development projects that represent the program's current investment), the past deployment successes, and future strategy. Headquarters judgments on filming allocations (Box 4) are influenced in part by past and present performance and by future plans. The arrow from Box ~ ~ to Box ~ depicts a flow of information that is useful for making a decision on whether to continue or terminate an ongoing technology development project. Project termination prior to the completion of the work could occur for several reasons. One reason is poor performance by the PI Prioritizations of needs and of solutions are frequently inseparable, especially for ongoing projects or site- specific demonstrations.

OCR for page 38
44 Decision Making in the DOE-OST (e.g., progress milestones are missed or funds are not spent in an appropriate or timely way). Research by the PI may show that the technology will not work as anticipated initially. Another reason for early project termination might be that the need it was designed to address may have become less important in another year's reprioritization. interaction between the provider program and the community of users of technology takes place on a continuing basis, as reflected in Figure 4.l, by the monitoring function, the results of which are communicated from the provider to the user as appropriate. IMPLEMENTATION OF THE MODEL PROCESS Good decision making (and the process steps of Figure 4. I) can occur within many possible program structures. OST's Focus Area-based structure was built on recommendations developed in 1993 (see Chapter 2~. The overall method used by OST for allocating RD&D funds can be described within the framework of Figure 4. I, primarily by Boxes 6-9. In this framework, the STCGs at each major DOE-EM site interact with representatives of the user programs to identify and prioritize technology needs at the site level (Box 6~. The STCGs provide the Focus Areas with prioritized lists of technology needs. These inputs are used to develop the Focus Area's prioritized list of needs (Box 7), which will form the basis for solicitations for technology development project work (Box 8~. Funding decisions on specific new projects proposed in response to solicitations are made through a review of these proposals (Box 91. The Focus Area's priority ranking of needs, with the determination of its budget (Box 4), determines the funding cutoff. More detail on the steps represented by each box of Figure 4. ~ is presented in the following sections. User Program Funding (Boxes ~ and 2) Funding is provided to a user program office through the annual federal budget process. This process focuses on the two fiscal years following the year in which the budget is being formulated. That is, the budget deliberations for FY 1998 (October I, 1997, to September 30, 1998) began with the preparation and submittal of a top-level budget to OMB and Congress during the spring of ~996, with a more detailed and refined budget prepared and submitted to OMB and to Congress during the spring of ~ 997. For a long-term program such as DOE-EM, the budgets, plans, and forecasts are sometimes made decades into the future. Based on such future projections and past accomplishments, each user program office prepares budgets that are forwarded to successively higher levels of DOE administration up to the executive branch, and sent back, in a process with some iteration. Policy controls are applied by the OMB. The result is the President's two-year-out budget request to Congress in early January. Congress (and its staff) deliberates on this budget through formal hearings and informal direct discussions with upper-level DOE-EM management. Congress can then alter the budget package by increasing or decreasing funding, creating new programs, changing the goals of existing programs, or specifying in varying degrees of detail how fiends are to be spent. The result is budget legislation (a law) that is sent to the President to sign, usually before the new fiscal year begins. The process repeats for the next fiscal year. The budget legislation provides the basis for DOE-EM program planning, which thus can be finalized only after the law is passed.

OCR for page 38
DOE-KM RD&D Decision Making 45 Provider Program Funding (Box 3) One of the major forces that both dives and shapes DOE-EM decision making in its RD&D activities is the process by which OST's total annual funding is d~ete~n~ined~. Because DOE-EM is hundreds annually by appropriations from the U.S. Congress, the decision-making process must take place within the framework of the federal and DOE budget process. This process takes place over three fiscal years (FYs): Program budgeting in FY-2. During this year OST prepares a top-level budget to be executed two years hence in the program execution fiscal year. This budget is adjusted and approved within DOE to meet spending targets and then submitted to the OMB late in FY-2. Program planning in FY-~. The FY-2 submittal is often iterated at least once with OMB during FY-l, and there are frequently informal discussions between DOE and congressional staff members to define an acceptable budget. Then, on about February ~ of FY-l, the OST budget is submitted to the U.S. Congress as a part of the much larger President's budget. For OST, this submittal would includ~e a specific figure for the total OST budget and the budget of major subprogram units (e.g., each Focus Area). In parallel with this, the Focus Areas are acquiring technology development needs from the STCGs, developing a prioritized list of responses to the needs, obtaining the necessary concurrence from user representatives, and issuing guidance to their operating elements. Usually, but not always, both the congressional budget process and Focus Area activities are completed before the he~innina of the next fiscal vear on October I. A ~--- ~_ _ BAA_ ~A_^ ~ it, ~ - _ _ Program Execution in the current FY. During this year the OST program outlined in FY-2 and in some dovetail in FY-! is implemented. This includes formally sending funds to performers and monitoring ongoing work. Simultaneously, the activities described immediately above for FY-! and FY-2 are proceeding for future years. Events during the year often cause either DOE or the U.S. Congress to adjust the budgets, usually downward via rescissions. Therefore, at any one time, OST manages three budgets. One is the current fiscal year's budget, already allocated by Congress. The second is the upcoming fiscal vear's hugest for which ~necific non ~r--------c7 ----I- all - -- rig -- r-~- a~re prepared in the spring and forwarded to Congress. The budget of each OST program unit (e.g., Focus Areas and Crosscutting Programs) is represented at the level of specificity of "work packages," as defined and described later in this chapter. Negotiations with OMB and appropriate congressional committees result in final budget figures in the fall, ideally prior to the October ~ beginning of the fiscal year. In step with this process is the formulation of the two-year-out budget, represented at the level of specificity of the requested budget for each major OST program unit and used for planning purposes by these program units as budget targets. For the provider program to be responsive to the technology needs of the user program, a technology need in the long-term plans of the user program would have to be addressed by a technology development project finished by the required time. Hence these technology development project funds are contained in more immediate fiscal year budgets of the provider program. As a consequence, a change in the user program plans for later years affects the plans of the provider program in more immediate fiscal years. User or Site Program Planning (Box 5) The user programs develop new plans or modify existing ones to accomplish cleanup objectives within their purview. These cleanup objectives are often required to meet compliance agreements. Plans are typically defined for an entire user program (e.g., remediate tanks containing high-level waste) and detailed at successively lower levels so that work for individuals or small groups eventually is defined.

OCR for page 38
46 Decision Making in the DOE-OST Identification and Prioritization of Technology Needs (Box 6) Planners of user projects must assess the capability of existing technology relative to what is required to achieve the specified objectives, such as specifications of the end state of the waste products (NRC, 199Sa; 1999a) in terms of composition, form, location, and cost. If technology to meet these objectives is not available, this deficiency should define a technology need that user program personnel (such as site contractors) identify and forward to OST. OST collects these needs from site personnel via the STCGs (see Chapter 2 and Appendix B). The STCGs are responsible for developing and prioritizing a list of site problems and technology needs. A technology need can exist for several reasons: a gap exists in the baseline plan for managing an environmental problem; an opportunity exists to replace a baseline technology with a better and less expensive one; there are several possible alternative technologies to the baseline, none of which has yet been fully proven or demonstrated; or there is no proven technical solution to a problem. The STCGs provide the link between the problem owners at each site and the provider organization. At most sites, members of the STCGs are DOE staff, site contractors, and other stakeholders. Several STCGs have formed subgroups corresponding to the technical areas of the Focus Areas. Members of the subgroups are usually technical staff of DOE and contractors who are closely associated with the problem owners. The STCGs describe technology needs in terms of the nature of the problem, relevant information regarding existing technologies and costs, operational and regulatory requirements, and technical performance. Each STCG has developed its own process and methodology for providing technology needs to the Focus Areas; specific descriptions and observations are made in Appendix B. Most STCG processes employ a user-oriented approach to technology needs identification, involve stakeholder input, use a structured and documented process for prioritization of needs based on STCG established criteria, and use a formal evaluation methodology. The results of the STCG process are prioritized lists of technology needs that are delivered to the Focus Areas. The prioritized technology needs are described in a couple of paragraphs, and some qualitative reasoning behind the priority ranking or rating is provided, according to a reporting template agreed upon by all the STCGs. Aggregation and Prioritization of Technology Needs for Each OST Program (Box 7) Each provider program (e.g., a Focus Area or Crosscutting Program) selects a suite of technology development projects to fund. This is done by gathering and prioritizing technology needs from the STCGs and sites, usually with the approval of the potential technology users from relevant DOE-EM sites, a process described below in more detail. Focus Areas use STCG needs statements and other resources to create a prioritized list of needs Cat is reflective of national, or EM-wide, pnorities. Although the emphasis of the STCGs is by design on problems at a specific site, the emphasis of the Focus Areas is on a specific problem area across DOE-EM sites. The STCG needs statements are used to update the Focus Area's portfolio of needs and determine investment targets. if the Focus Area is aware of an available technology solution to a site problem, this information is transmitted to that site's STCG and the need is usually removed from further consideration. Focus Areas typically aggregate similar individual needs statements to generate a smaller number of

OCR for page 38
DOE-KM RD&D Decision Making 47 more general topical areas of need, which are then prioritized nationally. Each Focus Area has the flexibility to generate these broader categories of needs areas and prioritize them in its own way. For example, the Subsurface Contaminants Focus Area combined a larger number of site needs into approximately IS work packages, which are groups of needs based on similar characteristics, written in general language (see Table C. ~ and Figure C.2 for examples), and then prioritized into a smaller number of work packages. As another example, the Mixed Waste Focus Area determined national needs by using data from EM-30 Site Treatment Plans, which outlined mixed waste treatment capabilities. As a third example, the need to retrieve residual tank wastes at multiple sites has been combined into a multisite need of the Tanks Focus Area that is addressed by one robotics-related development effort. Results of the needs evaluation and prioritization form the basis for building each Focus Area program's portfolio and for soliciting technology development proposals. Generally, high priority is assigned to the most important and urgent end user needs. The importance and urgency of end user needs in turn are derived from top-level EM and site priorities. These provider programs solicit input from both the DOE-EM user community (i.e., other EM offices) and non-DOE-EM interested parties. The user community provides input to the OST STCGs in the form of statements of needs. Outreach to non-DOE-EM stakeholders is conducted by OST program units in different ways: Focus Area stakeholder outreach is through groups with a national focus, such as the Community Leaders Network; in contrast, the STCGs have a local focus only. OST's non-DOE-EM stakeholder outreach activities are generally to provide stakeholder input on valuations, with criteria and relative weights, to be applied to all technology development proposals (Appendixes B-C). It has not been OST's practice to challenge the "stakeholder agreements" struck with DOE-EM user programs or to use stakeholders as advocates of specific technologies. Funding Decisions on Technology Needs and Projects (Boxes 4 and X) There are two types of budget allocation decisions in the overall framework described in Figure 4. ~ . The first allocates fiends among the suite of provider program units (Box 4~. The second allocates funds to specific technology needs and projects within each provider program (Box 8~. Allocations Among User Program Units (Box 4) The primary funding decision at the headquarters level is how to divide the proposed total OST budget among Focus Areas, Crosscutting Programs, and other OST program units. Using budget guidance from Congress on a target figure for an overall OST budget, headquarters management proposes a budget for each program unit. The output of this proposal is an allocation that forms part of the executive Branch budget request to Congress. The OST budget is submitted by the executive branch and obtained Trough the congressional appropriations process, usually after some changes. Congress then approves the budget for each major DOE-EM program unit in the final energy and water appropriations bills for a federal fiscal year. Funds for RD&D activities within other EM offices (i.e., EM-30, 40, and 60) are part of the much larger budgets of these offices. OST Headquarters Prioritization Process Near the end of the committee's information-gather~ng phase, OST submitted a written description (DOE, 1997i) of and presented (Walker, 1997b; Barainca, 1998) a prioritization process that was in the final stages of development. This process would be used by OST to create an integrated priority list of

OCR for page 38
48 Decision Making in the DOE-OST work packages4 at the headquarters level based on prioritized lists created and submitted by each program unit of OST (e.g., Focus Areas and Crosscutting Programs). In the new prioritization process, OST personnel produce a numerical score for each work package. Weights are provided by "ranking factors," and scores are assessed using the scoring criteria within each ranking factor (see Table 4.2~. These assessments use data submitted by the site with the work package and data from the database underlying paths to closure (DOE, 1998a). Each work package is evaluated against each of the scoring criterion within each ranking factor, and these numerical evaluations are combined with We weight of the ranking factors to determine a quantitative score. The new process is stated to be based on multi-objective decision analysis methodology. Beyond this generality, the processes used to determine the scoring criteria, ranking factors, and weighting factors were not specified. OST used this process to establish the priority order of most work packages for the internal review budget (IRB) during the spring of 1998. in this system, the evaluations against the scoring criteria and the values of the weighting factors used are opportunities to exercise management judgment. Some projects (e.g., the EMSP) could not be scored under the new system, however, and were assigned priorities based solely on management judgment applied directly. For the work packages subjected to the process, OST representatives noted that the database underlying the prioritization system was not yet at a point where it was sufficiently accurate to provide the sole basis for prioritization, and management judgments were used In this prioritization process, the highest-scoring work packages are selected for OST funding. Work packages with successively lower scores are then funded until the sum of the funds associated with work package projects exhausts the total OST budget. The budget of each OST program unit (e.g., each Focus Area or Crosscutting Program) can also be determined as the sum of the funded work packages under its management. in principle, this prioritization process can therefore be used to set budget targets for OST program units. In the past, these budget targets were set in a more subjective way by top-level OST management, taking into consideration EM strategic goals (such as risk or mortgage reduction; see Chapter 2~. TABLE 4.2 Five Criteria Used by OST to Rank Work Packages in FY 1998. Ranking Factor Scoring Criteria . Site needs Project visibility Future technology deployments Closure technical Risks Technology cost savings Need priority, which includes consideration of need timing High visibility, which includes consideration of urgent or high risk and high cost Planned for deployment, which includes consideration of deployment commitments Critical path to closure risk, which includes consideration of waste steam technical risk Cost-saving potential 4A work package is a group of related projects, which in FY 1998, had annual costs ranging from less than a million dollars to tens of millions of dollars. In FY 1999, examples show the range to be from $0.2 million to $10 million, with an average of around $2 million. The total of these packages accounted for only about a third of the OST budget.

OCR for page 38
DOE-KM RD&D Decision Making Allocations of Funds to Specific Technology Needs and Projects (Box X) 49 The allocation of funds to projects within Focus Areas, Crosscutting Programs, and the industry Program is Riven by the program unit's prioritized lists of technology needs and projects. The highest- priority items within each program are funded, going down the list, until the budget is exhausted. Solicitation of RD&D Proposals and Select Providers (Box 9) The highest-ranking needs form the basis for soliciting proposals from industry, universities, and national labs, via appropriate solicitation announcements, and for continuation of worthwhile multiyear projects. Proposals received in response to these solicitations are evaluated against relevant technical and programmatic criteria, generally by technical review teams comprised of non-OST personnel, as well as OST program managers (see Appendixes C-E for furler detail on the review procedures used). The highest-rated proposals, scored against both technical and programmatic criteria, are selected for funding. Performance of Technology Development Projects (Box 10) Projects selected for funding by OST are carried out by their proposers, who are PIs in national laboratories, universities, and private industry. Each funded project is tracked using performance milestones, technical status reports, and spending records. The goal of some Crosscutting Program projects is to transfer technologies under development to Focus Areas that can then support the remainder of the development work needed for eventual successful demonstration and deployment. Project Monitoring and Evaluation (Box Il) A major OST initiative to evaluate and monitor its projects was the adoption of the "stage-and-gate" model. Other efforts were to employ peer review (NRC, 1997b) and to adopt a suite of performance indicators to aid in the fiscal and technical management of technology development projects (DOE, 19970. In the stage-and-gate mode! (Paladino and Fox, undated; Paladino and Longworth, 1995) for mapping and tracking the majority of a technology development effort, a project progresses from the basic research stage (stage I), through bench-scale testing, applied engineering and development, and pilot- and filll-scale demonstrations; to a final (seventh) stage, implementation by the end user (see Figure 4.2~. Each step in the gate process indicates a level of maturation of the technology and an OST decision point to continue its funding support. The stage-and-gate mode! has been refined (DOE, 19960) with the development of gate criteria specific to OST and a formal OST gate review process. Once a project is funded, it is tracked in the stage-and-gate system, where gate criteria serve as decision points to promote the technology development project to a higher stage (with a correspondingly higher filming level and higher level of programmatic commitment). Gate and peer reviews on works in progress are used in principle for this purpose. The application of stage-and-gate and Deer review enhancements to OST decision making is discussed in more detail in Chapter 5. Gate 4 is an important gate, because it is the point of transition from a small-scale project to a significant, applied engineering development effort, requiring the commitment of significant resources to run larger-scare tests and demonstrations. OST recognizes that these expenditures should not be conducted without a review of the project's likelihood of success~eaning, in addition to technical success, the likelihood of its being used in DOE-EM and the likelihood of its being made into a commercially viable

OCR for page 38
so ~- - - - - - l - - - - - - - - l - } - o ~ ~ - --- - - - - - -~ u ~ E e ' E _, _, _. , ., __ - _ ~ D O O C, S _ __ __, ,___ ~= o" ~ ~t 0 3- .E o o ~, C o X ~ _= ,o o C ~ _ ~ I" ~- oo ~ g 8 E uo Q i:i e C , ~ o _ sU Bo o _ ~ t= o ___ _ ,_ ... 1 oo 0 o. . _ . _ l t. , ~ H ~ _ - ~ _ O o~ c ~4) ~o ~ ", ~ - E ~ . . .. : C _ c: ._ ~ o o C) 3~ ~ o ~ ~ - ~ O .= ~c, s - ~ o .k o ~ ~ ~O ~ C z <) ~- a ts] z o 00 c: ~3 8 e z 6o a ~ e 00 2 e$~ c~ ~ ~_ 00 V ~7 ~ e P~ o v, c~ c~ c~ o o ~_ co o - o c, o ~2

OCR for page 38
DOE-EMRD&D Decision Making 51 product. Such considerations are part of the gate 4 review criteria. For example, one criterion to pass through gate 4 is that the PI of a technology development project involve a private-sector business partner who is in a position to commercialize the technique. Another criterion is to achieve a DOE-EM field site "buy-in" to use the technology, if it meets its advertised specifications. Technology Deployment (Box 12) During 1997, the historically slow pace of technology deployment caused OST to propose and Congress to fund a program initially called the Technology Deployment Initiative and later renamed the Accelerated Site Technology Deployment Program (described in Chapter 2 and Appendix E). The purpose of this program was to facilitate initial deployment of technologies developed by OST or the private sector by providing funding specifically for this purpose. A number of conditions were attached to this funding, including the need for problem owner endorsement and cofunding, and the identification of likely follow-on deployments. The initial set of ASTD projects was funded in FY ~ 998. CHAPTER SUMMARY: MAJOR DECISION PROCESS STEPS AND HOW THEY WERE DONE WITHIN OST DURING FY97-98 An important context for OST decision making is provided by the following features: the annual federal budget system and other oversight requirements, which drive each EM office to plan for several fiscal years into the future during any one fiscal year; the nature of funding for the OST program, with the budget not directly connected to immediate DOE-EM activities undertaken in support of compliance and cleanup agreements; and the fact that other EM offices (shown as user programs in Figure 4. I) are the ultimate customers of technology at EM sites and control both the baseline remediation planning (Box 5) and the deployment of technology (Box 12~. The decisions at these levels impact OST's program, yet they are largely outside the direct control of technology developers. Given this context and a functional description of the decisions that must be made, a general decision- making process can be formulated (Figure 4.~) to describe the current system. The process steps of Figure 4.! are carried out with input from various elements of the federal government, including Congress, OMB, the office of the Secretary of Energy, DOE-EM problem owners, DOE-EM technology developers, and site-specific stakeholders such as states, regional EPAs, Native American tribes, and the interested public. Figure 4.! depicts the external constraints on OST (in the top and left-hand sections) and the internal process that OST is attempting to implement (in the right-hand section). This figure can be used in several ways. One way, pursued in this chapter, is to offer a description of how OST accomplishes the functional steps represented by individual boxes. Another way, pursued in Chapter 5, is to offer evaluative comments on how well these OST methods perform the requisite functions. Still a third use of this figure is to raise issues such as whether the diagrammed process is viable or in need of additional functional steps, issues also considered in Chapters 5 and 6. The figure raises yet another question of whether radically different processes might be constructed to be effective. No finding or recommendation is offered on this last issue, insofar as the committee's focus was on assessing and improving the needs- based approach that OST is pursuing.

OCR for page 38
52 Decision Making in the DOE-OST The history of OST presented in Chapter 2 and the context, function, and process steps discussed in this chapter lead to findings and recommendations in Chapter 6. in developing these, Chapter 2 has looked at the decision-making practices of private industry to gather additional information and insight. The results of these assessments are presented in Chapter 5.