Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
4 ACHIEVING AND MAINTAINING THE LONG-TERM VISION FOR ENVIRONMENTAL QUALITY RESEARCH AND DEVELOPMENT Chapter 3 described a vision for a different Environmental Quality (EQ) research and development (R&D) portfolio that would have a strong, if not dominant, long-term component. To move towards this vision, the Department of Energy (DOE) will need to redesign and rebalance its EQ R&D portfolio in substantial ways to better focus on its long-term EQ problems.4 This chapter describes a new portfolio management process that could help achieve these goals. To be effective, R&D portfolio management must operate within an effective management system, which includes identifying the decision maker (or decision-making group) who will make the hard choices of prioritization, resource allocation, and balance. Portfolio management systems for federal R&D programs also commonly seek out and use input from broadly qualified individuals in generating a comprehensive set of R&D needs and project possibilities. In general, the generation and selection of R&D projects should have inputs from qualified persons both inside and outside the program. This chapter discusses several institutional mechanisms that DOE could use to improve the management of its EQ R&D portfolio, including ways to generate and incorporate such input. For the most part DOE can implement the recommended new portfolio management process through an evolutionary approach (i.e., by modifying and supplementing existing management processes). The committee believes this is possible because DOE is already using portfolio management techniques (DOE, 2000b,g), and external reviews have found that management based on these techniques is yielding positive results but could be greatly improved (DOE, 2000h). Such an approach avoids disruptive reorganizations and maintains management focus on the goal (i.e., realizing the new R&D vision). ' As noted previously, the term "EQ problems" refers to the set of technical problems that collectively make up the EQ challenges described in Chapter 2. This is a useful concept in planning an R&D portfolio, because the challenges are very broad, and must be broken down into manageable parts to be addressed by R&D. 71
72 A Strategic Vision for DOE Environmental Quality R&D R&D PORTFOLIO MANAGEMENT PROCESS The primary objective of portfolio management is to ensure that an R&D portfolio is aligned, valuable, and balanced. "Alignment" is intended to ensure that the portfolio supports the strategic objectives and strategic direction of the parent organization (i.e., DOE's EQ mission and objectives). "Value" measures that support in quantifiable terms, such as net social benefit or utility. "Balance" examines whether the portfolio covers the full scope of objectives and approaches or is too narrowly focused on certain categories of R&D, time frames, or topics. In practice, these three objectives are often treated in sequence. An alignment process typically generates a list of R&D "possibilities" to be considered for funding by a decision maker. The adequacy analysis that identified the extensive list of R&D gaps and opportunities (see Appendix C) was essentially an alignment exercise. A valuation process is one means of prioritizing the list of R&D possibilities so that scarce resources can be applied to deliver the maximum benefit. The Work Package Ranking System (WPRS) that is currently used to select R&D work packages within the Office of Environmental Management (EM) has many similarities to a value-based prioritization system.2 Balancing a portfolio is a formal process for examining and considering how resources are distributed across critical dimensions and is applied after valuation to offset any imbalances that are inconsistent with overall program objectives. Examples of the types of displays that can be used to evaluate balance include diagrams displaying funding distribution across R&D maturity (Figure 2-6 of DOE, 2000b) and the levels of involvement of universities, national laboratories, contractors, and industries at various stages of R&D maturity (Figure 2-7 of DOE, 2000b). In the following sections, the committee discusses DOE's EQ R&D portfolio management processes in terms of the objectives of alignment, value, and balance. Alignment: Generating Improved Project Ideas Each of the DOE organizations that support EQ R&D has its own process for generating R&D project ideas. These planning processes are designed primarily to gather site and repository needs, which tend to be focused on short-term problems, and to turn these into R&D projects. For example, the participants who determine EM's site needs typically are DOE employees and contractors who are closely involved with the site problems and issues (NRC, 1999ay, with some periodic input from the broader 2 DOE does not have a single evaluation method for prioritizing R&D activities across the entire EQ R&D portfolio. Each organization that supports EQ R&D activities has its own process for prioritizing and selecting R&D projects.
Achieving and Maintaining the Long Term Vision for EQ R&D 73 technical community, such as from the Environmental Management Advisory Board (EMAB, see Sidebar 4.1~. The R&D activities supported by the Office of Civilian Radioactive Waste Management (RW) are identified primarily by DOE staff and contractors at the Yucca Mountain Site, although
74 A Strategic Vision for DOE Environmental Quality R&D the Nuclear Waste Technical Review Board also plays a role in identifying science and technology needs for the RW program (see Sidebar 4.2~. The Office of Nuclear Energy, Science and Technology (NE) relies on its Nuclear Energy Research Advisory Committee to generate long-term R&D needs (see Sidebar 4.3), although these needs are primarily directed towards nuclear power R&D3 (and hence DOE's Energy Resources R&D portfolio), because that is NE's overall programmatic focus. In addition, as discussed in Chapter 2, both EM and RW are driven by short-term milestones and deadlines. The short-term drivers and the limited set of participants work together to limit the development of the broad R&D portfolio that was envisioned in Chapter 3. The recent adequacy analysis of the EQ R&D portfolio conducted by DOE's Strategic Laboratory Council (SLC) was DOE's first attempt to generate R&D project ideas for the entire EQ R&D portfolio (see Appendix 3 Although the strategic plan inciudecl sections on isotopes, space applications, and basic materials research.
Achieving and Maintaining the Long Term Vision for EQ R&D 75 C). In part because planning has focused primarily on short-term problems, the SLC's adequacy analysis found that the present EQ R&D portfolio does not include a longer-term vision and "strategic elements" and has significant gaps and opportunities (DOE, 20009~. How do these statements fit with the fact that DOE already supports a significant amount of long-term research? The answer lies in the term "strategic," which refers to a plan or method for achieving a goal, including the purposeful allocation of resources. The Office of Science (SC) supports nearly $3 billion in long-term, basic research and scientific user facilities primarily to advance science not to solve EQ problems (or the problems addressed by DOE's other business lines). Thus, SC research is not"strategically' oriented to EQ purposes, although some of it may provide information useful to the EQ mission. EM also supports problem-oriented, longer-term research in its Environmental Management
76 A Strategic Vision for DOE Environmental Quality R&D Science Program (EMSP). Although all the projects EMSP supports are problem-oriented, they do not, nor were they intended to, comprise a coherent, strategic effort at solving particular EQ problems (see http://emsp.em.doe.gov/~. One major reason is that the EMSP budget is small compared to the panoply of scientific problems covered by the program's scope, so that only relatively small, isolated research projects of limited duration are supported. In summary, the present bias of the EQ R&D portfolio toward short-term R&D (DOE, 2000b,g, h) is to be expected given: 1. the way that EM and RW (and to a lesser extent, NE) presently identify R&D needs, 2. EM's goal of closing the maximum number of sites (mostly smaller sites) by 2006, 3. RW's short-term focus on technical issues associated with site recommendation and licensing, 4. the strong emphasis that EM, especially, has placed on getting technologies deployed, and 5. declining EQ R&D budgets. Finding: The existing processes for generating EQ R&D needs are driven largely by DOE's regulatory mandates, contractor incentives, and short-term goals. Conclusion: The existing R&D planning processes are unlikely to generate the full scope of strategic R&D needed to address DOE's most challenging, long-term EQ problems. Recommendation: DOE should establish a new mechanism within its portfolio management process whose purpose is to develop a more strategic EQ R&D portfolio. The new process should supplement and operate in parallel with existing site-driven processes. The primary purpose of the recommended new process, which the committee terms the "Strategic Portfolio Review," would be to identify the gaps and opportunities in the existing portfolio that, when adequately addressed, would encompass the entire spectrum of EQ problems. This Strategic Portfolio Review would be similar to the SLC's adequacy analysis, except that a broader group of experts would participate in the analysis and more explicit criteria that emphasize long-term R&D would be used. Institutionalizing this process is consistent with recent recommendations made by EMAB (DOE, 2000h). Adequacy would be assessed and gaps and opportunities identified by the judgment of a group of knowledgeable, experienced, and collectively (i.e., as a group) unbiased experts, preferably from both within and outside
Achieving and Maintaining the Long Term Vision for EQ R&D 77 the DOE community (how to constitute such a group of experts is discussed more fully in the section "Broadening and Deepening the EQ R&D Portfolios. Gaps and opportunities would be identified using the criteria recommended in Chapter 3 (primarily criteria 1 through 7~. The broader scope of the Strategic Portfolio Review would generate a separate, broader, and deeper source of R&D needs on which to solicit, evaluate, and potentially fund additional projects, with emphasis on those that address the highest-priority EQ problems. An example is long-term stewardship, which raises issues involving policies and time scales beyond those now considered in the present R&D portfolio (NRC, 2000a). This requires that the Strategic Portfolio Review focus on the enduring, most challenging problems needing solution, and not on current activities whether in remediation, waste disposal, or waste management. The expanded set of EQ R&D projects to be considered for funding would consist of projects emerging from the traditional needs processes as well as the new Strategic Portfolio Review. Value: Measuring the Magnitude of the Benefit The gaps and opportunities identified by the Strategic Portfolio Review plus the existing R&D needs processes probably will generate far more demand for R&D activities than can be addressed by current or even greatly expanded resources. Therefore, potential R&D activities will need to be evaluated and prioritized. The goal of valuation is to measure the magnitude of the benefit expected as a result of successful R&D so that scarce resources can be applied to deliver the maximum benefit. Value measures this benefit in quantifiable terms, such as net social benefit or utility. Bjornstadt et al. (2000) have made a strong case for a value-based resource allocation approach for EQ R&D. They suggest risk reduction, cost reduction, and meeting unmet cleanup needs as three components of the potential value of cleanup R&D. They illustrate this approach using a formal non-linear programming model of the Oak Ridge National Laboratory cleanup effort developed for risk analysis (Bjornstadt et al., 1998~. Another application of the approach (though used to prioritize relatively short-term R&D needs) is Kaiser-Hill's work at Rocky Flats to identify, prioritize, and mitigate risks to closure project schedule and cost using what was described as an economic optimization approach to decision-making (Kaiser-Hill, 2000~. Both of these applications stress the importance of being able to quantify and evaluate risk and uncertainty reduction using "value-of- information" techniques, because the result of R&D is frequently better information as well as new technology. The value-of-information metric for allocating both basic and applied research resources also has been recommended by Fischhoff (2000~.
78 A Strategic Vision for DOE Environmental Quality R&D Jenni et al. (1995) discussed an extensive application of a decision support system called the Environmental Restoration Priority System (ERPS). At the heart of ERPS was a multi-attribute utility model, formally elicited from DOE managers, that accounted for six types of benefit: reduced health risks, reduced environmental impacts, reduced adverse socio-economic impacts, compliance with applicable laws and regulatory requirements, reduced ultimate cost of clean-up, and reduced uncertainties relating to risks and costs. Jenni et al. (1995) used a decision-analytic, value-of-information calculation to quantify the benefits of reducing uncertainty, much as the examples discussed above. Benefits 1-4 had explicit dollar value tradeoffs expressed, such as $200 million to eliminate a 1/10 per year risk of death to the maximally exposed individual, allowing the overall benefit to be translated to equivalent dollars. There were seven full-scale applications of the system between 1988 and 1991, which were"praised in technical review, but strongly criticized by stakeholders external to DOE" (Jenni et al., 1995~.4 A conclusion is that rational value-based systems do work and can in fact deliver most of the promised benefits in use in the DOE EQ environment. It is extremely difficult, however, to convince stakeholders and sites that local interests and site-specific needs can be served by a system explicitly designed with national objectives in mind. DOE does not have a method for prioritizing and selecting R&D activities across the entire EQ business line, as each DOE organization that supports EQ R&D activities has its own process. The current process used to prioritize EM's R&D needs (OST's WPRS) resembles ERPS as a multi- attribute scoring system. RW uses a "focused approach" that funds the R&D work required to allow submittal of the site recommendation report and, if the site is selected, the license application to the Nuclear Regulatory Commission. NE considers potential life-cycle cost savings, potential reduction in environmental safety, and health risks, technical viability, and regulatory requirements to prioritize its R&D investments. Because EM supports over 80 percent of the R&D activities within the EQ R&D portfolio, and because the WPRS is a fairly well documented, formalized process, the committee discusses it at some length in the following paragraphs. The purpose of this discussion is to explain why the WPRS currently emphasizes short-term R&D needs, and to suggest ways that the WPRS could be modified to be useful in identifying long-term R&D needs. The ranking system is based on five criteria: 4 It should be recognized that the sites also have reasons for parochialism, as discussed in Chapter 2.
Achieving and Maintaining the Long Term Vision for EQ R&D 79 1. project baseline summary"value" (i.e., a measure of the total life- cycle costs of the baseline technologies to be replaced by a given work package, which is intended to reward those work packages that address high-cost projects and that can be employed at more than one location); 2. future technology deployments (i.e., the number of times the technologies within a work package are expected to be deployed); 3. response to site science and technology needs (i.e., the number of site-identified priority needs addressed by the work package); 4. addressing technical risk (i.e., a measure of the baseline technology's technical risk); and 5. technology cost savings (i.e., a measure of the potential ability of the work package to achieve cost savings compared to baseline technologies). The WPRS offers a number of major benefits relative to earlier methods used in EM, including being based on end-user life-cycle planning data, better understanding of work package benefits, and direct alignment with EM's four corporate performance measures.5 The ranking system has been favorably reviewed by EMAB (DOE, 1999b) and appears to do a good job of concentrating on site needs and deployable technologies. There are, however, several features of WPRS that limit its usefulness as a valuation tool for the types of R&D that are under-represented in the EQ portfolio (DOE, 2000g). Because EM's four corporate performance measures are understandably oriented toward near-term accomplishments, the ranking system inherits that near-term focus. In particular it is directly tied to needs articulated by the sites, who by their nature have a more operational, shorter-term focus; one would not expect them to focus on needs beyond 2006. Also, the primary incentive for most sites is to meet their legal and contractual obligations, so a new technology that offers significant cost reduction but might delay the program is typically unwelcome. There are additional reasons why the WPRS is currently not well suited for evaluating the R&D oriented at strategic R&D. The five criteria included in the ranking system are not as reflective of society's priorities as they are of EM management's performance measures. While this is by design, the six criteria used in the ERPS, for example, are a better reflection of national needs. Other better alternatives would be to use the seven EQ objectives listed in the R&D Portfolio Overview (DOE, 2000i) or the five refined EQ objectives used in the adequacy analysis (DOE, 2000g), including stewardship. A guiding principle for the design of an improved evaluation system should be that it could apply equally well to all areas of the EQ portfolio, not just to EM. 5 (1) number of new technology deployments, (2) life-cycle cost reduction from use of science and technology, (3) number of high-priority needs that are met, and (4) reduction in critical pathway milestones and waste stream technical risk (DOE, 2000n).
80 A Strategic Vision for DOE Environmental Quality R&D
Achieving and Maintaining the Long Term Vision for EQ R&D 81 Findinn: The current Work Package Ranking System is heavily biased toward activities that are site generated and connected to the present baseline plans. Moreover, it is by design EM specific and therefore does not apply to other parts of the EQ R&D portfolio. Conclusion: The current Work Package Ranking System is unlikely to be effective in prioritizing R&D activities designed to address the long- term strategic gaps and opportunities identified in the Strategic Portfolio Review discussed above, especially those not within EM. Recommendation: DOE should develop and implement an evaluation method to address more strategic R&D for the entire EQ R&D portfolio. In the short term, it could be entirely separate from the EM's Work Package Ranking System, but in the longer term a new approach is needed that works for both site-driven and strategy-driven activities and is applied within all areas (i.e., EM, RW, NE) of the EQ R&D portfolio. Several good models for such a system that have been applied to elements of the DOE EQ portfolio but that are not EM specific have been discussed above. Balance: Ensuring Adequate Attention to Diverse Objectives A common experience in life is that the urgent overwhelms the important. It is typical in business R&D organizations that, without strategic guidance, requests for short-term product and process improvements can exhaust the available resources. The results of SLC's adequacy analysis indicate that this is likely true for EQ R&D. Balancing can offset such forces by examining how resources are distributed across various critical dimensions, such as how R&D is distributed across the strategic objectives of the parent organization, across time frames, across risk versus return, or across R&D stages.6 Balance is also important when the potential value of one objective is so large that projects addressing it tend to dominate projects addressing the other objectives, as might be the case for efforts to reduce the cost of some of DOE's most expensive cleanup problems. The diversity of these considerations demands that DOE seek and use the breadth of advice described in the next section. One of the most common balance metrics in business is the relation of technical risk (or the probability of technical success) to return (i.e., value). Allowing DOE to track technical risk and value would be another benefit of 6 For the EM portion of the EQ portfolio, how R&D is distributed across focus areas and sites also may be useful to DOE decision makers.
82 A Strategic Vision for DOE Environmental Quality R&D making probability of technical success an explicit part of the evaluation process. Three of the 10 adequacy criteria (8-10) developed in Chapter 3 pertain to elements of portfolio balance. The most fundamental balance issue is the proportion of the budget that should be allocated to strategic R&D as opposed to R&D driven by short-term needs. There is no simple answer to this question. For example, the appropriate proportion of strategic R&D would be quite different in DOE's Energy Resources R&D portfolio, where nearly all commercialization and deployment is done in the private sector, than in the EQ R&D portfolio, where deployment and application are mostly internal to DOE and its contractors. A number of recent analyses have concluded, however, that more strategic R&D is needed to adequately address DOE's EQ objectives (DOE, 2000g,h). In addition, the SLC's adequacy analysis (DOE, 2000g) examined the funding distribution across the technology maturity spectrum and concluded that it is unbalanced. The committee discusses methods for evaluating EQ R&D funding balance in Chapter 5. INSTITUTIONAL MECHANISMS This section discusses implementation of the recommendations made above and offers additional recommendations related to institutional mechanisms that could be used to make the EQ R&D portfolio more effective in addressing long-term problems, including the personnel needed to carry out the Strategic Portfolio Review and a new approach to long-term EQ R&D that could be added to existing programs. Broadening and Deepening the EQ R&D Portfolio Several reviews of the EQ R&D portfolio have concluded that the portfolio is too narrowly focused on short-term problems and needs a broader perspective to address the most challenging EQ problems and to limit contamination and materials management problems in ongoing and future DOE operations. For the portfolio to adequately address DOE's most challenging EQ problems, the agency must gather input for the Strategic Portfolio Review from a much wider range of people than it customarily involves in its program management (see for example: DOE, 2000h; NRC, 1998, 1999a). To achieve this, several kinds of individuals are needed. Although individual experts are almost by definition narrow in scope, a well- chosen group of informed individuals working together can achieve a very broad perspective. The following categories illustrate by example the range of knowledge individuals can bring so that collectively they match the breadth of the EQ problems to be addressed:
Achieving and Maintaining the Long Term Vision for EQ R&D . robotics); chemistry); 83 practical, problem-oriented, and technically trained experts with relevant experience (e.g., from a relevant industry or a foreign, federal, or state agency with a similar mission); · applied researchers in relevant technologies (e.g., radiation hardening of sensors); applied researchers in generically important technologies (e.g., "basic" researchers in relevant areas (e.g., actinide chemistry); "basic" researchers in generically important sciences (e.g., physical · individuals possessed with a broad, long-term perspective of where technology, science, and/or environmental problems and policies are trending but who are from outside the DOE family of employees, national laboratories, contractors, and others whose interests might appear to represent substantial conflicts of interest; and · technically qualified individuals representing nongovernmental organizations and other stakeholders. Participation must be broadened carefully to ensure the success of the strategic review process. That is, the composition of the group should balance the need for a diversity of expertise with the need for an efficient process. The intent is to select a group of individuals who collectively are not predisposed in favor of the existing portfolio or any particular approach to solving a specific EQ problem (that is, individual biases in the group will be balanced). Consideration should be given to experts from other countries who have the needed expertise because these individuals can bring valuable perspectives to the review and are less likely to have a stake in the outcome. Qualified individuals from outside the program bring a broader perspective and often can look "outside the box" for new approaches. Of course strong input also is needed from DOE staff who have responsibility for accomplishing the EQ mission and special familiarity with the difficulties they face, and who must have the final word if they are to be accountable. The purpose of the strategic review would be to attain the broader input and perspectives that seem lacking (DOE, 2000h). The group should be able to identify the full scope of R&D needed to solve EQ problems and to give rough priority rankings. DOE might chose to experiment with organizational approaches to find the best way to gather and use the information such a group can provide. For example, the group could be established as a new, ongoing EQ R&D advisory board. Another possible approach would be create this group largely from members of EMAB, the Nuclear Waste Technical Review Board, and the Nuclear Energy Research Advisory Committee (see Sidebars 4.1, 4.2, 4.3~. This group would differ from EMAB, the Nuclear Waste Technical Review Board, and the Nuclear Energy Research Advisory Committee in that it would focus on the EQ R&D portfolio, have continuity to see how its recommendations were carried out,
84 A Strategic Vision for DOE Environmental Quality R&D and be much more integrated into and a part of the regular EQ R&D management process. Such a board would report to the manager of the EQ R&D portfolio, and if such a manager does not exist, to the line managers of its components (e.g., the Assistant Secretary for EM, the Director of RW, the Director of NE). It would meet at least annually as needed to develop a strong R&D portfolio. Conclusion: An independent advisory group representing a broad spectrum of expertise and experience is necessary to assure a sustained, high-quality EQ R&D portfolio. Recommendation: DOE should establish an independent planning and review board specifically focused on the EQ R&D portfolio, with membership composed of leaders in the scientific and technical community, including experts from industry, academia, national laboratories, and affected communities. The purpose of this board would be to recommend to DOE management and justify in terms of program and mission a world-class R&D portfolio with the breadth and depth to address EQ problems. Technical Qualifications of Staff The EQ R&D portfolio represents a highly technical activity, and must be managed by a staff with strong technical qualifications (DOE, 2001a). The portfolio management techniques and the independent advisory board recommended above do not reduce the need for strong in-house technical management, because DOE staff still must make the final decisions. It takes considerable technical insight to identify a practical problem in the field and then determine whether current technology can resolve it, and if not, to translate the problem into researchable questions and eventually into R&D projects leading to understandings or technologies to mitigate the problem. EQ R&D managers must make researchers aware of how their work could lead to solutions to critical problems and convince them to pursue such useful results. Conversely, similar insight is needed when examining a proposal for fundamental research to visualize its application to practical problems and to know whether and how it should be funded. Finally, EQ R&D managers must work with operators in the field to implement R&D results. Thus, EQ R&D managers need the technical depth and breadth to span the conceptual range from the field to the lab and back. A DOE staff with such depth and breadth would be able to take advantage of technical advice from outside groups as recommended above. Individuals with such talents are rare, but two approaches can deal with this problem. First, management can partially compensate for the unavoidable limitations of individuals by bringing a broad range of views into
Achieving and Maintaining the Long Term Vision for EQ R&D 85 management processes. Second, because universities generally do not train individuals with such a comprehensive grasp of problems and solutions, DOE might have to adopt institutional approaches to develop them. A recent EMAB report on the role and status of basic science in EM recommended that EM address this issue by developing "operational procedures" for OST staff positions similar to those used by EPA and that OST establish requirements for those positions that reflect their scientific and technical nature (DOE, 2001 a). The new approach to addressing DOE's long-term EQ problems described below also could help develop such people. An Approach to Addressing DOE's Most Challenging, Long-Term EQ Problems Chapter 3 described a set of criteria that could be used to evaluate the adequacy of DOE's EQ R&D portfolio. These same criteria can be used to help design a new approach to EQ R&D that could improve its effectiveness in addressing these long-term currently intractable problems, and which would supplement existing R&D programs. For example, the approach should: · Address critical R&D gaps needed to address EQ goals (and when appropriate, to support the accomplishment of related DOE and national missions). · Encourage the development of alternatives to technologies that are costly, inefficient, or pose high technical risk. · Produce results that could transform the understanding, need, and abilities to address currently intractable problems, thus enabling breakthrough technologies. . Lead to improved performance, reduced human health or environmental risks, decreased cost, and advanced schedules. Help leverage other R&D, such as the Environmental Management Science Program. · Help to narrow and bridge the gap between R&D and application. · Improve the balance of long- versus short-term research. · Involve a diversity of participants from academia, national laboratories, other federal agencies, and the private sector, including students, postdoctoral associates, and other early-career researchers. · Include a balance of annual new starts, extensions of promising R&D, and periodic new initiatives. The committee believes that in order to meet these criteria, a significant fraction of R&D should be conducted in organizationally separate units to help maintain a focus on long-term results. Each of these units would be .
86 A Strategic Vision for DOE Environmental Quality R&D strongly coupled to an important, currently intractable EQ problem and evaluated according to progress on solving the problem, but not strongly coupled to short-term program needs. Based on these general criteria and considerations, the committee arrived at the following finding and recommendation. The committee then describes some of the characteristics of the recommended approach. Findinn: Given the long-term nature of many of DOE's EQ problems, there is a need to develop sustained support for R&D activities to solve such problems. Recommendation: DOE should implement a new approach to provide longer-term funding for organizationally separate, integrated, and coordinated R&D activities (i.e., R&D centers) designed to solve well- defined, high-priority EQ problems. The most important element of the recommended approach is that each R&D centers should focus on providing longer-term support for solving a particular long-term EQ problem, specifically countering the "going out of business within the next decade" philosophy that has permeated some views of the EQ portfolio (see discussion in Chapter 2~. Here it is appropriate to differentiate and clarify what is meant by the phrase "a particular long-term EQ problem" with respect to other concepts, such as "EQ challenge" and "focus area." The committee's term "EQ challenge" (see Chapter 2) refers to the broad challenges facing DOE in its EQ mission area. The management of EM has organized its R&D effort to address some of these challenges into "focus areas" and "crosscutting programs." All of these are very different from what the committee means by a "problem." First, they are much broader and more general. Second, they do not refer to an integrated, coherent effort to solve a problem, but to collections of R&D efforts. Third, focus areas and crosscutting programs sometimes mean problems, but usually mean R&D activities an unfortunate confusion. The universe of problems that might be assigned to R&D centers, taken together, overlaps with the long-term component of all the EQ challenges and of all the focus areas and crosscutting programs. What "problem" means here is an issue. a hindrance to progress, that is appropriate to be 7 The committee refers to the organizations carrying out the integrated and coordinated R&D efforts as R&D centers to indicate that the whole of each is greater than the sum of its parts, i.e., that integration and coordination to focus on a central, often multidisciplinary problem leads to synergy and a holistic solution. This synergy could be achieved through a variety of organizational approaches. For example, the centers could have a virtual aspect, using technology to involve experts at various locations. Thus one sort of balance to be struck is that between the benefits of daily face-to-face collaboration and achievement of critical mass in that sense versus the achievement of a different sort of critical mass by involving many geographically dispersed experts. A center could not be completely virtual, however. For example, there would have to be a locus of coherence, accountability, and problem ownership.
Achieving and Maintaining the Long Term Vision for EQ R&D 87 addressed by a single integrated, coordinated, focused R&D effort of the scale that can be supported realistically. DOE could initiate the new approach by identifying a few well-defined high-priority problems and releasing a set of competitive requests for proposals calling for integrated and coordinated R&D activities to solve each problem. As discussed above, each problem would be a manageable part of the larger EQ challenges of Chapter 2 (i.e., it would represent a barrier to program progress), not just a scientific question. The problems to be addressed could be based on (i.e., perhaps a subset of) the gaps and opportunities identified by the Strategic Portfolio Review. At its core each problem would have at least one unanswered scientific (including social science) or technical question (i.e., this is why R&D is needed) and the centers would pursue these questions in their problem context and in consultation with users, not as pure technical questions. Assigning an R&D center a real-world problem would give it flexibility to choose among technical approaches, indeed to choose more than one if appropriate. Success would be measured in terms of progress in solving the problem. The problems should not be too global (e.g., "reduce the amount of radioactive waste") or too narrow (e.g., "make a particular technology works. Sidebar 4.4 describes an example of a possible type of R&D center based on a recommendation from a recent NRC report (NRC, 2000c). The problem would determine the disciplines and the types of R&D (e.g., fundamental research, applied research, and development) and the number of investigators needed in each R&D center. Most centers probably would be highly multidisciplinary and would involve different types of R&D. For example, a center might include fundamental research, applied research, and perhaps some engineering research to demonstrate the efficacy and practicality of an idea. In addition, centers would be encouraged to involve participants from other agencies and other countries where appropriate. For some problems (e.g., those with high technical risks or of particular importance to EQ mission success), DOE might consider funding more than one center in order to increase the likelihood of success. The R&D center would be expected to frequently consult with and involve its user-clients, which would generate a "technology pull" from them. In a sense, the center thus would become a co-owner of the problem. Large downstream development funding might be needed to achieve application in the field, but the center would take responsibility for seeing its own results applied. The R&D center would thus support a technology's maturation through the development process (e.g., by consulting on problems that arise and perhaps doing some supportive research). In other words, the center would help bridge and narrow the gap between R&D and application (NRC, 2000b). R&D centers also would be encouraged to involve students and postdoctoral fellows to achieve the educational and training function described in Chapter 3 and mentioned above. Finally, the centers would be encouraged to coordinate and cooperate with related R&D activities,
88 A Strategic Vision for DOE Environmental Quality R&D including EMSP and SC projects, work in other agencies, and work in other countries.
Achieving and Maintaining the Long Term Vision for EQ R&D 89 Although the R&D centers would deal with currently intractable problems they would nevertheless be evaluated in terms of problem solution. The centers would be strongly encouraged to seek breakthroughs, even at the cost of some technical risk. To mitigate such risk and improve the probability of overall program success they would be responsible for seeking and developing alternative parallel paths. Each center would be overseen by an independent technical advisory committee familiar with the problem being addressed. Each center would be evaluated regularly on the basis of its progress in solving the assigned problem and overall technical soundness of its R&D. For funding to continue, the center would have to demonstrate first, that it is making progress in solving problems and, second, that it is sound scientifically and technically. For credibility, centers not making adequate progress toward solving EQ problems should be terminated by DOE, not by the Office of Management and Budget or Congress. Those R&D centers making adequate progress could be renewed if the problem remained important. The committee did not examine in detail the funding that might be required for the R&D centers, but based on its members' experience as R&D managers and knowledge of DOE's EQ R&D portfolio, it believes that an appropriate figure for each center would be approximately $1-4 million annually for five years. The suggested funding range is meant to balance at least two considerations: (1 ) given the limited funds available and the desire to start several such centers, each must be small; and (2) on the other hand, each R&D center should be large enough to make progress toward problem solution. A problem calling for multidisciplinary R&D might need a larger R&D center. Such considerations also should help identify the problems to be addressed (i.e., the problems must be of a size appropriate to available funding). Because the approach to EQ R&D recommended here would be new to DOE it should start small and grow only as long as justified by the problems. With the first set of R&D centers well underway, DOE could take steps to enlarge the program by selecting another small set of high-priority problems and repeating the process, fine-tuning the new centers to take advantage of lessons learned. The portfolio of problems addressed would grow as long as problem owners, stakeholders, DOE management, and other decision makers supported such efforts. By this process of continual improvement, EQ could build a portfolio of expertise to apply to its most important problems. Need for Coordination of EQ R&D R&D portfolio management, a recent innovation at DOE, begun in 1999 by its Under Secretary, covers the department's four programmatic business lines, with each having an R&D portfolio. The goal is "to integrate and
go A Strategic Vision for DOE Environmental Quality R&D strengthen the planning, management and administration" of the $8.0 billion DOE R&D enterprise (DOE, 2000i, p.1). The means for achieving this goal are unclear, however. DOE documents do not address how the goal will be achieved with any specificity or depth (DOE, 2000b,i). DOE is aware of this deficiency; both the SLC review of the portfolio (DOE, 2000g) and a recent letter report by EMAB (DOE, 2000h) found that it was a good start but needed to be improved to achieve its goal. As presented by DOE, the portfolio concept itself raises questions about whether it can achieve its stated goal. First, the portfolio is presented as being only a "context" for R&D: the portfolios "have no funding per se; they provide the context within which the funded programs and offices manage and execute their funding" (DOE, 2000i, p. 4~. The R&D portfolios are descriptive tools, and no decision, budget, or priority-setting authority is associated with them. Because of this, no accountability is associated with them. These limitations are common to all the R&D portfolios, including the EQ portfolio. EM, and its Office of Science and Technology, manages the great majority of the EQ R&D portfolio. Consequently, line managers could coordinate this part of the portfolio if given incentives to do so. However, other offices (see Sidebar 1.4) conduct some of the portfolio. In addition, as discussed below, there is much research in the Science portfolio in disciplinary areas of great interest to EQ programs, such as work on the movement of groundwater and on bioremediation. DOE has taken a first, important step toward integrating its R&D programs through portfolio analysis. However, DOE's portfolio concept (i.e., as a context only) offers no way to reach across organizational or portfolio lines to coordinate R&D. The portfolios do little to cross DOE's existing organizational stovepipes. The relationship between EM and SC (which wholly owns DOE's Science portfolio) illustrates the situation. DOE's 2000 strategic plan directs SC to "advance basic research and the instruments of science that are the foundations for DOE's applied missions... to support long-term environmental cleanup and management at DOE sites...." (DOE, 2000f, p.7~. In other words, although it supports applied missions, its research is "basic" (i.e., it looks within science for its research questions and justifications), whereas EQ R&D must address external problems directly. Put another way, SC sees research as an end in itself, but for EQ research is a means to an end. As discussed briefly in Chapter 3, these different world views make cooperation and coordination correspondingly difficult, and unlikely without conscious, continual effort. The Environmental Management Science Program, which is administered jointly by EM and SC, demonstrates that such cooperation and coordination are possible, however. DOE's portfolio approach also cannot compare programs between portfolios. For example, there is no common system for setting priorities or evaluating results. As the preceding example shows, different portfolios
Achieving and Maintaining the Long Term Vision for EQ R&D 91 have very different metrics for success and different definitions of what is a worthwhile problem. Portfolio coordination and management are needed at high and low levels. At higher, strategic levels they are needed to deploy resources on the main problems. At lower, tactical levels they are needed to minimize duplication and overlap, to create synergies, and to ensure stakeholder involvement. All this is another way of saying that there is a need for alignment, value, and balance across and within portfolios, and at both strategic and tactical levels (realizing that because one cannot balance within the smallest program elements, balance is sought among such elements). The generation of the R&D portfolios is a sound accomplishment that might provide a starting point for coordination. However, as presently described, DOE's portfolio management approach seems unlikely to achieve its goals. To be effective the portfolios will need to become a management tool, not just a descriptive tool. That is, the portfolio process would need to include explicit management functions and capabilities, especially accountability. Findinn: There is little evidence of effective coordination within the EQ R&D portfolio (e.g., for communication of results or for recommendations on priorities). Furthermore, there is little evidence of effective coordination between R&D portfolios. Conclusion: At present DOE's R&D portfolio process is unlikely to achieve its goal to integrate and strengthen the planning, management, and administration of its $8 billion R&D enterprise. DOE recognizes that "the portfolio process would benefit from improved coordination and a more integrated approach to ... interportfolio activities" (DOE, 2000b, p. xiii). Although an understatement, this does indicate that the process may improve. Accordingly, the committee is reluctant to make a specific organizational recommendation that might limit DOE's options. In the past, the Under Secretary chaired a group, the R&D Council, whose members included the DOE leadership responsible for each of the four R&D portfolios. Although the status of the R&D Council remains uncertain following the 2000 election, the charter of such a group would allow it to oversee coordination of the EQ portfolio, as well as coordination between portfolios. Because its members have other duties and loyalties, however, such a group alone is an unlikely vehicle for coordination. The committee believes that such a group could serve as a forum for discussion and agreement on plans for coordination developed by the Under Secretary's staff. The larger issue goes beyond specifics to whether DOE intends the portfolios to be more than a context (i.e., whether they should be actively managed). If they are to remain only descriptive (i.e., to reveal problems but
92 ~ Saga ~~n fir DOE Ames/ Out R&D not to address them), some other mesas for ach~v~g the goad of improved R&D management must be Fund.
