|
Items in cart [0] |
Questions? Call 888-624-8373 |
| Copyright © 2009. National Academy of Sciences. All rights reserved. Terms of Use and Privacy Statement |
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 71
4
ACHIEVING AND MAINTAINING THE
LONG-TERM VISION FOR ENVIRONMENTAL
QUALITY RESEARCH AND DEVELOPMENT
Chapter 3 described a vision for a different Environmental Quality (EQ)
research and development (R&D) portfolio that would have a strong, if not
dominant, long-term component. To move towards this vision, the
Department of Energy (DOE) will need to redesign and rebalance its EQ
R&D portfolio in substantial ways to better focus on its long-term EQ
problems.4 This chapter describes a new portfolio management process that
could help achieve these goals.
To be effective, R&D portfolio management must operate within an
effective management system, which includes identifying the decision maker
(or decision-making group) who will make the hard choices of prioritization,
resource allocation, and balance. Portfolio management systems for federal
R&D programs also commonly seek out and use input from broadly qualified
individuals in generating a comprehensive set of R&D needs and project
possibilities. In general, the generation and selection of R&D projects should
have inputs from qualified persons both inside and outside the program.
This chapter discusses several institutional mechanisms that DOE could
use to improve the management of its EQ R&D portfolio, including ways to
generate and incorporate such input.
For the most part DOE can implement the recommended new portfolio
management process through an evolutionary approach (i.e., by modifying
and supplementing existing management processes). The committee
believes this is possible because DOE is already using portfolio
management techniques (DOE, 2000b,g), and external reviews have found
that management based on these techniques is yielding positive results but
could be greatly improved (DOE, 2000h). Such an approach avoids
disruptive reorganizations and maintains management focus on the goal
(i.e., realizing the new R&D vision).
' As noted previously, the term "EQ problems" refers to the set of technical problems that
collectively make up the EQ challenges described in Chapter 2. This is a useful concept in
planning an R&D portfolio, because the challenges are very broad, and must be broken down
into manageable parts to be addressed by R&D.
71
OCR for page 72
72
A Strategic Vision for DOE Environmental Quality R&D
R&D PORTFOLIO MANAGEMENT PROCESS
The primary objective of portfolio management is to ensure that an R&D
portfolio is aligned, valuable, and balanced. "Alignment" is intended to
ensure that the portfolio supports the strategic objectives and strategic
direction of the parent organization (i.e., DOE's EQ mission and objectives).
"Value" measures that support in quantifiable terms, such as net social
benefit or utility. "Balance" examines whether the portfolio covers the full
scope of objectives and approaches or is too narrowly focused on certain
categories of R&D, time frames, or topics.
In practice, these three objectives are often treated in sequence. An
alignment process typically generates a list of R&D "possibilities" to be
considered for funding by a decision maker. The adequacy analysis that
identified the extensive list of R&D gaps and opportunities (see Appendix C)
was essentially an alignment exercise. A valuation process is one means of
prioritizing the list of R&D possibilities so that scarce resources can be
applied to deliver the maximum benefit. The Work Package Ranking
System (WPRS) that is currently used to select R&D work packages within
the Office of Environmental Management (EM) has many similarities to a
value-based prioritization system.2 Balancing a portfolio is a formal process
for examining and considering how resources are distributed across critical
dimensions and is applied after valuation to offset any imbalances that are
inconsistent with overall program objectives. Examples of the types of
displays that can be used to evaluate balance include diagrams displaying
funding distribution across R&D maturity (Figure 2-6 of DOE, 2000b) and
the levels of involvement of universities, national laboratories, contractors,
and industries at various stages of R&D maturity (Figure 2-7 of DOE,
2000b). In the following sections, the committee discusses DOE's EQ R&D
portfolio management processes in terms of the objectives of alignment,
value, and balance.
Alignment: Generating Improved Project Ideas
Each of the DOE organizations that support EQ R&D has its own
process for generating R&D project ideas. These planning processes are
designed primarily to gather site and repository needs, which tend to be
focused on short-term problems, and to turn these into R&D projects. For
example, the participants who determine EM's site needs typically are DOE
employees and contractors who are closely involved with the site problems
and issues (NRC, 1999ay, with some periodic input from the broader
2 DOE does not have a single evaluation method for prioritizing R&D activities across the
entire EQ R&D portfolio. Each organization that supports EQ R&D activities has its own process
for prioritizing and selecting R&D projects.
OCR for page 73
Achieving and Maintaining the Long Term Vision for EQ R&D
73
technical community, such as from the Environmental Management
Advisory Board (EMAB, see Sidebar 4.1~. The R&D activities supported by
the Office of Civilian Radioactive Waste Management (RW) are identified
primarily by DOE staff and contractors at the Yucca Mountain Site, although
OCR for page 74
74
A Strategic Vision for DOE Environmental Quality R&D
the Nuclear Waste Technical Review Board also plays a role in identifying
science and technology needs for the RW program (see Sidebar 4.2~. The
Office of Nuclear Energy, Science and Technology (NE) relies on its Nuclear
Energy Research Advisory Committee to generate long-term R&D needs
(see Sidebar 4.3), although these needs are primarily directed towards
nuclear power R&D3 (and hence DOE's Energy Resources R&D portfolio),
because that is NE's overall programmatic focus. In addition, as discussed
in Chapter 2, both EM and RW are driven by short-term milestones and
deadlines. The short-term drivers and the limited set of participants work
together to limit the development of the broad R&D portfolio that was
envisioned in Chapter 3.
The recent adequacy analysis of the EQ R&D portfolio conducted by
DOE's Strategic Laboratory Council (SLC) was DOE's first attempt to
generate R&D project ideas for the entire EQ R&D portfolio (see Appendix
3 Although the strategic plan inciudecl sections on isotopes, space applications, and basic
materials research.
OCR for page 75
Achieving and Maintaining the Long Term Vision for EQ R&D
75
C). In part because planning has focused primarily on short-term problems,
the SLC's adequacy analysis found that the present EQ R&D portfolio does
not include a longer-term vision and "strategic elements" and has significant
gaps and opportunities (DOE, 20009~. How do these statements fit with the
fact that DOE already supports a significant amount of long-term research?
The answer lies in the term "strategic," which refers to a plan or method for
achieving a goal, including the purposeful allocation of resources. The Office
of Science (SC) supports nearly $3 billion in long-term, basic research and
scientific user facilities primarily to advance science not to solve EQ
problems (or the problems addressed by DOE's other business lines). Thus,
SC research is not"strategically' oriented to EQ purposes, although some
of it may provide information useful to the EQ mission. EM also supports
problem-oriented, longer-term research in its Environmental Management
OCR for page 76
76
A Strategic Vision for DOE Environmental Quality R&D
Science Program (EMSP). Although all the projects EMSP supports are
problem-oriented, they do not, nor were they intended to, comprise a
coherent, strategic effort at solving particular EQ problems (see
http://emsp.em.doe.gov/~. One major reason is that the EMSP budget is
small compared to the panoply of scientific problems covered by the
program's scope, so that only relatively small, isolated research projects of
limited duration are supported.
In summary, the present bias of the EQ R&D portfolio toward short-term
R&D (DOE, 2000b,g, h) is to be expected given:
1. the way that EM and RW (and to a lesser extent, NE) presently
identify R&D needs,
2. EM's goal of closing the maximum number of sites (mostly smaller
sites) by 2006,
3. RW's short-term focus on technical issues associated with site
recommendation and licensing,
4. the strong emphasis that EM, especially, has placed on getting
technologies deployed, and
5. declining EQ R&D budgets.
Finding: The existing processes for generating EQ R&D needs are
driven largely by DOE's regulatory mandates, contractor incentives,
and short-term goals.
Conclusion: The existing R&D planning processes are unlikely to
generate the full scope of strategic R&D needed to address DOE's
most challenging, long-term EQ problems.
Recommendation: DOE should establish a new mechanism within its
portfolio management process whose purpose is to develop a more
strategic EQ R&D portfolio. The new process should supplement and
operate in parallel with existing site-driven processes.
The primary purpose of the recommended new process, which the
committee terms the "Strategic Portfolio Review," would be to identify the
gaps and opportunities in the existing portfolio that, when adequately
addressed, would encompass the entire spectrum of EQ problems. This
Strategic Portfolio Review would be similar to the SLC's adequacy analysis,
except that a broader group of experts would participate in the analysis and
more explicit criteria that emphasize long-term R&D would be used.
Institutionalizing this process is consistent with recent recommendations
made by EMAB (DOE, 2000h).
Adequacy would be assessed and gaps and opportunities identified by
the judgment of a group of knowledgeable, experienced, and collectively
(i.e., as a group) unbiased experts, preferably from both within and outside
OCR for page 77
Achieving and Maintaining the Long Term Vision for EQ R&D
77
the DOE community (how to constitute such a group of experts is discussed
more fully in the section "Broadening and Deepening the EQ R&D
Portfolios. Gaps and opportunities would be identified using the criteria
recommended in Chapter 3 (primarily criteria 1 through 7~. The broader
scope of the Strategic Portfolio Review would generate a separate, broader,
and deeper source of R&D needs on which to solicit, evaluate, and
potentially fund additional projects, with emphasis on those that address the
highest-priority EQ problems. An example is long-term stewardship, which
raises issues involving policies and time scales beyond those now
considered in the present R&D portfolio (NRC, 2000a). This requires that
the Strategic Portfolio Review focus on the enduring, most challenging
problems needing solution, and not on current activities whether in
remediation, waste disposal, or waste management. The expanded set of
EQ R&D projects to be considered for funding would consist of projects
emerging from the traditional needs processes as well as the new Strategic
Portfolio Review.
Value: Measuring the Magnitude of the Benefit
The gaps and opportunities identified by the Strategic Portfolio Review
plus the existing R&D needs processes probably will generate far more
demand for R&D activities than can be addressed by current or even greatly
expanded resources. Therefore, potential R&D activities will need to be
evaluated and prioritized. The goal of valuation is to measure the magnitude
of the benefit expected as a result of successful R&D so that scarce
resources can be applied to deliver the maximum benefit. Value measures
this benefit in quantifiable terms, such as net social benefit or utility.
Bjornstadt et al. (2000) have made a strong case for a value-based
resource allocation approach for EQ R&D. They suggest risk reduction, cost
reduction, and meeting unmet cleanup needs as three components of the
potential value of cleanup R&D. They illustrate this approach using a formal
non-linear programming model of the Oak Ridge National Laboratory
cleanup effort developed for risk analysis (Bjornstadt et al., 1998~. Another
application of the approach (though used to prioritize relatively short-term
R&D needs) is Kaiser-Hill's work at Rocky Flats to identify, prioritize, and
mitigate risks to closure project schedule and cost using what was
described as an economic optimization approach to decision-making
(Kaiser-Hill, 2000~. Both of these applications stress the importance of being
able to quantify and evaluate risk and uncertainty reduction using "value-of-
information" techniques, because the result of R&D is frequently better
information as well as new technology. The value-of-information metric for
allocating both basic and applied research resources also has been
recommended by Fischhoff (2000~.
OCR for page 78
78
A Strategic Vision for DOE Environmental Quality R&D
Jenni et al. (1995) discussed an extensive application of a decision
support system called the Environmental Restoration Priority System
(ERPS). At the heart of ERPS was a multi-attribute utility model, formally
elicited from DOE managers, that accounted for six types of benefit:
reduced health risks,
reduced environmental impacts,
reduced adverse socio-economic impacts,
compliance with applicable laws and regulatory requirements,
reduced ultimate cost of clean-up, and
reduced uncertainties relating to risks and costs.
Jenni et al. (1995) used a decision-analytic, value-of-information
calculation to quantify the benefits of reducing uncertainty, much as the
examples discussed above. Benefits 1-4 had explicit dollar value tradeoffs
expressed, such as $200 million to eliminate a 1/10 per year risk of death
to the maximally exposed individual, allowing the overall benefit to be
translated to equivalent dollars. There were seven full-scale applications of
the system between 1988 and 1991, which were"praised in technical
review, but strongly criticized by stakeholders external to DOE" (Jenni et al.,
1995~.4 A conclusion is that rational value-based systems do work and can
in fact deliver most of the promised benefits in use in the DOE EQ
environment. It is extremely difficult, however, to convince stakeholders and
sites that local interests and site-specific needs can be served by a system
explicitly designed with national objectives in mind.
DOE does not have a method for prioritizing and selecting R&D
activities across the entire EQ business line, as each DOE organization that
supports EQ R&D activities has its own process. The current process used
to prioritize EM's R&D needs (OST's WPRS) resembles ERPS as a multi-
attribute scoring system. RW uses a "focused approach" that funds the R&D
work required to allow submittal of the site recommendation report and, if
the site is selected, the license application to the Nuclear Regulatory
Commission. NE considers potential life-cycle cost savings, potential
reduction in environmental safety, and health risks, technical viability, and
regulatory requirements to prioritize its R&D investments.
Because EM supports over 80 percent of the R&D activities within the
EQ R&D portfolio, and because the WPRS is a fairly well documented,
formalized process, the committee discusses it at some length in the
following paragraphs. The purpose of this discussion is to explain why the
WPRS currently emphasizes short-term R&D needs, and to suggest ways
that the WPRS could be modified to be useful in identifying long-term R&D
needs. The ranking system is based on five criteria:
4 It should be recognized that the sites also have reasons for parochialism, as discussed
in Chapter 2.
OCR for page 79
Achieving and Maintaining the Long Term Vision for EQ R&D
79
1. project baseline summary"value" (i.e., a measure of the total life-
cycle costs of the baseline technologies to be replaced by a given work
package, which is intended to reward those work packages that address
high-cost projects and that can be employed at more than one location);
2. future technology deployments (i.e., the number of times the
technologies within a work package are expected to be deployed);
3. response to site science and technology needs (i.e., the number of
site-identified priority needs addressed by the work package);
4. addressing technical risk (i.e., a measure of the baseline
technology's technical risk); and
5. technology cost savings (i.e., a measure of the potential ability of
the work package to achieve cost savings compared to baseline
technologies).
The WPRS offers a number of major benefits relative to earlier methods
used in EM, including being based on end-user life-cycle planning data,
better understanding of work package benefits, and direct alignment with
EM's four corporate performance measures.5 The ranking system has been
favorably reviewed by EMAB (DOE, 1999b) and appears to do a good job
of concentrating on site needs and deployable technologies.
There are, however, several features of WPRS that limit its usefulness
as a valuation tool for the types of R&D that are under-represented in the
EQ portfolio (DOE, 2000g). Because EM's four corporate performance
measures are understandably oriented toward near-term accomplishments,
the ranking system inherits that near-term focus. In particular it is directly
tied to needs articulated by the sites, who by their nature have a more
operational, shorter-term focus; one would not expect them to focus on
needs beyond 2006. Also, the primary incentive for most sites is to meet
their legal and contractual obligations, so a new technology that offers
significant cost reduction but might delay the program is typically
unwelcome.
There are additional reasons why the WPRS is currently not well suited
for evaluating the R&D oriented at strategic R&D. The five criteria included
in the ranking system are not as reflective of society's priorities as they are
of EM management's performance measures. While this is by design, the
six criteria used in the ERPS, for example, are a better reflection of national
needs. Other better alternatives would be to use the seven EQ objectives
listed in the R&D Portfolio Overview (DOE, 2000i) or the five refined EQ
objectives used in the adequacy analysis (DOE, 2000g), including
stewardship. A guiding principle for the design of an improved evaluation
system should be that it could apply equally well to all areas of the EQ
portfolio, not just to EM.
5 (1) number of new technology deployments, (2) life-cycle cost reduction from use of
science and technology, (3) number of high-priority needs that are met, and (4) reduction in
critical pathway milestones and waste stream technical risk (DOE, 2000n).
OCR for page 80
80
A Strategic Vision for DOE Environmental Quality R&D
OCR for page 81
Achieving and Maintaining the Long Term Vision for EQ R&D
81
Findinn: The current Work Package Ranking System is heavily biased
toward activities that are site generated and connected to the present
baseline plans. Moreover, it is by design EM specific and therefore
does not apply to other parts of the EQ R&D portfolio.
Conclusion: The current Work Package Ranking System is unlikely to
be effective in prioritizing R&D activities designed to address the long-
term strategic gaps and opportunities identified in the Strategic
Portfolio Review discussed above, especially those not within EM.
Recommendation: DOE should develop and implement an evaluation
method to address more strategic R&D for the entire EQ R&D portfolio.
In the short term, it could be entirely separate from the EM's Work
Package Ranking System, but in the longer term a new approach is
needed that works for both site-driven and strategy-driven activities
and is applied within all areas (i.e., EM, RW, NE) of the EQ R&D
portfolio.
Several good models for such a system that have been applied to
elements of the DOE EQ portfolio but that are not EM specific have been
discussed above.
Balance: Ensuring Adequate Attention to Diverse Objectives
A common experience in life is that the urgent overwhelms the
important. It is typical in business R&D organizations that, without strategic
guidance, requests for short-term product and process improvements can
exhaust the available resources. The results of SLC's adequacy analysis
indicate that this is likely true for EQ R&D. Balancing can offset such forces
by examining how resources are distributed across various critical
dimensions, such as how R&D is distributed across the strategic objectives
of the parent organization, across time frames, across risk versus return, or
across R&D stages.6 Balance is also important when the potential value of
one objective is so large that projects addressing it tend to dominate
projects addressing the other objectives, as might be the case for efforts to
reduce the cost of some of DOE's most expensive cleanup problems. The
diversity of these considerations demands that DOE seek and use the
breadth of advice described in the next section.
One of the most common balance metrics in business is the relation of
technical risk (or the probability of technical success) to return (i.e., value).
Allowing DOE to track technical risk and value would be another benefit of
6 For the EM portion of the EQ portfolio, how R&D is distributed across focus areas and
sites also may be useful to DOE decision makers.
OCR for page 82
82
A Strategic Vision for DOE Environmental Quality R&D
making probability of technical success an explicit part of the evaluation
process.
Three of the 10 adequacy criteria (8-10) developed in Chapter 3 pertain
to elements of portfolio balance. The most fundamental balance issue is the
proportion of the budget that should be allocated to strategic R&D as
opposed to R&D driven by short-term needs. There is no simple answer to
this question. For example, the appropriate proportion of strategic R&D
would be quite different in DOE's Energy Resources R&D portfolio, where
nearly all commercialization and deployment is done in the private sector,
than in the EQ R&D portfolio, where deployment and application are mostly
internal to DOE and its contractors. A number of recent analyses have
concluded, however, that more strategic R&D is needed to adequately
address DOE's EQ objectives (DOE, 2000g,h). In addition, the SLC's
adequacy analysis (DOE, 2000g) examined the funding distribution across
the technology maturity spectrum and concluded that it is unbalanced. The
committee discusses methods for evaluating EQ R&D funding balance in
Chapter 5.
INSTITUTIONAL MECHANISMS
This section discusses implementation of the recommendations made
above and offers additional recommendations related to institutional
mechanisms that could be used to make the EQ R&D portfolio more
effective in addressing long-term problems, including the personnel needed
to carry out the Strategic Portfolio Review and a new approach to long-term
EQ R&D that could be added to existing programs.
Broadening and Deepening the EQ R&D Portfolio
Several reviews of the EQ R&D portfolio have concluded that the
portfolio is too narrowly focused on short-term problems and needs a
broader perspective to address the most challenging EQ problems and to
limit contamination and materials management problems in ongoing and
future DOE operations. For the portfolio to adequately address DOE's most
challenging EQ problems, the agency must gather input for the Strategic
Portfolio Review from a much wider range of people than it customarily
involves in its program management (see for example: DOE, 2000h; NRC,
1998, 1999a). To achieve this, several kinds of individuals are needed.
Although individual experts are almost by definition narrow in scope, a well-
chosen group of informed individuals working together can achieve a very
broad perspective. The following categories illustrate by example the range
of knowledge individuals can bring so that collectively they match the
breadth of the EQ problems to be addressed:
OCR for page 83
Achieving and Maintaining the Long Term Vision for EQ R&D
.
robotics);
chemistry);
83
practical, problem-oriented, and technically trained experts with
relevant experience (e.g., from a relevant industry or a foreign, federal, or
state agency with a similar mission);
· applied researchers in relevant technologies (e.g., radiation
hardening of sensors);
applied researchers in generically important technologies (e.g.,
"basic" researchers in relevant areas (e.g., actinide chemistry);
"basic" researchers in generically important sciences (e.g., physical
· individuals possessed with a broad, long-term perspective of where
technology, science, and/or environmental problems and policies are
trending but who are from outside the DOE family of employees, national
laboratories, contractors, and others whose interests might appear to
represent substantial conflicts of interest; and
· technically qualified individuals representing nongovernmental
organizations and other stakeholders.
Participation must be broadened carefully to ensure the success of the
strategic review process. That is, the composition of the group should
balance the need for a diversity of expertise with the need for an efficient
process. The intent is to select a group of individuals who collectively are not
predisposed in favor of the existing portfolio or any particular approach to
solving a specific EQ problem (that is, individual biases in the group will be
balanced). Consideration should be given to experts from other countries
who have the needed expertise because these individuals can bring
valuable perspectives to the review and are less likely to have a stake in the
outcome. Qualified individuals from outside the program bring a broader
perspective and often can look "outside the box" for new approaches. Of
course strong input also is needed from DOE staff who have responsibility
for accomplishing the EQ mission and special familiarity with the difficulties
they face, and who must have the final word if they are to be accountable.
The purpose of the strategic review would be to attain the broader input
and perspectives that seem lacking (DOE, 2000h). The group should be
able to identify the full scope of R&D needed to solve EQ problems and to
give rough priority rankings. DOE might chose to experiment with
organizational approaches to find the best way to gather and use the
information such a group can provide. For example, the group could be
established as a new, ongoing EQ R&D advisory board. Another possible
approach would be create this group largely from members of EMAB, the
Nuclear Waste Technical Review Board, and the Nuclear Energy Research
Advisory Committee (see Sidebars 4.1, 4.2, 4.3~. This group would differ
from EMAB, the Nuclear Waste Technical Review Board, and the Nuclear
Energy Research Advisory Committee in that it would focus on the EQ R&D
portfolio, have continuity to see how its recommendations were carried out,
OCR for page 84
84
A Strategic Vision for DOE Environmental Quality R&D
and be much more integrated into and a part of the regular EQ R&D
management process. Such a board would report to the manager of the EQ
R&D portfolio, and if such a manager does not exist, to the line managers
of its components (e.g., the Assistant Secretary for EM, the Director of RW,
the Director of NE). It would meet at least annually as needed to develop a
strong R&D portfolio.
Conclusion: An independent advisory group representing a broad
spectrum of expertise and experience is necessary to assure a
sustained, high-quality EQ R&D portfolio.
Recommendation: DOE should establish an independent planning and
review board specifically focused on the EQ R&D portfolio, with
membership composed of leaders in the scientific and technical
community, including experts from industry, academia, national
laboratories, and affected communities. The purpose of this board
would be to recommend to DOE management and justify in terms of
program and mission a world-class R&D portfolio with the breadth and
depth to address EQ problems.
Technical Qualifications of Staff
The EQ R&D portfolio represents a highly technical activity, and must
be managed by a staff with strong technical qualifications (DOE, 2001a).
The portfolio management techniques and the independent advisory board
recommended above do not reduce the need for strong in-house technical
management, because DOE staff still must make the final decisions. It takes
considerable technical insight to identify a practical problem in the field and
then determine whether current technology can resolve it, and if not, to
translate the problem into researchable questions and eventually into R&D
projects leading to understandings or technologies to mitigate the problem.
EQ R&D managers must make researchers aware of how their work could
lead to solutions to critical problems and convince them to pursue such
useful results. Conversely, similar insight is needed when examining a
proposal for fundamental research to visualize its application to practical
problems and to know whether and how it should be funded. Finally, EQ
R&D managers must work with operators in the field to implement R&D
results. Thus, EQ R&D managers need the technical depth and breadth to
span the conceptual range from the field to the lab and back. A DOE staff
with such depth and breadth would be able to take advantage of technical
advice from outside groups as recommended above.
Individuals with such talents are rare, but two approaches can deal with
this problem. First, management can partially compensate for the
unavoidable limitations of individuals by bringing a broad range of views into
OCR for page 85
Achieving and Maintaining the Long Term Vision for EQ R&D
85
management processes. Second, because universities generally do not
train individuals with such a comprehensive grasp of problems and
solutions, DOE might have to adopt institutional approaches to develop
them. A recent EMAB report on the role and status of basic science in EM
recommended that EM address this issue by developing "operational
procedures" for OST staff positions similar to those used by EPA and that
OST establish requirements for those positions that reflect their scientific
and technical nature (DOE, 2001 a). The new approach to addressing DOE's
long-term EQ problems described below also could help develop such
people.
An Approach to Addressing DOE's Most Challenging, Long-Term
EQ Problems
Chapter 3 described a set of criteria that could be used to evaluate the
adequacy of DOE's EQ R&D portfolio. These same criteria can be used to
help design a new approach to EQ R&D that could improve its effectiveness
in addressing these long-term currently intractable problems, and which
would supplement existing R&D programs. For example, the approach
should:
· Address critical R&D gaps needed to address EQ goals (and when
appropriate, to support the accomplishment of related DOE and national
missions).
· Encourage the development of alternatives to technologies that are
costly, inefficient, or pose high technical risk.
· Produce results that could transform the understanding, need, and
abilities to address currently intractable problems, thus enabling
breakthrough technologies.
. Lead to improved performance, reduced human health or
environmental risks, decreased cost, and advanced schedules.
Help leverage other R&D, such as the Environmental Management
Science Program.
· Help to narrow and bridge the gap between R&D and application.
· Improve the balance of long- versus short-term research.
· Involve a diversity of participants from academia, national
laboratories, other federal agencies, and the private sector, including
students, postdoctoral associates, and other early-career researchers.
· Include a balance of annual new starts, extensions of promising
R&D, and periodic new initiatives.
The committee believes that in order to meet these criteria, a significant
fraction of R&D should be conducted in organizationally separate units to
help maintain a focus on long-term results. Each of these units would be
.
OCR for page 86
86
A Strategic Vision for DOE Environmental Quality R&D
strongly coupled to an important, currently intractable EQ problem and
evaluated according to progress on solving the problem, but not strongly
coupled to short-term program needs. Based on these general criteria and
considerations, the committee arrived at the following finding and
recommendation. The committee then describes some of the characteristics
of the recommended approach.
Findinn: Given the long-term nature of many of DOE's EQ problems,
there is a need to develop sustained support for R&D activities to
solve such problems.
Recommendation: DOE should implement a new approach to provide
longer-term funding for organizationally separate, integrated, and
coordinated R&D activities (i.e., R&D centers) designed to solve well-
defined, high-priority EQ problems.
The most important element of the recommended approach is that each
R&D centers should focus on providing longer-term support for solving a
particular long-term EQ problem, specifically countering the "going out of
business within the next decade" philosophy that has permeated some
views of the EQ portfolio (see discussion in Chapter 2~. Here it is
appropriate to differentiate and clarify what is meant by the phrase "a
particular long-term EQ problem" with respect to other concepts, such as
"EQ challenge" and "focus area." The committee's term "EQ challenge" (see
Chapter 2) refers to the broad challenges facing DOE in its EQ mission
area. The management of EM has organized its R&D effort to address some
of these challenges into "focus areas" and "crosscutting programs." All of
these are very different from what the committee means by a "problem."
First, they are much broader and more general. Second, they do not refer
to an integrated, coherent effort to solve a problem, but to collections of
R&D efforts. Third, focus areas and crosscutting programs sometimes
mean problems, but usually mean R&D activities an unfortunate confusion.
The universe of problems that might be assigned to R&D centers, taken
together, overlaps with the long-term component of all the EQ challenges
and of all the focus areas and crosscutting programs. What "problem"
means here is an issue. a hindrance to progress, that is appropriate to be
7 The committee refers to the organizations carrying out the integrated and coordinated
R&D efforts as R&D centers to indicate that the whole of each is greater than the sum of its
parts, i.e., that integration and coordination to focus on a central, often multidisciplinary problem
leads to synergy and a holistic solution. This synergy could be achieved through a variety of
organizational approaches. For example, the centers could have a virtual aspect, using
technology to involve experts at various locations. Thus one sort of balance to be struck is that
between the benefits of daily face-to-face collaboration and achievement of critical mass in that
sense versus the achievement of a different sort of critical mass by involving many
geographically dispersed experts. A center could not be completely virtual, however. For
example, there would have to be a locus of coherence, accountability, and problem ownership.
OCR for page 87
Achieving and Maintaining the Long Term Vision for EQ R&D
87
addressed by a single integrated, coordinated, focused R&D effort of the
scale that can be supported realistically.
DOE could initiate the new approach by identifying a few well-defined
high-priority problems and releasing a set of competitive requests for
proposals calling for integrated and coordinated R&D activities to solve each
problem. As discussed above, each problem would be a manageable part
of the larger EQ challenges of Chapter 2 (i.e., it would represent a barrier to
program progress), not just a scientific question. The problems to be
addressed could be based on (i.e., perhaps a subset of) the gaps and
opportunities identified by the Strategic Portfolio Review. At its core each
problem would have at least one unanswered scientific (including social
science) or technical question (i.e., this is why R&D is needed) and the
centers would pursue these questions in their problem context and in
consultation with users, not as pure technical questions. Assigning an R&D
center a real-world problem would give it flexibility to choose among
technical approaches, indeed to choose more than one if appropriate.
Success would be measured in terms of progress in solving the problem.
The problems should not be too global (e.g., "reduce the amount of
radioactive waste") or too narrow (e.g., "make a particular technology
works. Sidebar 4.4 describes an example of a possible type of R&D center
based on a recommendation from a recent NRC report (NRC, 2000c).
The problem would determine the disciplines and the types of R&D
(e.g., fundamental research, applied research, and development) and the
number of investigators needed in each R&D center. Most centers probably
would be highly multidisciplinary and would involve different types of R&D.
For example, a center might include fundamental research, applied
research, and perhaps some engineering research to demonstrate the
efficacy and practicality of an idea. In addition, centers would be encouraged
to involve participants from other agencies and other countries where
appropriate. For some problems (e.g., those with high technical risks or of
particular importance to EQ mission success), DOE might consider funding
more than one center in order to increase the likelihood of success.
The R&D center would be expected to frequently consult with and
involve its user-clients, which would generate a "technology pull" from them.
In a sense, the center thus would become a co-owner of the problem. Large
downstream development funding might be needed to achieve application
in the field, but the center would take responsibility for seeing its own results
applied. The R&D center would thus support a technology's maturation
through the development process (e.g., by consulting on problems that arise
and perhaps doing some supportive research). In other words, the center
would help bridge and narrow the gap between R&D and application (NRC,
2000b). R&D centers also would be encouraged to involve students and
postdoctoral fellows to achieve the educational and training function
described in Chapter 3 and mentioned above. Finally, the centers would be
encouraged to coordinate and cooperate with related R&D activities,
OCR for page 88
88
A Strategic Vision for DOE Environmental Quality R&D
including EMSP and SC projects, work in other agencies, and work in other
countries.
OCR for page 89
Achieving and Maintaining the Long Term Vision for EQ R&D
89
Although the R&D centers would deal with currently intractable
problems they would nevertheless be evaluated in terms of problem
solution. The centers would be strongly encouraged to seek breakthroughs,
even at the cost of some technical risk. To mitigate such risk and improve
the probability of overall program success they would be responsible for
seeking and developing alternative parallel paths. Each center would be
overseen by an independent technical advisory committee familiar with the
problem being addressed. Each center would be evaluated regularly on the
basis of its progress in solving the assigned problem and overall technical
soundness of its R&D. For funding to continue, the center would have to
demonstrate first, that it is making progress in solving problems and,
second, that it is sound scientifically and technically. For credibility, centers
not making adequate progress toward solving EQ problems should be
terminated by DOE, not by the Office of Management and Budget or
Congress. Those R&D centers making adequate progress could be
renewed if the problem remained important.
The committee did not examine in detail the funding that might be
required for the R&D centers, but based on its members' experience as
R&D managers and knowledge of DOE's EQ R&D portfolio, it believes that
an appropriate figure for each center would be approximately $1-4 million
annually for five years. The suggested funding range is meant to balance at
least two considerations: (1 ) given the limited funds available and the desire
to start several such centers, each must be small; and (2) on the other
hand, each R&D center should be large enough to make progress toward
problem solution. A problem calling for multidisciplinary R&D might need a
larger R&D center. Such considerations also should help identify the
problems to be addressed (i.e., the problems must be of a size appropriate
to available funding).
Because the approach to EQ R&D recommended here would be new
to DOE it should start small and grow only as long as justified by the
problems. With the first set of R&D centers well underway, DOE could take
steps to enlarge the program by selecting another small set of high-priority
problems and repeating the process, fine-tuning the new centers to take
advantage of lessons learned. The portfolio of problems addressed would
grow as long as problem owners, stakeholders, DOE management, and
other decision makers supported such efforts. By this process of continual
improvement, EQ could build a portfolio of expertise to apply to its most
important problems.
Need for Coordination of EQ R&D
R&D portfolio management, a recent innovation at DOE, begun in 1999
by its Under Secretary, covers the department's four programmatic business
lines, with each having an R&D portfolio. The goal is "to integrate and
OCR for page 90
go
A Strategic Vision for DOE Environmental Quality R&D
strengthen the planning, management and administration" of the $8.0 billion
DOE R&D enterprise (DOE, 2000i, p.1). The means for achieving this goal
are unclear, however. DOE documents do not address how the goal will be
achieved with any specificity or depth (DOE, 2000b,i). DOE is aware of this
deficiency; both the SLC review of the portfolio (DOE, 2000g) and a recent
letter report by EMAB (DOE, 2000h) found that it was a good start but
needed to be improved to achieve its goal.
As presented by DOE, the portfolio concept itself raises questions about
whether it can achieve its stated goal. First, the portfolio is presented as
being only a "context" for R&D: the portfolios "have no funding per se; they
provide the context within which the funded programs and offices manage
and execute their funding" (DOE, 2000i, p. 4~. The R&D portfolios are
descriptive tools, and no decision, budget, or priority-setting authority is
associated with them. Because of this, no accountability is associated with
them. These limitations are common to all the R&D portfolios, including the
EQ portfolio.
EM, and its Office of Science and Technology, manages the great
majority of the EQ R&D portfolio. Consequently, line managers could
coordinate this part of the portfolio if given incentives to do so. However,
other offices (see Sidebar 1.4) conduct some of the portfolio. In addition, as
discussed below, there is much research in the Science portfolio in
disciplinary areas of great interest to EQ programs, such as work on the
movement of groundwater and on bioremediation.
DOE has taken a first, important step toward integrating its R&D
programs through portfolio analysis. However, DOE's portfolio concept (i.e.,
as a context only) offers no way to reach across organizational or portfolio
lines to coordinate R&D. The portfolios do little to cross DOE's existing
organizational stovepipes. The relationship between EM and SC (which
wholly owns DOE's Science portfolio) illustrates the situation. DOE's 2000
strategic plan directs SC to "advance basic research and the instruments of
science that are the foundations for DOE's applied missions... to support
long-term environmental cleanup and management at DOE sites...." (DOE,
2000f, p.7~. In other words, although it supports applied missions, its
research is "basic" (i.e., it looks within science for its research questions and
justifications), whereas EQ R&D must address external problems directly.
Put another way, SC sees research as an end in itself, but for EQ research
is a means to an end. As discussed briefly in Chapter 3, these different
world views make cooperation and coordination correspondingly difficult,
and unlikely without conscious, continual effort. The Environmental
Management Science Program, which is administered jointly by EM and SC,
demonstrates that such cooperation and coordination are possible,
however.
DOE's portfolio approach also cannot compare programs between
portfolios. For example, there is no common system for setting priorities or
evaluating results. As the preceding example shows, different portfolios
OCR for page 91
Achieving and Maintaining the Long Term Vision for EQ R&D
91
have very different metrics for success and different definitions of what is a
worthwhile problem. Portfolio coordination and management are needed at
high and low levels. At higher, strategic levels they are needed to deploy
resources on the main problems. At lower, tactical levels they are needed
to minimize duplication and overlap, to create synergies, and to ensure
stakeholder involvement. All this is another way of saying that there is a
need for alignment, value, and balance across and within portfolios, and at
both strategic and tactical levels (realizing that because one cannot balance
within the smallest program elements, balance is sought among such
elements).
The generation of the R&D portfolios is a sound accomplishment that
might provide a starting point for coordination. However, as presently
described, DOE's portfolio management approach seems unlikely to
achieve its goals. To be effective the portfolios will need to become a
management tool, not just a descriptive tool. That is, the portfolio process
would need to include explicit management functions and capabilities,
especially accountability.
Findinn: There is little evidence of effective coordination within the EQ
R&D portfolio (e.g., for communication of results or for
recommendations on priorities). Furthermore, there is little evidence
of effective coordination between R&D portfolios.
Conclusion: At present DOE's R&D portfolio process is unlikely to
achieve its goal to integrate and strengthen the planning,
management, and administration of its $8 billion R&D enterprise.
DOE recognizes that "the portfolio process would benefit from improved
coordination and a more integrated approach to ... interportfolio activities"
(DOE, 2000b, p. xiii). Although an understatement, this does indicate that
the process may improve. Accordingly, the committee is reluctant to make
a specific organizational recommendation that might limit DOE's options. In
the past, the Under Secretary chaired a group, the R&D Council, whose
members included the DOE leadership responsible for each of the four R&D
portfolios. Although the status of the R&D Council remains uncertain
following the 2000 election, the charter of such a group would allow it to
oversee coordination of the EQ portfolio, as well as coordination between
portfolios. Because its members have other duties and loyalties, however,
such a group alone is an unlikely vehicle for coordination. The committee
believes that such a group could serve as a forum for discussion and
agreement on plans for coordination developed by the Under Secretary's
staff.
The larger issue goes beyond specifics to whether DOE intends the
portfolios to be more than a context (i.e., whether they should be actively
managed). If they are to remain only descriptive (i.e., to reveal problems but
OCR for page 92
92
~ Saga ~~n fir DOE Ames/ Out R&D
not to address them), some other mesas for ach~v~g the goad of
improved R&D management must be Fund.
Representative terms from entire chapter:
portfolio management
