Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 22
Put Discipline into Decision Making
DOE-EM has spent billions of dollars to introduce technologies to
remediate environmental problems at DOE's weapons complex sites and yet has
continued to be subjected to strong criticism for lack of performance (i.e., few
deployed technologies) (GAO, 1996, 1998~.2° BROOM reports have pointed to a
lack of discipline in DOE-EM and OST's decision-making processes as a
significant cause of this problem. In particular, We reports indicate Mat Me
decision-making process often has been ad hoc and frequently has varied from
site to site and from decision to decision. Unless decision making can be
improved significantly, criticism likely will intensify. The findings and
recommendations from these reports address three components of an effective
decision-making process: (1) framing decisions, (2) making decisions, and (3)
prioritizing needs and evaluating results.
FRAMING DECISIONS
The six reports emphasize two broad issues related to framing decisions
about technology development: defining goals and considering alternatives. The
reports provide three main examples of how OST's technology development
20 The Subsurface Contaminants Committee pointed out that DOE is not alone in its
limited application of innovative technologies. For example, the committee cited an EPA
estimate that innovative remedies have been selected for contaminated ground water at
just 6 percent of all CERCLA sites as of 1995 (Subsurface report, p. 203~. The
committee also noted that despite slow progress in deploying innovative remediation
technologies, the SCFA has helped to develop a number of technologies that have shown
considerable promise (Subsurface report, p. 11~.
22
OCR for page 23
PUT DISCIPLINE INTO DECISION MAKING
23
goals could be better defined: (1) through the use of an end state based
methodology, (2) through consideration of waste forms, and (3) through the use
of baseline functional flowsheets.2i The reports also emphasize the importance
of developing alternative technologies and alternatives to baseline functional
flowsheets.
Defining Goals
To be effective, a technology development program must begin by defining
and specifying particular problems to be solved,22 rather than by developing a
solution and then looking for a problem (D&D report, p. 24~. An important
theme from the six reports is that such an approach had not been used by DOE-
EM to manage its technology development activities. For example, the Tanks
Committee concluded that DOE's process for screening and formulating
technology needs lacks a systematic basis (Tanks report, p. 1~. Similarly, the
D&D Committee found that the prospective uses of facilities that will undergo
decontamination and decommissioning have not been defined adequately, and
this has prevented OST from assessing the technology needs, cost, and schedule
for D&D projects (D&D report, p. 3~. The committee found no evidence of
significant progress in defying or even proposing goals to be targeted by new
technologies (D&D report, p. 3~.
End State Based Methodology
Both of the aforementioned reports (the Tanks report and the D&D report)
make frequent reference to the concept of "end states." As used in the Tanks
report, "an end state can be expressed as the desired composition,
configuration, performance, and location of a particular waste product at the
completion of remediation activities, frequently wastes emplaced in a disposal
facility. If the phased-decision approach previously recommended by the
National Research Council (1996bJ were to be used, the end state my be that
associated with the end state of one of the phases" (Tanks report, p. 15-16,
emphasis added). As the italicized passage makes clear, within this conceptual
approach an end state is not necessarily a final state. In other words, further
phases with new end states may occur.
2} A baseline functional flowsheet is the sequence of steps that comprise the waste
treatment process from the initial waste configuration to the final state.
22 Of course, the technology program must base these problems on technology needs
identified by the sites (see discussion on "Input from Sites/Users"~.
OCR for page 24
24
TECHNOLOGIES FOR ENVIRONMENTAL MANAGEMENT
The Tanks Committee recommended that DOE adopt a systems engineering
approach for identifying technology requirements to remediate stored tank
wastes (Tanks report, p. 7~. The recommended approach involves the analysis
of remediation scenarios23 to identify the technologies needed to achieve
specified goals. The committee concluded that such an approach would serve
two fundamental objectives: (1) greatly facilitate the efficiency and visibility of
EM's efforts to provide the technologies required to remediate high-level waste
tanks throughout the DOE complex, and (2) clearly expose the underlying basis
of the technology development program (including the ability to reach a
prescribed end state in a cost-effective manner), which is critical to gaining
public understanding and support for the program (Tanks report, p. 2, 6~. The
committee argued that this can be done even though substantial uncertainties
exist in the end states (Tanks report, p. 6~. The committee further suggested
that such an approach should be generally applicable to any of the waste tank
farms throughout the DOE nuclear weapons complex and possibly other DOE-
EM problems (Tanks report, p. 1~.
Waste Forms
As an example of how OST should better define goals, Me board uses the
results of Me Mixed Waste Committee. In its report, the committee evaluated
Me state of development of waste forms for mixed wastes and found that
currently available waste forms are adequate (sufficiently developed) to enable
proper disposal of DOE's known and expected mixed waste inventory. These
waste forms resulted mainly from Me intensive worldwide efforts and
experience in developing waste forms for high- and low-level radioactive waste,
and include grout, glass, ceramics, polymers, and compacted waste (Mixed
Waste report, p. 3~.24 The committee therefore recommended Cat waste form
development no longer be a primary focus of the Mixed Waste Focus Area
(MWFA) (Mixed Waste report, p. 98~. Instead, the committee recommended
Cat Me MWFA emphasize engineering design, integration, and scale-up of its
proposed treatment processes and Weir demonstration and deployment at the
DOE sites (Mixed Waste report, p. 98~.
23 For this report, a scenario is loosely defined as the sequence of events that takes
wastes in their current status to desired end states.
24 The Mixed Waste report did not give explicit credit to new OST technology
development. A similar conclusion was reached in the 1996 report: "The MWFA has
determined that 90 percent of the mixed-waste streams can be treated with technologies
that currently exist or that can be modified" (NRC, 1996a, p. 69~.
OCR for page 25
PUT DISCIPLINE INTO DECISION MAKING
25
The Mixed Waste Committee also concluded that conceptual design of
treatment processes and waste form selection can proceed, but the upfront
characterization is inadequate for detailed engineering design of treatment
processes or process optimization (Mixed Waste report, p. 100~. To address
this issue, the committee recommended that the MWFA: (1) develop simplified
methods to characterize the waste up front, with emphasis on nondestructive
examination and assay techniques; (2) continue to develop and implement
techniques and procedures to ensure that all new waste streams are
characterized adequately; and (3) strive for a balance between the effort and
cost of up-front characterization and the effort and cost to develop more robust
technologies (Mixed Waste report, p. 1004. The MWFA assigned improved
waste characterization first priority in its list of technology needs.
Flowsheets
The Decision Making Committee discussed the use of baseline functional
flowsheets (i.e., the sequence of steps that comprise the waste treatment process
from the initial waste configuration to the final state) as fundamentally
important to determining technology needs.25 The committee found that one
major problem with the system has been that OST has had no direct role in
establishing such flowsheets, which have been developed at the sites by the site
remediation problem owners (Decision Making report, p. 75~. The committee
concluded that the expertise of OST and its contractors could be valuable to the
site problem owners in formulating and maintaining technically sound and
practicable cleanup functional flowsheets and recommended that efforts be
made to have substantial involvement of OST and OST contractors in reviews
of functional flowsheets (Decision Making report, p. 75~. The committee
argued Mat such OST participation would have Me dual benefits of (1) ensuring
that OST technology developers fully understand Me sites problem owners'
technical needs and Weir bases, and (2) increasing Me sites' confidence in
OST's dedication and ability to meet their needs (Decision Making report, p.
75-76~.
25 For OST-developed technologies to be adopted at the sites, OST must persuade
site managers and contractors to adopt different technologies from those they are already
committed to use (i.e., its baseline technologies). OST therefore needs to undertake
studies that compare existing baseline technology costs with more favorable costs of
OST-proposed technologies (Decision Making report, p. 75~.
OCR for page 26
26
TECHNOLOGIES FOR EMlIRONMENTAL MANAGEMENT
Alternatives
In the face of technical uncertainty, sound decision making demands that
multiple paths to achieve goals be explored. The importance of considering
alternatives in technology development was emphasized in both the Tanks and
Decision Making reports, as well as other recent NRC reports (e.g., NRC,
1996b). The Tanks Committee found that one major weakness of EM's
approach to technology development for the Hanford Site tanks was a lack of
consideration of end states other than those baseline scenarios codified in
various site-specific compliance agreements (Tanks report, p. 6~. The
committee concluded that scenarios involving alternative end states may need to
be considered for such reasons as life cycle costs, technical failures, and delays
in meeting schedules when originally selected end states present problems
(Tanks report, p. 5~. The committee did recognize that such an approach would
require a major change in DOE-EM technology development philosophy
because alternative end states are outside the present plans of both the
remediation programs and the technology development organization (Tanks
report, p. 5~. Indeed it could require either renegotiation of some of DOE's
current legally binding agreements, or legislation by the Congress. For this
reason, recommendations related to alternatives (in the Tanks report and
included in this report) are directed primarily to the Assistant Secretary for EM
and the Secretary of DOE.
The Decision Making Committee found that good decision making practices
(e.g., hedging against technical uncertainty and insisting on alternatives) imply
that DOE-EM should plan for alternatives to the site baseline functional
flowsheets, especially when the baseline flowsheet involves high cost, high or
poorly defined risk, and/or substantial probability of technical failure (Decision
Making report, p. 76~. To address this issue, the committee recommended that
EM seek out and acknowledge the potential vulnerabilities in cost, risk, and
technological failure-of the baseline functional flowsheets and processes, and
with OST's assistance, develop alternative flowsheets as appropriate. The
committee recommended that OST encourage this course of action and seek to
collaborate in it. In particular, the committee recommended that OST identify
specific technology development opportunities aimed at supporting alternative
functional flowsheets and processes designed to enhance the overall probability
of remediation successes and to minimize program delays (Decision Making
report, p. 76~.
OCR for page 27
PUT DISCIPLINE INTO DECISION MAKING
27
Recommendations
DOE-EM should implement an end state based methodology26 (similar
to that described in the Tanks report) to identify the technology needs and
research and development required to achieve specific remediation goals.
Such an approach would include the following characteristics:
· DOE management and legislative decision makers should allow for
consideration of a wider range of alternative end states that may be
needed in the future, and this should be reflected in the DOE
remediation and technology development programs (Tanks report,
p. 5~.
· Alternative end states and alternative scenarios should be
considered for remediation scenarios that involve high uncertainty
or high risk.
· OST should be allowed to commit a portion of its resources to
developing technologies to address needs derived from alternative
functional flowsheets, in addition to developing technologies to
meet the needs derived from the baseline flowsheets (Decision
Making report, p. 76~.
· If initial conditions cannot be adequately characterized and end
states cannot be adequately specified, a plan leading to the timely
resolution of open items should be prepared and executed. In the
interim, assumptions that allow scenario development and
identification of technology needs to proceed should be emplaced
and clearly stated, preferably by problem owners, but by
technology providers if necessary (Tanks report, p. 7).
MAKING DECISIONS
The Decision Making Committee found that, as OST's decision-making
process has matured, it has functioned reasonably well to prioritize technologies
for funding at individual sites and to make decisions for funding among sites
within OST's operational framework. However, some of He process steps are
26 As noted previously, with this conceptual approach, the term "end state" does not
necessarily connote a final disposition of the waste or waste sit; further phases with new
end states may occur.
OCR for page 28
28
TECHNOLOGIES FOR ENVIRONMENTAL AL4NAGEMENT
cumbersome and ill conceived. Furthermore, there has been little effort to apply
a carefully considered process uniformly throughout OST (Decision Making
report, p. 874.27 Other committees, including the Peer Review Committee, the
Mixed Waste Committee, the Subsurface Contaminants Committee, and the
D&D Committee, also analyzed parts of OST's decision-making process. Based
on its review of these reports, the board has identified four main areas where
OST can improve its decision-making process: (1) adequate documentation and
communication of results; (2) use of independent, external expert review; (3)
use of input from sites and other affected parties; and (4) increased
transparency.
Documentation and Communication of Results
The importance of adequate technical documentation and communication of
results has been emphasized in several recent board reports. For example, the
Decision Making Committee noted that the type and quality of information
provided to Congress and other interested review groups are critically important
to OST. The committee recommended that OST ensure that the decisions
underlying the technologies it develops are well documented and supported by
sound technical reviews (Decision Making report, p. 79~. The Peer Review
Committee commended OST's peer review program for modifying Me
documentation required for peer reviews to focus on technical issues. However,
Me committee pointed out Cat Me program does not require a detailed technical
proposal or statement of work, and recommended Hat such a document be
required for every peer review (Peer Review report, p. 10~. The Subsurface
Contaminants Committee found Mat Subsurface Contamination Focus Area
(SCFA) periodic technology summary reports were not sufficiently detailed to
evaluate potential technology performance and effectiveness relative to other
technologies, and recommended Hat such reports include well-documented
27 OST recently pilot tested a new system (its Work Package Ranking System) to
create an integrated priority list of "work packages" (i.e., a group of related projects) at
the OST headquarters level. The new process employs five criteria to rank the work
packages: site needs, project value/need, future technology deployments, technological
risks, and technology cost savings (DOE, l999a,d). In this new system, a numerical
score for each work package is produced through a process stated to be based on multi-
objective decision analysis methodology. Beyond this generality, the processes used to
determine the scoring criteria, ranking factors, and weighting factors have not been
specified. The Decision Making Committee noted that in principle, this prioritization
process could be used to set budget targets for OST program units. However, the
committee also pointed out that this system was not yet developed to a point where OST
staff used it as the sole basis for prioritization (Decision Making report, p. 481.
OCR for page 29
PUT DISCIPLINE INTO DECISION MAKING
29
performance data, detailed cost estimates, design information useful to
practitioners, and lessons learned (Subsurface report, p. 246~. The committee
also recommended that a key future role for the SCFA be the development of
design manuals for technologies that could be used across the DOE weapons
complex (Subsurface report, p. 2164. In addition, the D&D Committee
concluded that one factor that limited the acceptance of OST-demonstrated
technologies was the lack of prompt dissemination of technical and cost data on
these projects (D&D report, p. 2~. The committee recommended that the D&D
Focus Area communicate its program results in a more effective manner (D&D
report, p. 4~.
Independent, External Expert Review
The Peer Review Committee conducted a detailed evaluation of the peer
review28 process that OST established in 1996 to perform independent technical
assessments of technology projects. During the course of the study, OST
continued to change its peer review procedures, partly in response to comments
in the committee's interim report. The changes were intended to improve the
peer review program and were generally found by the committee to be steps in
the right direction. In particular, OST (1) revised its review criteria to focus on
four technical issues; (2) developed a more systematic approach for selecting
projects to be reviewed; and (3) modified its list of documentation required for
peer reviews (Peer Review report, p. 6~. The committee also noted that OST
senior management appears to be committed to this improvement process (Peer
Review report, p. 7~. Based on this report and its contrast with prior critical
reviews of OST's peer review efforts (NRC, 1996a), it appears that OST has
made significant improvements In this program.
Despite these improvements in the procedures used to conduct peer review,
however, the Peer Review Committee identified several important issues Cat
remain to be addressed: peer reviews are not being incorporated adequately into
decision making (Peer Review report, p. 7~; OST is overwhelmed with a
backlog of needed peer reviews (Peer Review report, p. 11~;29 for peer reviews
to make a difference, new procedures, while important, are not enough-
instead a culture change also is needed (Peer Review report, p. 13~; and unless
OST uses peer reviews judiciously, it runs the risk of adding yet more reviews
and paperwork, even as its budget continues to shrink (Peer Review report, p.
13~.
28 Peer reviews are defined as evaluations by technical experts who are independent
of, and external to, the program of work being reviewed.
29 As of May 1998, only 43 of 226 active projects had been peer reviewed.
OCR for page 30
30
TECHNOLOGIES FOR ENVIRONMENTAL MANAGEMENT
The committee made a number of recommendations to address these issues.
First, OST program managers should be required to clearly identify the
upcoming decision or milestone for which the peer review results will be used,
before a project is chosen to be reviewed (Peer Review report, p. 7~. Second,
prioritization should be applied to the selection of projects for peer review and
how the peer reviews are conducted; for example, OST should consider
adopting a triage approach30 to allow more effective in-depth reviews (Peer
Review report, p. 11~. Third, OST leadership should develop a strategy to
accomplish a change in its organizational culture so that OST staff recognize
and accept the value of independent expert advice (Peer Review report, p. 14~.
The Decision Making and Subsurface Contaminants Committees also
considered the role of independent external review in OST's decision-making
process. The Decision Making Committee concluded that independent expert
reviews are a vital part of a credible decision-making process (Decision Making
report, p. 76), and that peer reviews of technology projects should be part of
OST's decision-making process. lathe committee also pointed out that the
evaluation of technology development projects is but one step in OST's
decision-making process (Decision Making report, p. 84. The Subsurface
Contaminants Committee recommended that the SCFA significantly increase the
use of peer review for determining technology needs and for evaluating projects
proposed for funding (Subsurface report, p. 9~.
Both the Peer Review and Decision Making Committees identified
additional areas to which external independent reviews could be applied
usefully. The Peer Review report included a detailed discussion on how peer
reviews could be used to assess We technical merit of programs (Peer Review
report, p. 46-47~. The Decision Making Committee recommended Rat OST use
an external independent body to review Me bases of annual decisions Mat
establish budget targets for OST program units (Decision Making report, p. 81.
The committee also recommended Rat OST have a role in reviewing site
remediation fimctional flowsheets (Decision Making report, p. 3~. Bow
committees cautioned, however, that before adding any more reviews, OST
should carefully assess Me purpose and value of the many reviews already being
used (Decision Making report, p. 77; Peer Review report, p. 131. The Decision
Making Committee pointed out Mat peer reviews of projects should not require
an overly burdensome commitment of OST resources, and therefore
30 Such an approach involves a formal prescreening of projects by peer reviewers
based exclusively on the project's written documentation. The results of this
prescreening review could then be used to determine (1) highly ranked low-budget
projects that should be considered for funding without additional review; (2) highly
ranked projects that should receive a more detailed evaluation; and (3) technically weak
projects that should not be considered for funding (Peer Review report, p. 11-12~.
OCR for page 31
PUT DISCIPLINE INTO DECISION ASKING
31
recommended that the peer review of projects be streamlined by reducing the
number and types of reviews based on an analysis of the objectives being served
by each (Decision Making report, p. 77~.
Input from Sites and Users
The necessity of incorporating input from sites and users in OST's
decision-making process also has been a prominent theme in recent board
reports (Decision Making, D&D, Mixed Waste, Subsurface reports), as well as
GAO's recent report.3i OST has had limited success in getting the technologies
it developed or procured deployed at the sites with cleanup problems. The
Decision Making committee pointed out that this problem is due in part to
conditions outside OST's control (see Chapter 5~. However, the committee
found that the problem also was the result of the way OST operated in the past
when it developed technologies without adequate input from site problem
owners (Decision Making report, p. 9~. In more recent times, these site inputs
have been obtained through Site Technology Coordination Groups (STCGs)32
(Decision Making report, p. 9), but the Decision Making committee found
weaknesses in the STCG structuring of criteria and in the STCG evaluative and
prioritization methods (Decision Making report, p. 3~33. To address this issue,
the committee recommended that OST use the best available information on
DOE-EM site technology needs as a guide to tailoring program goals and
RD&D projects. As one way to acquire this information, the committee
recommended that OST establish (or increased its direct contact with site
personnel at the problem-solving and decision making levels (Decision Making
report, p. 3~. The Decision Making Committee also concluded Hat longer-term
technology needs be derived from OST's consideration of the fi=ctional
flowsheets for site remediation Hat over DOE-EM offices already develop, use
3] In its evaluation of the extent to which innovative technologies developed by OST
have been deployed, GAO concluded that lack of user involvement is one of the major
remaining obstacles to more widespread use of technologies developed by OST (GAO,
1998).
32 OST has formed STCGs at each major DOE-EM site to interact with local
contractor personnel and others to obtain that site's environmental restoration and waste
management technology needs (Decision Making report, p. 21. STCGs are responsible
for developing and prioritizing a list of site problems and technology needs based on the
environmental management issues relevant to a specific site (Decision Making report, p.
105~. Each STCG evaluates and prioritizes (i.e., ranks or rates) technology needs
according to a set of criteria established by the STCG (Decision Making report, p. 3~.
33 The criteria were different at each site and at some sites were not rigorously
constructed (Decision Making report, p. 3~.
OCR for page 32
32
TECHNOLOGIES FOR ENVIRONMENTAL MANAGEMENT
in their planning, and subject to reviews (Decision Making report, p. 3). At
present, OST has no direct role in establishing or reviewing these flowsheets,
which are activities conducted by other EM organizations and contractors at the
site level. The committee recommended that, in conjunction with the other
DOE-EM offices responsible for site cleanups, OST participate to the extent
possible (e.g., by establishing a role for its contractors) in a review of site
remediation functional flowsheets. OST's technology development projects
should be responsive to technology needs identified from baselines remediation
plans and their alternatives (Decision Making report, p. 3~.
Similarly, the Mixed Waste Committee recommended that OST continue to
address technology deficiencies identified by the STCGs (Mixed Waste report,
p. 69~. The Subsurface Contaminants Committee found that a major barrier to
deployment of SCFA's technologies is a lack of demand from individual DOE
sites (Subsurface report, p. 7), and that one important factor in limiting demand
for SCFA technologies (among other factors) is insufficient involvement of
technology end users in setting SCFA's technology development priorities
(Subsurface report, p. 9~. The committee recommended that the SCFA strive to
increase the involvement of technology end users in planning the technology
demonstrations it funds (Subsurface report, p. 12~.
Transparency
The Tanks Committee concluded mat although DOE has a participatory
process for screening and formulating technology needs, this process lacks
transparency (in terms of being easily understood by all concerned decision
makers and other interested and affected parties34) (Tanks Report, p. 1~. The
Peer Review Committee discussed at some length the pros and cons of
conducting peer reviews in an "open" manner (i.e., identifying reviewers, fully
informing Me public of Me nature of Me reviews, and employing a known
process) (Peer Review report, p. 35-36~. The committee concluded Mat Me
strengths of open reviews (e.g., enhanced credibility of Me process, Me
potential for more constructive evaluations) far outweigh Me potential
weaknesses (e.g., possible lack of candor by some reviewers when evaluating
weak proposals), especially for Me peer review of projects or programs (Peer
Review report, p. 79~. The committee emphasized Cat openness does not imply
34 "Interested and affected parties" can include members of the public, technology
users, the affected state or tribal nation, and regulators. All of these should, on a
selective case-by-case basis, have the opportunity to be involved in decisions concerning
remediation technologies. See Understanding Risk: Informing Decisions in a Democratic
Society (NRC, 1996c).
OCR for page 33
PUT DISCIPLINE INTO DECISION MAKING
33
Mat all deliberative sessions are held In public, however. The ability of an
evaluating body to discuss frankly He merits and weaknesses of a project and to
reach consensus In a closed session is an important attribute of some open
reviews (Peer Review report, p. 361. The committee encouraged OST to
continue to promote openness of its peer reviews and to fully inform members
of Me public and others attending Be reviews of Heir nature (Peer Review
report, p. 111. The importance of transparency to OST's decision-making
process also was noted by the Decision Making Committee, which
recommended Hat OST ensure that He decisions underlying He technologies it
develops are well documented, traceable to customer needs, and supported by
sound technical reviews (Decision Making report, p. 791. The board believes
Cat such transparency should be a standard characteristic of much of DOE's
decision-making processes.
Recommendations
.
OST's decision-making process should be transparent; include
participation from relevant interested and affected parties (such as
the technology users, the surrounding community, the affected
states and tribal nations, and regulators); incorporate adequate
documentation; involve peer review; and lead to setting priorities.
The decision points at which (1) budget allocations are made and
(2) user-defined technology needs are established are important to
shaping the OST program and are therefore opportunities for
independent external review (Decision Making report, p. 81. Before
adding any additional reviews, however, OST should carefully
assess the purpose and value of the many reviews already being
conducted (Decision Making report, p. 77, Peer Review report, p.
13).
OST should use the best available information on DOE-EM site
technology needs as a guide to tailoring program goals and RD&D
projects. As one way to acquire this information, OST should
establish (or increase) its direct contact with site personnel at the
problem-solving and decision making levels (Decision Making
report, p. 3~.
OCR for page 34
34
TECHNOLOGIES FOR ENVIRONMENTAL MANAGEMENT
· OST should continue to address technology deficiencies identified
by the Site Technology Coordination Groups, but prioritized
among the sites by upper-level DOE-EM management.
PRIORITIZING NEEDS AND MEASURING RESULTS
There have been no general OST-wide guidelines applied among the OST
programs for setting criteria for the selection and prioritization of technology
development needs, although individual OST program units have developed
their own guidelines (Decision Making report, p. 84~. The Decision Making
Committee recommended that OST establish general selection and prioritization
criteria, and guidelines for applying these criteria, to include allowance for
instances when exceptions to the criteria may be appropriate (Decision Making
report, p. 84~. Similarly, the D&D Committee recommended that OST and the
D&D Focus Area (DDFA) develop and apply a consistent approach to
comparative technology assessment across all projects (D&D report, p. 31~.
The Decision Making Committee emphasized that the decisions underlying
the technologies OST develops should be traceable to user needs (Decision
Making report, p. 79~. However, this does not mean that all RD&D needs
should be derived from user requests. For example, alternative technical
approaches to site remediation baseline flowsheets are another important source
of technology development needs (see "Alternatives" earlier in this Chapter).
The committee therefore concluded that there is a need for exploratory RD&D
to meet the needs of alternatives to baseline flowsheets (Decision Making
report, p. 79~. Moreover, the Decision Making Committee emphasized that the
general criterion that technologies should be applicable to multiple sites, while
useful, is flawed when applied without exception because it may lead to a
failure to develop technologies for potentially very important problems that
exist at only one or two sites (Decision Making report, p. 84~.
The Decision Making Committee also emphasized the importance of
establishing quantifiable attributes and follow-up procedures to measure (and
hopefully improve) organizational results (Decision Making report, p. 3-4~.
Based on its review of private sector decision-making practices, the committee
recommended that OST adopt relevant principles of private sector decision
making, including the need to agree on clear and measurable goals and to
measure and evaluate results as a guide to resource allocation (Decision Making
report, p. 74. The committee found that OST's stage-and-gate system for
monitoring ongoing projects has more development stages than necessary, and it
does not seem to assist effectively in making decisions to select a project for
funding or to terminate a project. The committee therefore recommended that
OST use the minimum number of stages and gates needed to track a project and
OCR for page 35
PUTDISCIPL1NE INTO DECISION MAKING
35
use peer reviews at key decision points (gates), especially in Me selection of a
new project (Decision Making report, p. 5).
Recommendations
.
.
.
.
For decisions involving the allocation of significant resources, OST
should institute a decision-making structure wherein projects and
proposals are evaluated against consistently defined criteria, such
as project cost, probability of technical success, probability of
implementation, potential cost savings, and human health risk
reduction (Decision Making report, p. 801.
To the extent practicable and with input from its various
organizational elements, OST should establish general selection and
prioritization criteria, and guidelines for applying these criteria,
including allowance for instances where exceptions to the guidelines
may be necessary (Decision Making report, p. 841.
Although the technology development projects should be based
primarily on specific site needs, some should be of an exploratory
research nature (Decision Making report, p. 791.
OST should establish measurable goals that can be used to quantify
its success in meeting organizational objectives (Decision Making
report, p. 7~.
Representative terms from entire chapter:
committee recommended
