Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
ROAD Decision-Making Practices In Other Organizations This chapter provides a discussion of good decision-making practices that are drawn from literature surveys and benchmarking studies of major U.S. companies and from committee member interviews with representatives of organizations with significant environmental remediation programs. This chapter covers a range of applicable good practices and methods used in industrial RD&D and in two nonprofit research organizations whose scope and operating environments are analogous in some ways to those of DOE-EM. Practices in profit-making organizations at some stages are generally similar but are subject to greater differences in criteria, operating environments, and measures of success. The wide range of inputs from many different organizational contexts provides a good basis for recommendations for refinements in the methods and practices used in DOE. Those practices that have particular relevance to OST are highlighted. BENEFITS OF A HIGH-QUALITY DECISION PROCESS Decisions are the product of management work. They are arrived at through a process, although that process is not always explicit. Decisions have customers and should conform to customer requirements. Research and application experience has shown that there are many benefits from using a high-quality decision process (Matheson and Matheson, 1998; Matheson and Menke, 1994; Menke, 1994~. Process benefits include the following: depolarizing high-conflict situations potentially involving strong differences of opinion, ensuring comprehensive consideration of relevant factors, ensuring consideration of a wide range of alternatives, leveling the playing field across multiple projects and decisions, maintaining a consistent approach as people change positions, developing consensus and building commitment to action, facilitating explanation ofthe decision to internal and external parties, and providing documentation for retrospective analyses and insight on post-project audits. These benefits come not only from the nominal decision process itself, but also from the skill and quality with which it is carried out. To expand upon the last item, one important benefit of a well-structured process is to facilitate learning from past experience, leading to continuous improvement in the decision process. Decision research has shown that failure to learn from experience and to apply the lessons learned is one of the most pervasive and handful problems in decision making (Russo and Schoemaker, 19891. 28
Good Practices in RD&D Decision Making 29 A decision process incorporating some of the best practices should lead to better results. Some evidence to support this conclusion is presented next. LESSONS FROM INDUSTRIAL RD&D ORGANIZATIONS Because of RD&D's importance to long-te~m success and sustainable competitive advantage, many industrial organizations have devoted a significant effort to developing and improving their RD&D decision- making process. This allows one to learn from a group of organizations that allocate substantial resources to RD&D and that can be judged to have achieved significant and valuable results from this expenditure. Benchmarking high-performance organizations to determine their "best practices" is a widely used approach for such learning. An extensive research project on how outstanding RD&D organizations actually make decisions defined 45 best practices (Matheson et al., 1994), which are presented in Appendix F and summarized below. Although more than 300 RD&D organizations participated in this project, a group of 79, selected on the basis of peer group nominations, was designated as the "best company database" and used to quantify the importance and implementation of these practices. The use of these best practices is clearly associated with higher performance as indicated by the number, revenue, and profitability of new products (Menke, 1997a). As a further result of work by Matheson et al. (1994), 10 of these 45 best practices were shown to stand out as especially important by having a potential contribution to decision quality with a mean score higher than 6.0 (on a scale of ~ to 7, with 7 as the highest possible score). These 10 practices have been identified as essential for RD&D strategic excellence in industrial organizations (Menke, 1997a), and are listed in Appendix F. Further analysis has identified 10 other practices for gaining competitive advantage (Menke, 1997b), which are also listed in Appendix F. These are the 10 least well implemented from a set of 28 (out of the 45) practices judged to be very important In terms of performance impact or potential contribution. Because these 10 practices are important but not very well implemented (even by some of the most outstanding RD&D organizations in the worId), organizations that want to gain a competitive advantage or improve their operational efficiency can very likely do so by implementing these additional practices. The committee's conclusion, based on the information in Appendixes B-E, is that many of the same practices should be relevant to malting good RD&D decisions in a public-sector RD&D organization, such as DOE OST. There was general agreement among committee members as to which practices should be most relevant. These are discussed In the next section. INSIGHTS FROM THE BENCHMARKING STUDY Using the descriptions of best practices shown in Appendix F. committee members selected what they believed where the most important practices for OST to achieve excellent technology RD&D decisions. There was very strong agreement among committee members' selections, and the practices that the committee members felt were most important were, with only a few justifiable exceptions, the same as the 10 essential practices mentioned above. The most important of these was the following: · H~re the best people possible and maintain expertise. iSince DOE-EM is a government organization and therefore not a priori in a competitive environment, an appropriate analogy to gaining competitive advantage might be improving efficiencies of operation. On a technical level, any RD&D result must "compete" with prior and alternate ways of doing something.
30 Decision Making in the DOE-OST Committee members thought that this applied to OST as well. However, these general statements beg translation to be appropriate in We OST application. Since OST is essentially a federal program of DOE program managers, who award funds to contractors to perform technology development projects, a suitable translation of this thought for OST might be, "Select the most competent contractors." The committee prefers to leave these translations to OST management, who are well positioned to appreciate how best to implement these general guiding ideas. Committee members thought that dealing with end customer needs was also critical for OST, as represented by the following: · Understand, focus on, and monitor changes in customer needs and requirements. This statement combines the ideas of 3 ofthe top 10 essential practices: (~) focus on end customer needs; (2) determine, measure, and understand end customer needs; and (3) refine projects with regular customer feedback. The needs-focused approach helps ensure that the technologies are relevant to customer requirements. Successful companies make every effort to determine, understand, and measure their customer's needs. Their RD&D organizations solicit frequent input. Many techniques can be used to gain customer feedback and to inspire customer support and loyalty. The key is a structured interaction process that returns knowledge, insight, and information. That is, the structure advocated here is Mat of having a well-def~ned (e.g., recorded) process by which (~) needs are assessed, (2) new projects are evaluated, and (3) funds are allocated. For example, criteria used to evaluate proposed projects should be adequately descriptive so that their meanings are clear to all interested parties. This structure is not intended to constrain or limit inputs from certain sources; indeed, the decision-making process should allow for all appropriate inputs. Rather, structure enables the process to be widely understood and critically examined for improvement over time. Two other essential decision practices that committee members agreed were very Important for the DOE-EM environment are the following: · Agree on clear and measurable goals. · Use a formal (i.e., common, consistent, structured, and rational) technology development decision- making process. To have the greatest likelihood of success, several management levels should share strategic objectives. Ideally these goals are directly related to the prioritized needs of the customers of RD&D, needs that are related to the goals of the user organizations (in this case EM-30 and EM-40. The most successful companies also use a formal RD&D decision-making process with phases, checkpoints, and milestones to frame decisions and hack their Implementation. The process includes features and documentation that make continuous improvement possible through learning and iterative refinement. There are 17 other decision practices out of Me full 45 that the committee thought could be particularly important to the goal of improving OST's decision-making process. They fall into several important categories:2 · Think strategically (i.e., long range and high impact), which combines the following ideas represented by the best practices: I. Frame RD&D decisions strategically. These statements may read like general platitudes but, again, are offered for illustrative purposes as descriptive of methods that work well in other institutions; OST management would have to evaluate how far these concepts can be applied within the OST program.
Good Practices in RD&D Decision Making 2. Understand the drivers of change. 3. Insist on alternatives 4. Coordinate long-range business (for DOE-EM, the analogue to "business" would be the "site user") and RD&D plans. 5. Balance Innovations and incremental RD&D. 6. Hedge against technical uncertainty. 31 . Measure and evaluate to guide resource allocation, which includes the following: I. Ensure credible, consistent inputs. 2. Quantify decision inputs. 3. Measure the contribution to strategic objectives. 4. Evaluate the RD&D portfolio. 5. Manage the pipeline (and communicate how this is done). 6. Evaluate projects quantitatively. · Communicate across organizational boundaries, which includes the following ideas of best practices: I. Use cross-functional teams. 2. Coordinate development with commercialization (i.e., deployment). 3. Maintain inhmate contact with internal customers. . Continually improve the RD&D management process, which includes the following: 1. Learn from post-project audits. 2. Measure RD&D electiveness. "Think strategically" encompasses six of these decision quality best practices to stress the importance of a {ong-range strategic viewpoint in developing an elective RD&D program. "Measure and evaluate to guide resource allocation" encompasses six more. "Communicate across boundaries" refers to more open, cross-functional communication, which has been a major factor in improving industrial RD&D productivity over the past decade. The last category, "continually improve your RD&D management process," deals with improvement that comes Dom evaluations and audits of ongoing and past work. The practice of learning from post-project audits is advisable even for simple, straightforward decisions, and can be facilitated by keeping consistent logs or narrative records of the main criteria, factors, and measurements that were considered and evaluated. The recording formality is not just useful for supporting the decision-making process per se. Its other functions are to provide a coherent record and database for consistent administration when personnel or organizational assignments of responsibility change in the course of the project. Most important, the records also provide the basis for deriving "lessons learned." Decisions are rarely "optimum" when viewed in hindsights. The log of the decision and implementation process can be used to reinforce good choices and practices in other projects, to help others foresee and avoid repeating the inadvertent pitfalls that are encountered, and to benefit from the preventive arid remedial measures developed. 3 Indeed, optimization is difficult in part because it is rarely, if ever, possible to know all the alternatives and their outcomes. The decision process then strives for a "good enough" result among likely candidates of relevant alternatives (March, 1994; 1999; Simon, 1997~.
32 Decision Making in the DOE-OST Decisions always have to be revisited after changes in goals or missions are promulgated, whether by the industry, agency directorate, or Congress. This requires re-evaluating the alternatives to be considered, the information available, and the applicable criteria. Whenever there is a change (or even a more crisp definitions of goals, criteria for success, and measurement methods, many prior decisions become subject to change. For decisions in such less-than-simple conditions, the use of a formal decision process Including record-keeping practices) becomes prudent, even for instances in which it appears to some participants that one or two well-und~erstood factors dominate the decision (typically these are budget cuts or line-item d~irectives). At stake is not merely failure to achieve optimum results. An obvious hazard in decision making is the case in which an administrative decision maker believes that one or two "obvious" decision factors or goals are so dominant that the decision is automatic and requires no further thought. This can leads to prolonged pursuit of poor or ineffectual policies. There are many historical examples of large, persistent, and costly blunderers in government, industry, and the military. 4 A favorable outcome does not necessarily reflect an optimum or even a good decision practice, nor does a less favorable outcome necessarily reflect poor decision practice. An important measure of a successful decision is that its most likely outcomes are consistent with the stated intentions (finals) of the project and of the organization charred with responsibility. ~ ~, The outcome of a decision may only partly fulfill the intended goals. Limited or no success in producing the expected outcome of a decision is not necessarily a sign of a flawed decision process. A basic measure of good decision-making methods and practices is that the decision remains "robust" in hindsight, regardless of outcome. For this purpose, the term robust means that the decision considered all possible outcomes and their probabilities of occurrence and that a better decision was not available with the information available or obtainable at the time. This recognizes the commonsense observation that almost any decision may be found to be suboptimal as further information is acquired or develops. The valuation of possible outcomes should involve the group or community that is most directly affected by the decision. In a business context, the most likely outcomes are evaluated in terms of business values for example, effects on market share, profitability, development of a viable new product or service, or better image as seen by clients. In the public sector, evaluation commonly also includes criteria such as equity, meeting stakeholder perceived needs, cost-benefit ratio, and both short-term and {ong-term impacts. In selecting and guiding RD&D, many decisions occur over extended periods. For example, a selected technology development project is subject to continuing measurement and re-evaluation for both performance and consistency with the ultimate goals the project is intended to address, as it moves through states of development. In the evaluation of a project over time, as it matures from an initial proof-of-concept experiment to a large-scale demonstration of a properly designed and engineered prototype, better performance of the memos under development is only part of the evaluative challenge. Well-managed RD&D is subject to continuing refinement of the approach being taken; that is, the organization's view of the optimal technology solution to a given need or problem can change during the development of a particular technology designed to address the need. The original decisions-on goals, budgets, measures of performance, relative likelihood of success and of application, and the priorities of various needs are open to change as new information becomes available. The decision to continue, modify, or stop a given technology development project may arise from a reassessment of the need for it irrespective of the project's performance. Overall decision quality is achieved by periodic examination of these issues, resulting in a succession of decisions made over hme on the scope and duration of any technology development project. 4The March of Folly by Barbara Tuchman recounts examples of large organizations and governments that persisted in decisions or policies that were manifestly and grossly to their own disadvantage, given any reasonable view of plausible intended goals (Tuchman, 1992; see also Scott, 19981.
Good Practices in RD&D Decision Making INSIGHTS FROM INDUSTRIAL ORGANIZATIONS WITH ENVIRONMENTAL PROGRAMS 33 The committee was fortunate in being able to interview key environmental management executives from several private-sector organizations that have large environmental programs including significant remediation needs. Outside institutions were selected based on several criteria, including the following: I. complexities in institutional structure that offer some analogue to OST within DOE-EM; 2. a larger community of interested parties that could be analogous to the institutions and individuals interested in DOE-EM site activities (e.g., the technology users at DOE-EM sites, the sites' surrounding communities, regulatory authorities, and the U.S. Congress); 3. a large RD&D program in technological areas similar to those of the OST program; and 4. a large environmental program with at least the potential for significant remediation efforts at multiple sites. The committee could examine in depth only a limited number of institutions; by selecting the ones listed in this chapter, other reasonable candidates are not represented, and their omission from the few specifically mentioned here should not be construed pejoratively. Certainly other RD&D programs in federal agencies and private industries are valid models that offer insight into effective decision-making practices. The organizations selected and examined by the committee were Amoco, DuPont, Exxon, Mallinckro~t, Monsanto, the Electric Power Research Institute, and the Gas Research Institute. The focus of these interviews was not limited to technology RD&D selection, but covered the full scope of strategic decisions for the environmental programs of these organizations. This was necessary since several of these companies are themselves developing new or improved environmental cleanup methods only when necessary; that is, when commercially available methods will not suffice. Although some of the environmental challenges and consequences of the bureaucratic environment faced by DOE-EM are unique, DOE can still learn from the environmental decision-making and technology development practices of these leading industrial organizations. Practices for Environmental Technology Development Decision Making and Management The practices relevant to DOE-EM that the committee learned from more than one of these organizations include the following: · Link environmental decisions to the business planning process as much as possible. · Use qualitative and quantitative technical risk assessment approaches (with varying degrees of formality). · Do a sensible prioritization of needs, and tackle the most costly and complex jobs first (good for "mortgage reduction"~. For application to OST, this would mean putting resources in places of maximum benefit or return that is, in areas where technological innovation would have the greatest impact on cleanup costs or risks. . Do not always wait to be forced to address environmental issues; it can be much cheaper to solve them voluntarily. · Try to standardize the technologies used for recutting problems (do not "reinvent the wheels. For application to OST, this would mean building and using a catalogue of"best technological approaches" that would provide a technological baseline of practices. · Try to develop a proactive relationship with regulators, including seeking pre-approval of technologies for specific needs.
34 Decision Making in the DOE-OST · Share insights learned and technology development costs with other companies or organizations with similar problems to the degree possible, through industry associations such as the American Petroleum Institute and the Petroleum Environmental Research Forum. ~1 ~ ~ ~ 1 · ~ 1 1 · · ~ · . · ~ . ~ l he first three environmental dec~s~on-mak~ng practices relate directly to several of the best practices for excellent RD&D decision making discussed above, even though they lied not specifically refer to technology development in these companies. The first is almost identical to the essential practice "coordinate {ong- range business and RD&D plans," whereas the second and third provide some additional specificity such as the use of risk assessment approaches and prioritization based on the magnitude of the problem to be solved. The fourth, however, is a new insight. If problems are dealt with before "solutions" are externally imposed by regulators and stakeholders, the range of options available is usually much broader and much more cost- e~ective. The converse of this can be seen at several DOE sites where the most cost-effective technical solutions are precluded by legal agreements, such as the triparty agreements operative at Hanford.s A proactive approach could be valuable for DOE in going forward to the extent that flexibility still exists in some of its problem areas. INSIGHTS FROM VISITS TO DUPONT, EPRI, AND GRI The committee sought input Dom large institutions that in some ways could be compared to OST within DOE-EM. These institutions had as common features a centralized technology development program (the counterpart of OST) and separate, quasi-independent "business units" or "member utilities" (the counterparts of other EM offices, such as EM-30, EM-40, and EM-601. Another common feature of these analogous institutions was a context of external regulation In an industry faced with some environmental cleanup issues that served as a basis for technology development activity. These features were criteria in selecting institutions for StU6Y. Me curoose of gathering information from these organizations was to orobe their cat - ~ decision malting to discover effective practices that might lend themselves to adoption by OST. Three institutions DuPont, EPRI, and GR} were chosen for in-depth scrutiny, based on He above criteria. Duling field trips to these organizations, committee members interviewed upper-level managers from the technology development programs. The information learned from these interviews and from published company literature is described in detail in Appendixes G. H. and I. Salient points that He committee extracted for emphasis and application to OST are summarized below. Institutional Structure . . · There is nothing wrong per se with individual business units' or member utilities' (hereafter referred to as companies; their EM counterparts are the other EM offices, such as EM-30, EM-40, and EM-60) having their own internal RD&D operations. In each institution, a healthy, long-term relationship was established by which certain RD&D projects (those of a short-term nature or tied to specific company processes) were channeled to each company's internal RD&D group, while other RD&D projects (those representing common and long-term needs, with universal application to many companies) were directed to the central RD&D program. The internal RD&D groups of the separate companies or divisions tended to have a narrow focus on unique, local problems and on RD&D projects with a short time horizon. These projects are better charactenzed as development work rawer Han as RD&D. Since He central RD&D facility does not find sit is widely recognized that some of the legal agreements involve high marginal cost and low risk reduction benefits for specified actions. The use of RD&D to identify alternative technical approaches has merit in these situations, as discussed farther in Chapter S.
Good Practices in RD&D Decision Making 35 every proposal directed to it, the internal RD&D operations of the individual companies are necessary to ensure funding of proposed work that is truly critical to only one or two companies, that is, that they can meet their own immediate needs. Effective Top-Leve' Strategic Goals to Define the Suite of Technology Projects Another feature common to DuPont, EPRT, and GRI is that top-level, strategic goals are cast in measurable, quantitative terms and are elective Givers to define the suite of technology development projects proposed for funding. These goals were written as achievements for each separate business unit to strive to accomplish in the time frame of a few years. For example, DuPont set a goal in 1993 to reduce toxic air emissions from its domestic facilities by 60 percent over three years from the 1993 benchmark level. Similarly, one of GRI's program units developed a goal to "develop and transfer technologies that lower the costs of gas-well drilling by 5 percent by year 2000," (GRI, 1997b), commensurate with its more top-level objective to keep natural gas competitive with other energy sources in consumer demand. Such goals then defined the types of RD&D proposals that were generated by researchers and technology developers and entertained for review by program managers. RD&D goals and performance specifications are selected after careful negotiations between the "marketers," who survey and determine the needs of clients, and the "developers," who intensively estimate the results available for a given time and cost. This provides a guarantee to potential sponsors of We RD&D efforts that the results will be provided at a specified cost and schedule. Measure and Evaluate to Guide Resource ANocation via Return-on-Investment Evaluations The centralized technology development programs of DuPont, EPRI, and GR} seek to operate in an objective, matter-of-fact environment in which the selection of projects to fiend is a business decision. Hence, the scoring of proposals by some appropriate ROT measure (e.g., the GRI Project Appraisal Methodology scoring described in Appendix H or the DuPont method of converting consequences to dollars as a way to provide a relative ranking) is crucial to informing the decision of which funding candidates to select. As reported to committee members, these three organizations perform an ROl-type calculation on each technology development proposal to compare all the new proposals in any given year prior to the annual funding commitments. Customer Buy-In At GRI and DuPont, the central technology development program was fimded by the parent organization with funds separate from those of the business units or member utilities. In these institutions, the parent organization's funds came from revenue or dues from each business unit or member utility. By contrast, the EPRT member utilities that wanted access to the products of specific technology development activities provided the funding for those activities. Both of these approaches achieved member buy-in, because in both cases the business unit or member utility proposed ways to spend technology development funds that, directly or indirectly, came from its contributions.
36 Decision Making in the DOE-OST PROCEDURES AND METHODOLOGIES TO SUPPORT GOOD DECISION MAKING Sources such as Matheson et al. (1994) and the DuPont, EPRI, and GR! interviews indicate the importance of measurement and the use of structured evaluation methods to support RD&D project priont~zation and decision making. The committee's selection of best practices most relevant to the DOE- EM situation included six directly concerned with measurement and quantitative evaluation. In interviews with industrial organizations, the importance of qualitative and quantitative technological risk assessment techniques was stressed. Finally, the detailed site visits to GRI, EPRI, and DuPont revealed further evidence of the use and value of quantifiable, probabilistic cost-benefit evaluations to help prioritize projects and allocate resources for maximum benefit. GR} in particular demonstrated that this could be done in a multicriteria setting; DuPont's experience showed that such methods can help achieve the maximum environmental cleanup per dollar expended; and the experience of many excellent industrial organizations indicates that such quantitative methods can have a powerful impact on improving the return on the RD&D expenditure. Utility and Limitations of Quantitative Evaluations The committee believes that structured methods and evaluation criteria can play a very important role in the DOE-EM technology development decision process, subject to several very important caveats. First, the quantifiable methods and results should not be viewed as making the decisions, but rather as providing very important input to the decision-making process that must be discussed and reviewed in light of all relevant factors. Second, a poorly done quantitative process is worse than no quantification at all, and often, effort and money can be spent generating poor or even misleading numerical results. Third, quantifiable results must be judged by recognizing possible weaknesses in the methodology used, uncertainty in the numerical inputs, and important decision factors that either were considered unquantifiable or were not quantified. There is often justifiable resistance to quantification since quantifying poor information may lead to misplaced confidence in the results. For decisions involving large allocations of resources, modern decision methods (e.g., Bayesian analysis) should be used, since this also quantifies the uncertainties and displays their influence on the overall decision or outcome. If uncertainty is unacceptably high, it forces deferral of the decision and additional work to improve the quality of the information inputs. The large body of methodology and experience encompassed by decision and risk analysis, probabilistic risk analysis, and cost-benefit analysis provides a wealth of procedures, methodologies and tools to implement a sound, carefully structured approach. These approaches have many common features, such as (~) focusing on finding the most productive ways to allocate scarce resources; (2) measuring benefits explicitly; in quantifiable terms whenever possible; and (3) dealing explicitly with uncertainty and technological risk using probabilities. A project evaluation system embodying these principles and applied consistently across He potential projects to be funded would help greatly to improve decision making at the project Finding {ever, as well as provide insight into portfolio evaluations, budget allocations among major program areas, and the effectiveness of the total OST budget. The published experience of Eastman Chemical Company, which has applied quantitative evaluation to guide its entire RD&D quality program, demonstrates a doubling in the expected value of the RD&D results accomplished over just a few years (Holmes, 19941. To date, OST has attempted to (~) address productive ways to allocate resources by emphasizing user input to the decision-making process and (2) measure benefits by counting technologies implemented and estimating cost reductions from using alternative-to-baseline technologies. There has been no apparent attempt to deal with uncertainty or technological risk using probabilities. These OST applications, including a way to treat uncertainties using probabilities, are discussed in more detail in Chapter 5, after Chapter 4 outlines the decision process called for and how OST has implemented the requisite process steps.
Good Practices in RD&D Decision Making 37 CHAPTER SUMMARY Although a plan for improving decision making must be tailored to the needs of a specific organization, the results (Matheson et al., 1994) from benchmarking highly regarded industnal and consortium RD&D organizations clearly demonstrate some basic points that the committee thinks are relevant for government as well as industrial organizations. There are decision-making practices, well established In leading industrial organizations as well as in EPRI and GRI, that can contribute to decision quality in an RD&D environment. High-performance orgaruzations are more likely to implement most of these practices better than organizations with lower performance, especially practices related to making quality decisions (see Appendix F for a list of these practices). Since decision processes exist in the context of an organizational structure, some of the practices apply to RD&D management, whereas others apply to the decision process itself. Drawing on published practices, the major ones that the committee recommends DOE-OST focus on are the following: Understand, focus on, and monitor changes in customer needs and requirements. Agree on clear and measurable goals. Use a formal (i.e., common, consistent, structured, and rational) technology development decision- malcing process and apply it uniformly. Think strategically (i.e., long-term and high impact). Measure and evaluate to guide resource allocation. Communicate across orgaruzational boundaries (i.e., with technology users). Continually improve the research and development (R&D) management process. Hire the best people possible and maintain expertise. Each of these recommended practices encompasses one or more of the best decision-making practices listed in Appendix F. However, it is also important to note that RD&D organizations use a strategic management process that embodies many of these practices in a mutually reinforcing manner (Lander et al., 1995~; therefore, OST should not expect that simply selecting one or two practices for implementation will be adequate to achieve excellence. These concepts have to be translated into appropriate statements for OST, a translation that the committee prefers to leave to OST management. The results from industry environmental decision malting and the planning and budgeting processes of GR] and EPR! demonstrate the use of such good practices in settings analogous in some ways to OST. Therefore, the committee thinks that these best practices are relevant to DOE's technology development decisions. The committee has identified the practices thought to be most relevant to DOE and finds that there is substantial room for improvement by OST, as indeed there is in most of the industrial organizations benchmarked by Matheson et al. (1994~. The descriptors of the best practices used here are fashioned to capture one or more related ideas taken from already published work and are offered here to describe the attributes of a successful RD&D management environment. These descriptors are written in general languages but it is not the committee's _ . _ _ _ , . . .. . . ... . ~ .. ~ . . . ~ · . .. purpose to translate the way each should be applied to obi l, to evaluate Os 1 against these statements, or to define all appropriate contexts in judging how well they apply to OST. Rather, the intent is to identify practices that OST management can use in informal internal evaluation (and improvement) of the prioritization and decision-making activity within the OST program. in the next chapter, the present status and realities of the DOE-EM situation and its limitations are examined in light of these best decision-making practices.
