Part III

Priority-Setting, Timing, and Staging



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 57
Improving the Environment: An Evaluation of DOE's Environmental Management Program Part III Priority-Setting, Timing, and Staging

OCR for page 57
Improving the Environment: An Evaluation of DOE's Environmental Management Program SUBCOMMITTEE ON PRIORITY-SETTING, TIMING, AND STAGING EDWIN H. CLARK II (Chair), President, Clean Sites, Inc. HUGH J. CAMPBELL, Jr., Environmental Manager, DuPont MARY R. ENGLISH, Associate Director, Energy, Environment, and Resources Center, University of Tennessee DONALD R. GIBSON, Department Manager, Systems Analysis, TRW Environmental Safety Systems ROBERT E. HAZEN, Chief, Bureau of Risk Assessment, New Jersey Department of Environmental Protection THOMAS LESCHINE, Associate Professor, School of Marine Affairs, University of Washington ROBERT H. NEILL, Director, Environmental Evaluation Group, New Mexico LYNNE M. PRESLO, Senior Vice President, Technical Programs, Earth Tech ANNE E. SMITH, Principal and Vice President, Decision Focus, Inc. MERVYN L. TANO, General Counsel, Council of Energy Resource Tribes Staff Robert Andrews, Senior Program Officer Patricia Jones, Project Assistant

OCR for page 57
Improving the Environment: An Evaluation of DOE's Environmental Management Program Introduction The Subcommittee on Priority-Setting, Timing, and Staging was established to review two areas of concern for Assistant Secretary Grumbly. The first, linked directly to priority-setting, dealt with the process of setting priorities for environmental management activities and how the process incorporates societal values, costs, current regulations, and risks to the environment, public health, and worker safety. The second, focusing on the issues of timing and staging, dealt with how the environmental management program can schedule technology development and remediation and restoration efforts to maximize cost savings and minimize risks to the environment, public, and workers. The two issues are closely related. When to undertake a particular activity and how best to organize its components—timing and staging —depend on the priority that the Department of Energy (DOE) attaches to completing the activity. Similarly, which activities should be undertaken first—priority— depends on the options, requirements, and advantages and disadvantages related to the timing and staging of the possible activities. The subcommittee has concluded that priority-setting for DOE's Environmental Management Program has been problematic more for management reasons than for technical reasons. Important features that are essential for a sound priority-setting process at DOE, but that the subcommittee perceives as still lacking are the following: Clearly stated goals that are the fundamental end point of the priority-setting decisions.

OCR for page 57
Improving the Environment: An Evaluation of DOE's Environmental Management Program Stakeholder involvement in the priority-setting process that is both timely and integrated between local and national levels. Priority-setting that is comprehensive in scope (including intersite rankings among different geographic regions) and that goes beyond risk-ranking. A range of organizational and cultural changes are also necessary to achieve as part of filling these gaps. Tools exist that can support most of the priority-setting needs in a technically sound manner once the management issues have been addressed, although they will need refining and adjustment to suit the specific needs. The rest of this report explains the management issues in more detail and provides some general guidance on the usefulness of different supporting tools.

OCR for page 57
Improving the Environment: An Evaluation of DOE's Environmental Management Program Current Realities and Historical Context The subcommittee recognizes the extreme difficulty of establishing priorities in an agency as complex as DOE. DOE has a number of missions. It has a national-defense mission, an energy-security mission, an environmental-quality mission, and a basic-research mission in support of its other missions. Moreover, DOE seeks to contribute to the nation 's economic productivity by collaborating with industry wherever its established missions have provided an expertise that some industrial partner wishes to share. The programs of DOE that are undertaken in the pursuit of its missions inevitably have some goals that are inconsistent and some that are actually in conflict. In addition, DOE's priority-setting efforts will be affected by such factors as shrinking budgets; institutional relationships between DOE Headquarters, field offices, and contractors; and local and national political considerations. The subcommittee has sought to develop recommendations that will be useful and durable in the face of those disparate and changing circumstances. DOE must and does decide what actions to take and how to spend its resources. In making its decisions and undertaking its actions, DOE is perforce establishing priorities. The subcommittee has gained, in the short time available, as much understanding as it could of the historical context and current practices for setting priorities in the DOE Office of Environmental Management (EM). EM has its roots in a 1989 reorganization of DOE. At that time, it was apparent that the activities associated with waste management and environmental restoration were increasing in budget and complexity and that if the demands of federal and state regulators were to be met, a centralized planning

OCR for page 57
Improving the Environment: An Evaluation of DOE's Environmental Management Program process within DOE that could take into account the different situations across the entire DOE complex was needed. Before then, activities had been dictated largely by the desires of site managers, and their needs were not the principal concerns of the programmatic assistant secretaries who had responsibilities for the sites. With the creation of the Office for Environmental Restoration and Waste Management (later the Office of Environmental Management) came an attempt at more centralized planning; meanwhile DOE was continuing in its efforts to accommodate federal and state environment regulators who had only recently been given some jurisdiction over the sites. While the 1989 reorganization was taking place, dramatic reductions occurred in the defense-related activities of DOE with the conclusion of several agreements between the former Soviet Union and the United States on reducing the number of nuclear weapons. DOE, states, and the Environmental Protection Agency (EPA) were establishing consent agreements establishing the outline of site-specific remediation efforts. It is not inconsequential that DOE has engaged in a massive environmental-remediation effort at the same time that the defense weapons complex has been declining. At some DOE sites, states and localities may see it as a high-priority matter of DOE's EM program to “fill the gap” with respect to employment and economic activity, whereas DOE and some taxpayers may see the expeditious, economic, and safe return of sites to the local communities as having high priority. Another turn of events important for understanding the present context of the EM program was DOE's loss of some of its self-regulatory status in the environmental arena. Before 1980, DOE generally considered itself to be largely responsible for its own environmental performance. During the 1980's, however, state and federal environmental regulators gained partial jurisdiction over DOE sites—often in an atmosphere of distrust and hostility. Today, many of the priorities in the EM program are set by the 100-odd compliance agreements that DOE has entered into with EPA and the states. These agreements have often become the primary “legal” drivers for EM budgetary decisions in DOE. Requests for funds from field sites and the later requests by DOE to the Congress for funding are driven largely by compliance with federal and state statutes and agreements. EM developed and tested a highly sophisticated priority-setting tool for setting environmental restoration priorities called the Environmental Restoration Priority System (ERPS) from 1988 to 1991. However, the DOE discontinued the development and use of this system because of strong opposition by the states and other stakeholders who felt that the system had been developed without their input (Jenni et. al., 1995). More recently EM established a set of 6 goals to guide its budget formulation process: Urgent risks and threats.

OCR for page 57
Improving the Environment: An Evaluation of DOE's Environmental Management Program Workplace safety. Managerial and financial control. Outcome orientation. Focused technology development. Strong partnerships with stakeholders. The subcommittee defined its task as providing recommendations to DOE for improving its priority-setting system so that it allocates its available resources at its facilities to manage wastes, restore degraded environments, and otherwise protect the public's health and welfare in a cost-efficient and credible manner.

OCR for page 57
Improving the Environment: An Evaluation of DOE's Environmental Management Program An Inclusive System Priority-setting is often thought of in a limited sense, for instance, as applied to deciding which of many items on an agenda should be undertaken first or how much of an available budget should be spent. In the context of DOE, it can be used for screening activities, intra-site ranking of similar projects, intra-site ranking of projects in different areas, site-to-site ranking, etc. To be successful, a priority-setting system1 should be comprehensive in scope, addressing all forms of EM decisions and activities and addressing all DOE sites as a group. It must be technically sound, but it also needs to be rooted in the organization 's basic visions about its purpose and goals. At the same time it is important to note that DOE does not require a sophisticated system to identify the highest-risk cases first; we recommend that DOE continue to act immediately to identify high-risk cases. The subcommittee believes that priority-setting, timing, and staging are comprehensive planning activities that must take place within an organized and effective management context. Organization and management set the context for achieving progress in priority-setting, timing, and staging decisions. Progress in such decisions cannot be achieved simply through application of new or improved tools and analytical techniques. DOE requires a fundamental and pervasive change throughout the organization. 1   The subcommittee uses the term “priority-setting system” as opposed to “priority-setting process” to emphasize that we believe that priority-setting must extend in many ways throughout many aspects of the DOE organization, and cannot be limited to a specific process that functions independently of these other parts of the system. The term “system” should not be taken to mean a specific tool or methodology.

OCR for page 57
Improving the Environment: An Evaluation of DOE's Environmental Management Program This is not a trivial undertaking. Indeed, EM has a particularly daunting assemblage of sometimes inconsistent and even conflicting responsibilities and activities that it needs to harmonize if it is to establish an effective and efficient priority-setting system. EM must deal with: A need to balance fairness against efficiency and optimization. Substantial differences in what is perceived to be acceptable risk for workers, the general public, and the environment. Missions that range from the correction of environmental releases and the prevention of releases to safeguarding nuclear materials vital to the nations defense. A large variety and number of sites and contractors. Multiple regulatory requirements in DOE, EPA, Occupational Safety and Health Administration, state, and multiple-party agreements. Different beliefs of affected parties regarding the goal of ultimate land use. All government organizations have to deal with such conflicts and justify their actions and requests for funds to fulfill their responsibilities. EM, however, faces a particularly great challenge because of its poor record of environmental restoration, the extremely high costs of carrying out its responsibilities (cost estimates for the cleanup alone exceed $230 billion dollars), and the absence of yardsticks to measure progress (DOE, 1995a). EM recognizes its problems and has made initial attempts to improve the way in which it makes and implements decisions. The recent Report to Congress, Risks and the Risk Debate: Searching for Common Ground (DOE, 1995d), is a step toward recognizing that funding and other constraints will preclude complete environmental restoration and preservation to everyone's satisfaction. The subcommittee did not review the report and cannot endorse its specific methodology or accuracy. Although the report and the recently adopted changes in the DOE budget process demonstrate an initial effort to resolve the conflict between limited resources and unlimited wishes, future attempts at evaluating risks in relation to budget priorities should give more consideration to the optimum utilization of quantitative techniques and of outside peer-review panels, verify the values assigned to different elements of the risk assessment, and provide stakeholders with assurance of the quality of the analyses. It will also be important to include stakeholders earlier if the process is to serve as a means of consensus-building for setting priorities (NRC, 1994a). DOE has taken a step in this direction by initiating the Consortium for Risk Evaluation with Stakeholder Participation (CRESP) to provide independent peer review and structured interactions with stakeholders. Although they constitute an improvement, the actions that the agency has already taken are only a start, and EM has not yet achieved a comprehensive and inclusive priority-setting system that will provide direction and guide the decisions that the agency must make in the coming years. Only when

OCR for page 57
Improving the Environment: An Evaluation of DOE's Environmental Management Program DOE has established a more-comprehensive system will it be able to spend money wisely to manage risks to workers, the public, and the environment and to instill confidence in the public and Congress that it is doing so. Only by establishing a more-coherent system of priority-setting will DOE be able to break out of the incrementalism that characterizes its current decisions and prevents true priority-setting from taking place. The agency needs to define what it is about in a coherent set of statements and processes that extend from the general and abstract to the specific and measurable, from defining its role to the specific steps that it will take to implement that role. It needs to define EM's mission, vision, goals, and objectives. Mission. Congress usually specifies an agency's mission in the legislation that defines its programs and activities. These are the work programs that allow the organization to achieve its vision. Are they consistent with the agency's vision? Are they consistent with one another? Do they define what the agency has to do to arrive at the state it has defined in its vision? How much flexibility does the agency have to modify these statements to make them consistent with its vision? Vision. The vision provides the agency, its staff, and the public an integrated look at the organization's future state. What does the agency want to accomplish? How does it want to view itself? How does it want the public to view it? Goals. Goals are targets for components within the mission, i.e., what the organization is trying to achieve in the short and long term. For example, is EM trying to maximize the amount of DOE land that will be available for public use? Is it trying to contain waste and contamination and restrict land use to the greatest possible extent to minimize costs? Is it going to have a comprehensive technology-development program to reduce costs of waste management and environmental restoration activities? EM must establish specific goals (both short-term and long-term) to implement a priority-setting system. After national goals have been established, sites should be allowed to develop alternatives in consultation with stakeholders and Headquarters. Sites, with strong input from local stakeholders, should relate each activity to national goals or their corresponding site-specific objectives. However, this should be done according to standard protocols, guidance, and formats developed by EM Headquarters to permit inter-site comparisons. Open reviews should be held at the local and national levels for site priority-setting. Revised priorities that result from stakeholder or EM review should be communicated to all parties. Objectives. Objectives are more-specific, short-term, and quantifiable measures of accomplishment in pursuit of the agency's goals, mission, and vision. Goals can pertain to many facilities or activities; objectives generally

OCR for page 57
Improving the Environment: An Evaluation of DOE's Environmental Management Program pertain to single facilities or activities. Objectives provide answers to such questions as these: What parts of each installation will be cleaned up with the intent of release to public use? What types of wastes will be accepted for storage or treatment at each installation? What will be the role of repositories as part of the long-range management of risks? What backup plans are made? Where will the repositories be, and what volumes of waste will they be able to accommodate? What areas of the current complex will retain long-term access restrictions? What types of risks will be managed through long-term DOE stewardship rather than complete remediation? The success of a priority-setting system ultimately depends upon how well it is actually implemented. For example, will the vision for EM be achieved by dismantling all or selected DOE production facilities? Should EM establish regional waste repositories for ultimate disposal of certain wastes? Should EM target technology-development activities for the most-costly problems or problems with no current technical solution?

OCR for page 57
Improving the Environment: An Evaluation of DOE's Environmental Management Program with the usual quantitative approaches is that they are “black boxes” and that the fundamental nature of differences among alternatives is lost when all the decision criteria are collapsed into a single numeraire, such as utils or dollars. (Thus, they are weak in transparency and to some extent in the clarity with which facts are distinguished from values from the observer's perspective.) It is not necessary, however, to address the multicriteria tradeoffs required in priority-setting through strictly quantitative techniques, such as Multi-Attribute Utility Analysis (MAU). All that is required of a priority-setting tool is that it explicitly recognize the nature of tradeoffs inherent in specific choices of actions. Alternative multicriteria approaches that avoid focusing on a single score or value estimate might also be useful. The drawback of such approaches is that information about multiple criteria can be voluminous and difficult for decision-makers to assimilate if not summarized into comparable (unimetric) units. It is possible, however, to devise visual displays in a variety of formats that assist interested parties and decision-makers in mentally processing and understanding the tradeoffs embodied in multicriteria information. Such approaches avoid the step of having the system produce specific rankings, or “recommended” actions; they leave such conclusions to be drawn by each person viewing the results, but they provide the basis for justifying one's own conclusions. In addition to greater transparency, nonaggregating multicriteria approaches have the advantage of using neither a single person's preferences nor a highly uncertain representation of societal preferences, so they avoid potential concerns with bias. Nonaggregating approaches also allow exploration of alternative viewpoints in a format that is more conducive to consensus-building: the process helps people to visualize the perspective that other interested parties have with respect to a particular choice.2 At the site and facility levels, a number of formal priority-setting tools appear to be in use for setting budget and other priorities. One example is the Laboratory Integration and Prioritization System (LIPS), which is a multiattribute utility-based approach that was developed at Los Alamos National Laboratory and used in demonstration applications at a number of DOE sites (B. Anderson at the subcommittee 's workshop of June 26–27, 1995). The subcommittee did not review LIPS and cannot provide an overall evaluation, but we believe that LIPS should be carefully considered for its alignment with DOE's goals, once DOE's overall priority-setting system is more effectively structured. LIPS might not be readily applied to the intersite 2   An example of such an approach is EPA's multicriteria decision-support system developed for setting priorities among strategies for adapting to global climate change, called the Adaptation Strategy Evaluator. EPA's experience in initial applications of this approach is that it is particularly useful for identifying strategic directions without the detailed analysis required of CBA, for gaining insight into why some strategies appear to meet goals better than others, and for building consensus.

OCR for page 57
Improving the Environment: An Evaluation of DOE's Environmental Management Program priority-setting step, but it can readily be used for priority-setting among the many activities possible at a given location. An intersite priority-setting tool could then be applied with such site inputs as LIPS might be able to produce. ERPS is a detailed application of the MAU approach that is tailored explicitly to deal with the many unique features of DOE's EM organization in the process of allocating budgets among sites. Although questions have been raised on particular technical points, its main limitations are that it had poor stakeholder involvement during its development and depends excessively on judgments made by the model builders rather than reflecting widespread stakeholder consensus; it is perhaps too ambitious in attempting to incorporate all values, no matter how subjective and qualitative, into a single quantitative metric; and, as a result it failed to meet the objectives of clarity, transparency, and simplicity. This subcommittee has not reviewed ERPS in detail and so cannot make any comprehensive evaluation of it. Nevertheless, it stands as a major contribution toward an MAU-based priority-setting tool. It is unlikely that any other MAU-based tool would substantially improve on the foundations that exist in ERPS, and we recommend that the DOE allocate tool-development resources to improving the usefulness of ERPS rather than trying to develop a new MAU model to replace it. Improvement of a system that has already been heavily funded is generally more appropriate than starting from the beginning on a new system. It must be recognized that the budget-allocation process that DOE faces is extremely complex, and MAU-type analyses, which combine all ranking criteria according to a system of weights, will probably be essential for obtaining a system wide sense of priorities. However, DOE might wish to consider using techniques that keep individual priority-setting factors separate in conjunction with those that merge priority-setting factors. In this way DOE can focus on generic priority-setting issues, such as technology development versus immediate cleanup, and also provide better communication and interaction with interested parties as DOE goes through its priority-setting system.

OCR for page 57
Improving the Environment: An Evaluation of DOE's Environmental Management Program Conclusions Establishing a robust priority-setting system for an organization as diverse and complex as EM is clearly not a simple task. The subcommittee has focused more on the attributes and characteristics of an effective priority-setting system than on the specific tools or criteria that it might employ. The organizational environment in which such a system is established and implemented may well be as important as the system itself. This organizational environment will probably be more important than the specific tools or criteria the system employs, and it will also help to determine whether the system satisfies the attributes of permanency, consistency, clarity, transparency, simplicity, and stakeholder involvement that the subcommittee recommends. An effective priority-setting system is more than just a process of making budget decisions. It needs to incorporate specific goals and objectives so that progress can be measured. It needs to include metrics for measuring how much progress is truly being made towards accomplishing these goals and objectives. It needs to incorporate feedback mechanisms that will stimulate corrections in the decision-making process if the metrics demonstrate inadequate progress. It needs to be reflected in all of the personnel, contracting, and other administrative and decision-making processes in EM, not just in the budget process. It must be accompanied by a clear series of incentives and disincentives that reflect goals and reinforce the system. It needs to demonstrate that decisions in all these areas, both within DOE sites and across sites, are being made consistently. Again, establishing such a system is not a simple task. It will require substantial effort and input by Headquarters and site personnel, technical

OCR for page 57
Improving the Environment: An Evaluation of DOE's Environmental Management Program experts, regulators at the state and federal level, elected officials, the general public, and other stakeholders. The subcommittee believes, however, that this is an investment that should be made. The decisions EM is making will affect the allocation of tens of billions of dollars, determine how significant human health and environmental risks are addressed, affect the jobs of hundreds of thousands of workers, determine the future use of millions of acres of land, affect local and regional economies throughout the nation, and have an impact on national and international security. Clearly, these are decisions that need to be made carefully and correctly. Clearly, the investment is worth it. The subcommittee does not want to suggest that all decision-making be put on hold until this investment is completed. Decisions must be and will be made regardless of whether an adequate priority-setting system exists. Certainly the DOE has some potentially high-risk situations that it must address. It does not require a sophisticated system to address these worst cases first, and we recommend that it do so. We also recommend that, in the interim, EM postpone those actions that do not significantly reduce risks or save money, and that are not required by current laws. Even when there appear to be current legal requirements, if the action is not addressing a significant risk, we recommend that the agency “push back” on the regulatory drivers. Ultimately, in this period of tight federal budgets, everyone will benefit if the agency can demonstrate that it is efficiently spending its resources on the most serious problems. In short, there is plenty to do and there is a need to do it right. We believe that EM has taken some useful first steps, but it has many more to go.

OCR for page 57
Improving the Environment: An Evaluation of DOE's Environmental Management Program References DOE (U.S. Department of Energy). 1991. Tank Waste Disposal Program Redefinition. Prepared for the US DOE Office of Environmental Restoration and Waste Management. WHC-EP-0475. Rev.0. Washington, D.C. DOE (U.S. Department of Energy). 1995a. Estimating the Cold War Mortgage: The 1995 Baseline Environmental Management Report. Volume I, March 1995. U.S. Department of Energy, Office of Environmental Management, Washington, D.C. DOE (U.S. Department of Energy). 1995b. Risks and the Risk Debate: Searching for Common Ground “The First Step”. The U.S. Department of Energy, Office of Environmental Management Washington, D.C. Jenni, Karen E., Miley W. Merkhofer and Carol Williams. The Rise of a Risk-Based Priority System: Lessons from DOE's Environmental Restoration Priority System. Accepted for publication by Risk Analysis. April 10, 1995. NRC (National Research Council). 1994a. Building Consensus Through Risk Assessment and Risk Management in the Department of Energy's Environmental Remediation Program. National Research Council, Washington, D.C. NRC (National Research Council). 1994b. Ranking Hazardous-Waste Sites for Remedial Action. National Academy Press. Washington, D.C.

OCR for page 57
Improving the Environment: An Evaluation of DOE's Environmental Management Program Appendix ONE APPROACH During the subcommittee's deliberations, one member was tasked with developing an approach to priority-setting that would incorporate the various characteristics and attributes that the subcommittee believed were important. This appendix contains the result of that effort. The proposal stimulated some controversy. Some subcommittee members thought it was a useful contribution which would clearly improve the budget formulation and other priority-setting processes. Other members and reviewers considered it naive, academic, infeasible, and bureaucratic. Because of time constraints, the subcommittee was unable to perfect the proposal in response to these comments. Thus the original proposal is included in this appendix, without the subcommittee's modification or endorsement, in the belief that it may contain some concepts that would be of interest and value to the Department. The proposed framework builds upon the existing budgeting process and incorporates substantial involvement at the local level although leaving the decision-making ultimately at the national level. The framework begins with the clear articulation of DOE's priorities for EM expenditures and, proceeding through a nine-step process, finally gives some leeway to the local stakeholders to make a final choice about the priorities which best suit them. Define Priorities: The proposed process would begin with EM, with substantial input from its various stakeholders, identifying the full list of factors that should be taken into account in setting priorities.

OCR for page 57
Improving the Environment: An Evaluation of DOE's Environmental Management Program We suggest that DOE consider developing the full list of factors through a “bottom-up” approach. In such a process, the stakeholders at the individual facilities would be asked to review an initial list of factors and recommend modifications or additions to it. These reviews and suggestions would be aggregated through the regional offices and combined at Headquarters. They would then undergo a process of consolidation and redefinition. The goal of this process would be to come up with a manageable set of priority factors which are inclusive and clearly defined so that they are interpreted consistently by everyone involved in the EM priority-setting system, from Headquarters staff to local advisory panels and other stakeholders. DOE might want to consider involving a national stakeholders panel in this winnowing and definition process to ensure that the criteria of inclusiveness and clarity are met. Select Primary Factor: The second step would be for DOE to select from among this list what it considers to be the most important priority-setting factor. This should be done at the Headquarters level with the full involvement of the Assistant Secretary and Secretary. The primary factor would most likely be related to the risk associated with an activity. Examples of possible primary factors related to risk would be the risk that would exist were the activity undertaken, the reduction in risk that would result from undertaking the activity, or the risk reduction per dollar spent that would be associated with undertaking the activity. It is important that the primary factor demonstrate two characteristics. One is that it can be quantified on a scale that allows activities to be ranked from most important to least important. The second is that the factor metric, i.e., the means of estimating the extent to which each activity satisfies this factor, be determined consistently across all of the activities. The value placed on the primary factor metric must be independent of where the activity would occur and free of any biases that the rater might have. Assuming that risk or risk reduction is a component of the primary factor, this second criterion suggests the advantage of having risk assessments carried out by a neutral third party. Appoint Trustees for Other Factors: The third step would be to designate trustees to represent each of the other priority-setting factors. These trustees should be Headquarters employees, probably assigned to the budget office. Their responsibility would be to help EM achieve the best possible priority-setting result by ensuring that the issue for which they are the trustee is given adequate consideration in this process. Their primary loyalty must be to the DOE, not to the issue. Their goal should be to ensure that EM's budget and other priority-setting systems reflect rational, efficient, and equitable weighing of all of the DOE's priorities. Their responsibility would be to see that their factor is given due consideration.

OCR for page 57
Improving the Environment: An Evaluation of DOE's Environmental Management Program This does not mean maximum consideration. They should not operate as representatives of particular interests who try to maximize the resources dedicated to these interests. They should operate as trustees of these interests who ensure that the interests are given fair and adequate consideration. This is a subtle but important distinction. It is to clarify their role as trustees rather than representatives that argue for them being employed in the Headquarters budget office. Another mechanism for discouraging them from taking on a representational rather then a trustee role would be to periodically reassign factors among the trustees. To fulfill their responsibility they should learn as much as possible about why local stakeholders may have ranked various activities as very important according to their factor. In many cases this importance would be communicated to them by the stakeholders most concerned about this factor. If this communication does not occur naturally, the trustee would have a responsibility to obtain such information by talking to the local interests after reviewing the activity summary sheets described below. Activity Summary Sheets: The budgeting or other priority-setting system should be based upon activity summary sheets similar to those DOE is apparently using at present. The summary sheets should include a terse description of the activity and a rating of the importance of the activity with respect to each of the priority-setting factors. To the extent possible, the rating for the primary factor should be quantitative on a cardinal scale. The ratings for the other factors would also benefit from reasonable quantification, but a qualitative indication of relative importance would suffice as well. The development of these summary sheets should begin at the local level and be fully informed by review and contributions of local advisory panels and other stakeholder input. There may be a need to modify the ratings as the summary sheets are reviewed at the regional and national level in order to improve their commensurability, but the reasons for such modifications should be indicated and be part of the public record. These summary sheets would provide the basis for, and a public record of, the priority-setting system. Initial Ranking: The activities would be initially ranked, from most important to least important, according to the primary factor. If the evaluation of this factor across activities has met the criteria of objectivity and consistency, this initial ranking would be a straightforward tabulation of the information on the summary sheets. The results of this initial ranking would then be distributed to the trustees and possibly throughout the system including the local stakeholder groups, in preparation of the re-ranking process. Re-Ranking: During the re-ranking process the trustees would argue for activities to be raised or lowered in the ranking because of the importance of the factor for which they are the trustee.

OCR for page 57
Improving the Environment: An Evaluation of DOE's Environmental Management Program For example, the trustee representing cultural values might argue for a particular activity to be raised in the ranking because of the extent to which this activity would satisfy a strong tribal, cultural, or spiritual value in spite of the activity's relatively low impact on human health risks. The trustee representing the cost reduction priority might argue that certain activities be lowered in the ranking because they address risks that would be attenuated naturally if nothing were done. A containment and wait strategy might result in significant cost savings in such a case over the long run. The trustee representing compliance requirements, on the other hand, might argue that the same activity be raised in the ranking because it is explicitly identified in a tri-party agreement, and failing to undertake this activity could result in a strong enforcement action by the applicable regulatory agency. The trustee representing EM's innovative technology priority might argue that a particular innovative technology activity be raised in the ranking and several clean up activities be lowered, because the innovative technology activity promises to achieve significant cost savings in these types of cleanups in the future. If such a re-ranking were agreed upon, the individuals responsible for those cleanup activities would have an incentive to ensure that the technology development activity achieved that goal. To the extent possible, re-ranking decisions would be made by a consensus of the trustees and the individual responsible for presenting the ranking to the Assistant Secretary (usually the budget director). Where consensus is not reached, the disagreement would be noted for final resolution by the Assistant Secretary. In order to satisfy the objective of transparency, the reasons for any reranking decisions should be made explicit, written down on the activity summary sheet, and made part of the public record of the priority-setting exercise. Proposed Final Ranking: The proposed final ranking would be presented to the Assistant Secretary along with a summary of the disputes that could not be resolved by the re-ranking committee. If there were an unreasonably large number of unresolved disputes, each of the trustees might be limited to identifying one or two disputes that the particular trustee wanted raised to the Assistant Secretary's level. The ranking would not be changed for those disputes that were not raised. If a large number of disputes resulted from disagreement among the trustees about the relative importance that should be accorded particular priority factors, this question should be raised to the Assistant Secretary for guidance before the re-ranking is completed. The goal would be to have only the most important issues raised to the Assistant Secretary's level. Final Ranking: The Assistant Secretary would make decisions regarding the disputes raised to his or her level to create EM's final ranking. Depending upon the priority-setting system that is being undertaken, EM's

OCR for page 57
Improving the Environment: An Evaluation of DOE's Environmental Management Program final ranking will usually undergo further reviews and modifications before it becomes truly final. Depending upon the rules that the DOE, the Office of Management and Budget, or others might have imposed upon this process, it may not be possible to make the final EM ranking or subsequent modifications public. Nevertheless, at some point there will be a final “ranking” of some sort which will be public. In the case of the budget, the form of this final ranking will be a list of those activities which are included in the budget. The more public this process is, the more it will satisfy the criteria of transparency and credibility that are important for a priority-setting system. A Possible Final Local Review: The process described has substantial local involvement in defining and rating the importance of the activities. However, the ranking and re-ranking are carried out, albeit informed by this local information and assessment, predominately at the national level. The DOE may, if it is allowed to do so, want to provide the local level with a final opportunity to modify the ranking. This could be done by allowing the local site-specific advisory group to propose the substitution of a local activity that is below the cutoff line (e.g., unfunded in the budget) for another local activity of equivalent cost that is above the cutoff line. The presumption would be that such a substitution would be allowed as long as it was not unreasonably inconsistent with national priorities, particularly the primary priority factor.

OCR for page 57
Improving the Environment: An Evaluation of DOE's Environmental Management Program This page in the original is blank.