Building an Effective Environmental Management Science Program: Final Assessment (1997)

The Department of Energy's (DOE's) Environmental Management Science Program (EMSP) was created by the 104th Congress to stimulate basic research and technology development for environmental cleanup of the nation's nuclear weapons complex. The program was created in the conference report that accompanied the Energy and Water Development Appropriations Bill:¹

The DOE Office of Environmental Management (EM-—which is responsible for cleanup of the nation's nuclear weapons complex—and

the DOE Office of Energy Research (ER)—which manages the Department's basic research programs—formed a partnership to develop a mission-directed basic research program focused on EM's long-term cleanup problems. According to the fiscal year 1996 (FY96) announcement for this program,² the objectives of the EMSP are to

• Provide scientific knowledge that will revolutionize technologies and clean-up approaches to significantly reduce future costs, schedules, and risks;

• "Bridge the Gap" between broad fundamental research that has wide-ranging applicability such as that performed in DOE's Office of Energy Research and needs-driven applied technology development that is conducted in EM's Office of Science and Technology; and

• Focus the Nation's science infrastructure on critical DOE environmental management problems.

The FY96 program announcement invited investigators from universities, industry, and national laboratories to submit research ideas to the Department in the form of brief preproposals. The program received 2,149 preproposals in response to these announcements. These preproposals were reviewed by federal program managers, and the proposers of 793 projects were encouraged to submit full proposals. A total of 810 full proposals were received, covering a wide range of disciplines and research topics. The Department convened review panels to evaluate the scientific/technical merit of the proposals and their long-term relevance to EM's cleanup mission and used the advice of these panelists to make 140 three-year awards totaling about $112 million.

The Committee on Building an Environmental Management Science Program was established under the auspices of the National Research Council at the request of Thomas P. Grumbly, Under Secretary of Energy, to advise the Department on the structure and management of the EMSP. The committee met seven times during the period May to

November 1996 and produced three reports: an Initial Assessment Report³ (Appendix F) that addresses the near-term needs of the program related to the FY96 proposal competition; a Letter Report⁴ (Appendix G) that addresses the development of an FY97 program announcement; and the present report, which addresses longer-term challenges and opportunities for the program.

The committee's Initial Assessment Report (Appendix F) was released on July 9, 1996—just in time for the Department's use in decision making on awards in the FY96 proposal competition. The report provided a brief review of the DOE cleanup mission and provided comments on the value of basic research to that mission. The report also provided comments on several program "challenges," in particular, challenges related to attracting outstanding investigators to the program, obtaining innovative research, and applying the results of this research to the cleanup mission.

The Initial Assessment Report's findings and recommendations were preliminary in nature, but many bear repeating in this final report and are summarized below:

• Given the size, scope, and long-term nature of DOE's cleanup mission,⁵ the committee views the establishment of the EMSP as a prudent and urgent investment for the nation. The nation's first-year financial investment in the EMSP—$50 million—is modest compared to the Department's $6.1 billion annual investment in cleanup.⁶

• Many of the nation's better scientists and graduate students have the backgrounds and skills necessary to do work at the forefront in this area but are not currently involved in research of relevance to the EMSP. The Department will need to demonstrate a long-term commitment to this research program before some of these scientists will redirect their research and graduate student training activities to the program's concerns. To this end, the committee recommended that the Department take the following three actions: (1) maintain at least level funding for the program in FY97; (2) provide full funding for all approved projects in the FY96 competition out of FY96 funds;⁷ and (3) relax the initial allocation⁸ of $20 million for proposals from academia and industry and $20 million for proposals from national laboratories to the extent allowed by the law and, instead, support the most scientifically meritorious and relevant work, regardless of the institution of origin.

• To obtain highly meritorious research proposals, the Department must help investigators become more knowledgeable about its cleanup problems, both generic problems and site-specific problems. To this end, the committee recommended that DOE prepare concise technical summaries of the critical barriers to the solution of cleanup problems with the advice of the research and research-user communities.

• For the EMSP to contribute to the long-term cleanup mission, effective mechanisms must be found to transfer the results of research sponsored by the EMSP to potential ''users" in government, industry, and academia who can utilize this knowledge to develop new or improved cleanup methods.

The reception of the committee's Initial Assessment Report by the Department was positive, and Department staff moved expeditiously to implement many of the committee's recommendations. Most notably, the Department relaxed its initial allocation of funding for university/industry and national laboratory proposals and instead made funding decisions based on merit and relevance. The Department awarded about $43 million to university/industry projects and $69

million to national laboratory projects.⁹ The Department provided full funding for university proposals but did not fully fund proposals from national laboratories, arguing that it was unable to do so.

The Department's FY97 budget request had already been submitted to Congress when the committee issued its Initial Assessment Report. However, the Congress increased the funding for this program from the Department's request of $38 million to $50 million, noting that¹⁰

The committee also recommended in its Initial Assessment Report that the Department postpone the release of the FY97 program announcement until it had more time to identify and incorporate "lessons

learned" from the FY96 program competition. This recommendation prompted the Department to request additional advice from the committee on the FY97 program announcement,¹¹ which in turn led to the production of a Letter Report. This Letter Report (Appendix G) provided an initial assessment of the FY96 proposal competition and offered advice on several aspects of the FY97 program announcement. The committee concluded that it did not have enough time, nor was enough information provided, to assess the overall success of the FY96 competition. However, the committee noted that, where it did have firsthand information, it was able to confirm the overall quality of the proposals, the review process, and the review panelists.

The Letter Report also offered suggestions on several elements of the FY97 program announcement, most notably the following:

• Criteria for proposal review and selection. The committee recommended a continued focus on basic research in the program announcement, with scientific merit and long-term relevance to EM's cleanup mission as the primary proposal screening criteria.

• Research areas. The committee suggested that the EMSP would be more likely to attract innovative proposals from creative investigators if the program announcement contained information on EM's problems that require basic research. The committee also encouraged the Department to broaden the solicitation to include problems related to risk, quantitative methodologies, and health assessment.

• Review process. The committee reaffirmed its endorsement of the two-stage review process—consisting of a scientific and technical merit review followed by a relevance review—and recommended that this process continue to be managed as a partnership between ER and EM. In addition, the committee recommended that the Department maintain some continuity in the merit and relevance review panels to take advantage of the experience gained in the FY96 competition. The committee also recommended that ER convene the merit review panels under the Federal Advisory Committee Act (FACA) to allow the

panelists to provide formal consensus on scoring and ranking of proposals to DOE.

• Financial plan. The committee expressed its concerns about the "mortgage" from the FY96 proposal competition¹² and reaffirmed the recommendation in the Initial Assessment Report that successful proposals be fully funded "up front."

• Outreach. The committee encouraged the Department to explore additional mechanisms to make the research community more broadly aware of the FY97 proposal competition, for example, through the use of paid advertisements in professional journals.

At the time the present report was prepared, the Department had not released its FY97 program announcement; consequently, the committee was not able to determine the extent to which its advice was followed. The committee has received informal feedback from Department staff that suggests that many of the recommendations will be implemented, with the exception of full funding for national laboratory proposals and the use of FACA panels for merit review, which will not be implemented unless certain institutional obstacles are overcome. The committee provides some additional comments on these issues later in this report.

The purpose of the present report is to provide a more detailed assessment of the EMSP than was possible in the committee's Initial Assessment Report, which was prepared on a very tight schedule and with the benefit of only two committee meetings. The primary focus of the present report is on the long-term challenges and opportunities for the program as noted in the Statement of Task, which is given in Appendix A.

The chapters of this report have been structured to address the Statement of Task questions in Appendix A. Chapter 2 addresses the value of basic research to the cleanup program, recapitulating and extending the comments made by the committee in its Initial Assessment Report and Letter Report. Chapter 3 addresses the five questions under "science needs" in the Statement of Task in the context of the development of a science plan for the program. Chapters 4 and 5 address the four questions under "management needs" for the program. Chapter 4 deals with proposal selection and funding, whereas Chapter 5 addresses the structure and operation of the program. In addressing its task statement the committee focused on the "big picture" issues that are likely to be of value to the Department, recognizing that the details are best left to program staff.

The committee was not able to address comprehensively all of the task statements for the following two reasons:

1. The committee worked to a series of deadlines set by the FY96 proposal competition and FY97 program announcement processes. The committee was able to affect the initial EMSP program only by producing its reports in a much more rapid fashion than is usual for NRC studies. The committee has been successful in providing guidance to the initial program but has not gone into depth in some areas, most notably the selection of specific research areas for emphasis, because this task would have exceeded the time allotted to the committee for the completion of its work and possibly its expertise.

2. The EM program itself lacks clear objectives, including what will be the land uses at the DOE sites and what the standards are by which "cleanup" will be judged to be completed. These issues have been addressed by other reports,¹³ which indicate that DOE has expended large

amounts of funds but accomplished little. The new 10-year vision plan of the Assistant Secretary for Environmental Management, which is discussed in Chapter 3, is based on setting objectives. However, this plan acknowledges that the most difficult problems, dealing with transuranic (TRU) waste and high-level waste (HLW),¹⁴ will not be resolved in the 10-year period. This lack of objectives in the EM program itself, in the view of some committee members, is a serious flaw in trying to develop a needs-based basic research program.

The committee obtained a wide range of oral briefings and written documentation during this study. A list of briefings received at the committee's open meetings is given in Appendix B. The committee received several detailed briefings from EM and ER program staff on the structure and management of the EMSP, proposal review and award procedures, and results of the FY96 proposal competition. The committee also received extensive written documentation from the Department, including a multi-volume record and a data table that provided a list of all projects funded in the FY96 proposal competition that included principal investigator (P.I.) and co-P.I. names and affiliations, biographical sketches of the P.I.s, abstracts of funded projects, and current DOE funding. Additionally, the committee received in confidence about half of the names of the FY96 relevance review panelists from EM.

The committee received oral briefings from staff at DOE, the National Science Foundation (NSF), the U.S. Environmental Protection Agency (EPA), and the National Institutes of Health (NIH) on other federal programs of relevance to the EMSP. The committee also received oral briefings from federal and private-sector managers on effective R&D program management and assessment strategies.

In its efforts to understand the cleanup challenges at the weapons complex, the committee solicited and received an extensive set of

briefings over two meetings from DOE, contractor, and national laboratory staff. The first set of briefings reviewed cleanup problems and R&D needs at five of the largest sites—Hanford, Savannah River, Rocky Flats, Idaho Falls, and Oak Ridge. The second set of briefings reviewed cleanup problems arranged by focus area—landfills and plumes, tanks, mixed wastes, and decontamination and decommissioning. The committee found these briefings to be helpful in clarifying its thinking about the need for basic research in the weapons complex.

During the course of this study, the committee made numerous and significant requests for information from Department staff. The committee found the staff to be responsive to requests for information, with one exception as noted below, and the committee generally was satisfied with the quality and completeness of the information it received and the willingness of Department staff to respond in a timely fashion. The committee also was pleased by the candor of Department staff in discussing program problems and their willingness to engage in wide-ranging and vigorous discussions of the program at the committee's open sessions.

The one exception involved the committee's request for the names of the merit review panelists from ER staff. The committee requested these names as part of its efforts to assess the quality of the peer review in the FY96 proposal competition (see Chapter 4) but was told that it was ER practice to keep the names confidential. After discussions with ER staff over the course of three committee meetings, ER staff agreed to contact the panelists to see if they would agree to have their names released to the committee. The panelists had not been contacted by the time of the committee's penultimate meeting.