Executive Summary
THE current strategic plan of the U.S. Environmental Protection Agency (EPA), published in 1997 in response to the Government Performance and Results Act, states that the agency's overall mission is to protect human health and to safeguard the natural environment. Charged to implement a disparate collection of federal laws that address various categories of environmental problems, EPA has been primarily a regulatory agency. It has not had a primary “science” mission in the same sense that the National Institutes of Health (NIH) or the National Science Foundation (NSF) have primary missions to advance scientific and technical knowledge through research. Yet, EPA's strategic plan strongly acknowledges that environmental protection efforts need to be “based on the best available scientific information,” and “sound science” is one of the agency's avowed major goals.
Over the 3 decades since EPA was created, great progress has been achieved in cleaning up the nation's worst and most obvious environmental pollution problems, but many complex and difficult tasks remain. The environmental problems of today are often difficult to diagnose and treat; they cross state and national boundaries, entail difficult trade-offs, and sporadically present unpleasant surprises. Past illusions about simple and easy solutions to environmental problems have been replaced by the realization that environmental protection is often complicated and challenging.
Scientific knowledge and technical information are essential for determining which environmental problems pose important risks to human health, ecosystems, the quality of life, and the economy. We need scientific information to avoid wastefully targeting inconsequential problems while ignoring greater risks. We need such information to reduce uncertainties in environmental decision-making and to help develop cost-effective strategies to reduce risks. We need science to help identify emerging and future environmental problems and to prepare for the inevitable surprises.
This report is the fourth and final one in a series prepared by two companion expert committees convened by the National Research Council (NRC) in response to a request from Congress and to subsequent, related requests from EPA for an independent assessment of the overall structure and management of the agency's research program, as well as for an evaluation of scientific peer-review procedures used by EPA. To carry out the study, the NRC appointed the Committee on Research and Peer Review in EPA, which prepared an interim report addressing the initial request from Congress, and this, the final report in the study. Also as part of the study, the NRC appointed the Committee on Research Opportunities and Priorities for EPA, which prepared an interim report and the 1997 report Building a Foundation for Sound Environmental Decisions. This final report expands on issues discussed in the previous reports and addresses related questions.
The members of both committees are experts from the academic community and other organizations chosen by the NRC for their expertise in relevant scientific and technical disciplines. Special emphasis was placed on selecting committee members with research management experience and knowledge of the research and other scientific activities of EPA and other agencies. The chairman and two other members of the Committee on Research Opportunities and Priorities for EPA were also members of the Committee on Research and Peer Review in EPA.
In developing this report, the Committee on Research and Peer Review in EPA drew on the expertise and experience of its members and considered more than 300 relevant documents obtained from EPA and other sources. The committee consulted with more than 200 scientists, engineers, managers, and other persons within and outside EPA to obtain relevant information and insights on research-program struc-
ture, planning, funding, and management; organizational matters; and scientific career development, performance evaluation, recruitment, and morale issues. The committee held seven 2-day plenary meetings, six at facilities of the National Academies and one at the EPA laboratory facilities in Research Triangle Park, NC. In addition, smaller teams of committee members and staff made site visits to 12 EPA laboratory facilities across the country, in addition to all EPA regulatory offices and 5 of EPA's 10 regional offices. At these locations, committee members and staff interviewed a cross section of EPA personnel, including senior officials, middle managers, staff scientists and engineers, and support staff. During the course of the study, the committee also interviewed officials knowledgeable about EPA from Congress, the General Accounting Office (GAO), NSF, NIH, the Office of Management and Budget (OMB), and the Office of Science and Technology Policy (OSTP). In addition, most members of our committee have previously served on one or more groups that independently evaluated the research programs and scientific practices of EPA and other federal agencies under the auspices of the NRC, the Carnegie Commission, EPA's Scientific Advisory Board (SAB), the Office of Research and Development (ORD) Board of Scientific Counselors (BOSC), or other organizations. Such previous evaluations are cited and reviewed throughout our report.
On the basis of those documents, interviews, site visits, and previous experience with scientific practices in EPA, the committee recommends the following measures to strengthen the scientific performance of ORD and the agency overall.
SCIENTIFIC LEADERSHIP AND TALENT
Establish a new position at EPA: Deputy administrator for science and technology.
In the 30 years since EPA was created, the agency's scientific practices and performance have been criticized many times in reports from the NRC, EPA's SAB, the General Accounting Office, and many other organizations; in congressional oversight and judicial proceedings; and
in countless criticisms and lawsuits from stakeholders with interests in particular EPA regulatory decisions. In one such report, Safeguarding the Future: Credible Science, Credible Decisions, a panel of academicians, including two members of our committee, concluded, “Currently, EPA science is of uneven quality, and the Agency's policies and regulations are frequently perceived as lacking a strong scientific foundation.” While acknowledging that EPA had a number of knowledgeable scientists on its staff, the panel reported that the science base at EPA was not perceived to be strong by the university community, and that many EPA scientists at all levels throughout the agency believed that EPA did not use their scientific knowledge and resources effectively. The panel further observed, “A perception exists that regulations based on unsound science have led to unneeded economic and social burdens, and that unsound science has sometimes led to decisions that expose people and ecosystems to avoidable risks.” The panel commented that EPA had not always ensured that contrasting, reputable scientific views were well-explored and well-documented from the beginning to the end of the regulatory process. It pointed out that the agency was often perceived to have a conflict of interest because it needed science to support its regulatory activities, and it described a widely held perception by people both outside and inside the agency, that EPA science was “adjusted” by EPA scientists or decision-makers, consciously or unconsciously, to fit policy.
As discussed in many places throughout this report, EPA has made significant improvements in some of its scientific practices since that panel issued its report in 1992. However, the committee concludes that there is a continuing basis for many of the scientific concerns raised in that panel's report and others, such as the 1999 Resources for the Future report Science at EPA: Information in the Regulatory Process. We base this conclusion on the extensive experience of the members of our committee in assessing EPA's scientific practices and performance, including the matters discussed and documents cited in this report and other independent investigations of EPA science in which members of our committee have participated.
Throughout EPA's history, no official below the level of the administrator has had overall responsibility or authority for the scientific and technical foundations of agency decisions, and administrators of EPA have typically been trained in law, not science. The agency's most se-
nior science official has traditionally been the assistant administrator for research and development, but that official has never had agency-wide responsibility or authority for overseeing the scientific and technical basis for regulatory and policy decision-making, and EPA's regulatory offices are not required to follow scientific advice from ORD. In the committee's unanimous judgment, the lack of a top science official is a formula for weak scientific performance in the agency and poor scientific credibility outside the agency. In our 1995 interim report, this committee recommended “that the assistant administrator for research and development be designated as EPA's chief scientific and technical officer, responsible not only for ORD, but also for coordinating and overseeing agency-wide scientific policy, peer review, and quality assurance, as well as EPA's outreach to the broader domestic and international scientific community for scientific knowledge relevant to the agency's mission.” Shortly thereafter, in partial response to that recommendation, the deputy administrator of EPA asked the head of ORD to coordinate the agency's scientific-planning and peer-review activities.
Although the 1995 designation appears to have been a small step in the right direction, our committee judges it to be insufficient. First, the head of ORD was not given real authority for agency-wide scientific policy. Second, although the agency subsequently achieved some commendable progress through its interoffice Science Policy Council and ORD-led efforts to begin developing an agency-wide inventory of scientific activities and a “Strategic Framework for EPA Science, ” all those efforts, relying on consensus and voluntary cooperation of the agency's regulatory and regional offices in the absence of central science-policy authority, have had slow and limited success. The heads of EPA's regulatory and regional offices are of equal rank to the head of ORD and are generally not required to follow ORD' s guidance regarding scientific activities or science policy. Third, the ability of the head of ORD to coordinate agency-wide peer-review and quality-assurance practices was diminished in 1999 with the reassignment of some peer-review functions from ORD to the agency's newly created Office of Environmental Information.
Furthermore, based on our observations of these developments in the 5 years since our interim report, the committee has become convinced that our 1995 recommendation to designate the head of ORD as
EPA's chief scientific and technical officer also was insufficient. The committee now concludes that it underestimated in 1995 the level of authority needed to achieve the necessary degree of cooperation and coordination of scientific activities and policy in the regulatory and regional offices. In addition, the committee has become more aware of the enormous amount of scientific activity occurring in EPA's regulatory and regional offices, and it concludes that no single individual could reasonably be expected to direct a world-class research program in ORD while also trying to improve scientific practices and performance throughout the rest of the agency. These jobs are inherently different. Moreover, assigning agency-wide scientific authority to the assistant administrator for ORD might produce a conflict of responsibilities, because many decisions about science in the regulatory programs could affect ORD's budget or favor ORD 's research over research done elsewhere.
EPA needs an appropriately qualified science official at a sufficiently high level to carry both the authority and the responsibility for agency-wide scientific performance. No official below the level of deputy administrator could perform that role, because interrelated scientific and technical activities are conducted throughout the agency. The requisite operating authority with accountability for agency-wide scientific performance cannot be established by assigning the scientific gate-keeper function to any assistant administrator, regardless of the qualifications or abilities of the individual holding that position. It is unrealistic to expect that an official at the level of an assistant administrator (i.e., an official in charge of one office of EPA) could effectively coordinate and oversee the scientific and technical programs and work products of other EPA offices and regions. That includes the assistant administrator in charge of ORD.
EPA needs a top science official with the authority and responsibility to coordinate and oversee scientific activities throughout the agency. This official should obtain and use the best possible science in support of the agency's mission and identify the scientific uncertainties and conflicting evidence relevant to the agency's regulatory and policy decisions. The importance of science in EPA decision-making should be no less than that afforded to legal considerations. Just as the advice of the agency's general counsel is relied upon by the administrator to determine whether a proposed action is “legal,” an appropriately qual-
ified and adequately empowered science official is needed to attest to the administrator and the nation that the proposed action is “scientific” – that it is consistent, or at least not inconsistent, with available scientific knowledge – and that the agency has done a proper job of ascertaining and applying that knowledge and recognizing and characterizing the relevant uncertainties. Achieving these goals will require a level of accountability for EPA's scientific performance that cannot reasonably be expected from an administrator who is not trained in science, a staff advisor to the administrator without management authority, or an assistant administrator for research and development who has no authority over the use of scientific information by other offices of the agency.
The creation of a new position of deputy administrator for science and technology will require authorization from Congress, appointment by the President, and confirmation by the Senate. Such an action would send a strong message that Congress and the administration are committed to strengthening science at EPA. The current position of deputy administrator could perhaps become deputy administrator for policy and management.
The new deputy administrator for science and technology would be responsible for identifying and defining the important scientific issues facing EPA, including those embedded in major policy or regulatory proposals; developing and overseeing an integrated agency-wide strategy for acquiring, disseminating, and applying scientific information; coordinating and overseeing scientific quality-assurance and peer-review practices throughout the agency; developing processes to ensure that appropriate scientific information is used in decision-making throughout the agency, and ensuring that the scientific and technical information underlying each EPA regulatory decision is valid, appropriately characterized in terms of scientific uncertainty and cross-media issues, and appropriately applied.
The deputy administrator for science and technology would be the administrator's principal science advisor and would have managerial authority to coordinate and oversee the agency's ORD, the newly created Office of Environmental Information, the SAB, the Science Policy Council, and the scientific and technical activities of the agency 's regulatory program and regional offices. The individual appointed to this position would need to have an outstanding background, including
research accomplishments, scientific reputation, and experience in public forums.
Convert the position of assistant administrator for research and development to a statutory term appointment of 6 years.
Under the present political-appointment model, the leadership of ORD changes at least as often as the administration changes. Historically, the typical tenure of ORD assistant administrators has been only about 2 or 3 years. Frequent changes in the leadership of ORD have been disruptive and have had devastating effects on the continuity of programs, and sometimes on the morale of ORD scientists and staff. Over the years, the assistant administrator for ORD has typically been one of the last senior EPA officers appointed in a new administration, and although ORD has had some very capable assistant administrators, there have been a few cases in which little weight seems to have been given to the candidate's scientific or managerial qualifications.
The assistant administrator for ORD should have an advanced degree in an appropriate scientific or technological discipline, a substantial record of scholarly achievement, and administrative experience that includes successful management of a substantial research program. The position should be defined to make it attractive to an eminent scientist or engineer who is willing to remain in the position for a sufficiently long period of time to bring stability to the direction and leadership of ORD. A statutory term appointment would make the position more like those of the leaders of NIH or NSF. Congressional action would be required to convert the position to the recommended 6-year term.
Seek ways to give research managers in ORD a high degree of flexibility and commensurate accountability. Empower and charge them to make research program decisions at the lowest appropriate management level consistent with EPA policy and ORD's strategic goals and budget priorities.
Excellence at EPA, or elsewhere, requires effective leadership at many levels, not just at the top. In the selection and advancement of managers at all levels in ORD laboratories and centers, competence in management and supervision should be emphasized, but scientific and
technical qualifications and accomplishments should also be given strong consideration. Research managers should understand the work and merit the respect of their research staff while having the ability to select, inspire, lead, and elicit the best efforts from other scientists and engineers. They should be strong advocates and defenders of the continuity and core capabilities required for the conduct of a good research program. EPA's ability to recruit, develop, and retain such leaders depends on many factors, including the agency 's commitment to reducing bureaucratic impediments and finding ways to increase the latitude afforded to research managers to fulfill their responsibilities.
Enhance research leadership and ORD's scientific stature by creating the equivalent of endowed academic research chairs in ORD's national laboratories.
In research, perhaps even more than in other fields, pre-eminent leadership sets the standard and tone for the rest of the work force. A single world-class investigator can generate ideas and enthusiasm to elevate a research program dramatically. In research, leadership is not synonymous with management or administration.
The 1992 Safeguarding the Future report recommended that ORD recruit and support on a long-term basis four to six senior research scientists and engineers with world-class reputations in areas vital to EPA's long-term strategy and direction. The panel envisioned that these eminent scientists and engineers would serve as examples and mentors for all scientists in EPA and would bring access to networks of world-class scientists to benefit the agency. The panel recommended that EPA's SAB be asked to form a search committee.
ORD responded to the 1992 recommendation by obtaining and using special authority to recruit and promote research scientists and engineers to senior, nonmanagerial federal career positions – so-called ST (scientific and technical) positions at the Senior Executive Service level. Using merit-review panels that included outside experts to evaluate candidates, ORD has recruited or promoted eight such individuals in its laboratories. Our committee fully supports ORD's use of the ST program and urges that it be continued. However, recognizing today 's intense job-market competition with industry and academic institutions for top research talent, the committee concludes that even greater
measures are warranted and practicable to attract and retain outstanding research leaders in the ORD laboratories.
To establish the recommended “research chair” positions, ORD could explore the possibility of awarding prestigious 5-year renewable grants or distinguished fellowships to distinguished academic scientists to work at ORD laboratories on site, full time. The committee suggests that it is not necessary for all such researchers to be regular federal employees. Alternatively, ORD could seek authority to create and fill positions similar to NIH Title 42 senior research appointments, perhaps in connection with internal grants to ensure sustained research support for these distinguished investigators. The SAB or the BOSC should be asked to assist ORD in developing this program and selecting the candidates.
Our committee envisions the recruitment of at least one additional distinguished investigator per ORD national laboratory in areas of research that are rapidly advancing and important to ORD's future. These individuals should be given maximum intellectual freedom to pursue productive lines of research that are consistent with laboratory missions and ORD priorities. The distinguished investigators should be expected to serve as mentors and role models for other ORD research scientists but should have no managerial responsibility beyond their own on-site research teams.
Continue to place high priority on the ORD graduate fellowship and postdoctoral programs.
To achieve scientific and technical excellence, EPA must attract, retain, and properly support a first-rate, dedicated professional staff. Our committee is aware of many outstanding scientists and engineers in ORD and other parts of the agency, but the ORD work force is aging. More than 47% of ORD's employees are at least 50 years old, and more than 550 ORD employees will be eligible to retire within the next 5 years. Periodic EPA hiring freezes, combined with intense scientific and technical job-market competition from the private sector and academic institutions, have made it extremely difficult for ORD to recruit the new talent needed to sustain and enhance its research work force or even to retain some of the best of those on board.
Since 1995, ORD has established excellent programs of graduate
fellowships and postdoctoral appointments. These programs have brought a stream of fresh scientific and technical talent into EPA's research program and are helping to train future research leaders in environmental science, engineering, and other disciplines. The committee urges EPA to continue to place high priority on these programs.
RESEARCH CONTINUITY AND BALANCE
Continue steadily on the major courses set in the 1995 reorganization of ORD.
EPA's ORD conducts research in its in-house laboratories, funds extramural research at academic institutions and other organizations, performs a variety of activities in the development and application of risk-assessment methods and regulatory criteria, and provides technical services in support of the mission of the agency and its regulatory and regional offices. In fiscal year 1999, ORD had 1,976 staff members at 12 geographically dispersed laboratory facilities, three field stations, three assessment offices, and the headquarters office in Washington, DC. ORD also had an extramural research budget of about $315 million – more than half of its $559 million total budget in fiscal year 1999 – for grants, cooperative and interagency agreements, contracts, and fellowships. Over the past 20 years, the resources of ORD generally comprised about 7% of the agency's total budget.
Frequent changes in goals, priorities, practices, structure, or funding can disrupt any organization, but they are especially damaging to a research organization, which has special requirements for continuity in the development and maintenance of scientific and engineering talent, experience, and infrastructure to be productive. Maintaining the requisite degree of stability in ORD has been a continuing challenge because of many expansions and other changes in EPA's legislative mandates and priorities, directives from Congress and different administrations, pressures from regulated parties and other interest groups, lawsuits and court decisions, inadequate budgets to meet competing demands, and changes in the leadership of ORD.
ORD has changed its research goals, priorities, and practices often and abruptly in the past. Greater stability, continuity, and predictabil-
ity are among the most important needs in the ORD program. The limited financial and human resources of ORD should be managed with a steady hand and a clear and persistent vision of how to maximize the gains in scientific understanding resulting from its work.
Our committee's interim report endorsed the general scope and direction of the major reorganization of ORD in 1995, which adopted as a principal organizing concept the reduction of uncertainty in risk assessment and risk management, initiated a new strategic research planning process, consolidated ORD's laboratories and centers, and expanded and strengthened the research grants, centers, fellowships, and peer-review programs. Our interim report called those measures the most important changes in the history of EPA's research program. In the long run, the courses ORD set in 1995 should have a stabilizing effect on the research program, and our committee continues to support them. In some respects those changes are still a work in progress and need more time to mature.
Continue and expand ORD's new multiyear planning approaches for both problem-driven and core research areas.
In the past few years, ORD has begun to explore a multiyear approach for research planning to foster continuity and strategic integration of some research efforts. ORD has developed multiyear plans for research on particulate matter, endocrine disruptors, drinking water, environmental monitoring, global climate change, and pollution prevention. Multiyear plans for additional areas of research are under development. The plans are developed by research teams from ORD laboratories and centers and are peer reviewed. Our committee expects that ORD's recent efforts in multiyear planning will contribute to research program continuity and the achievement of strategic goals, and the committee commends ORD for these initiatives.
Maintain approximately an even balance between problem-driven research and core research.
ORD is challenged to lead the agency by means of research while continuing to assist its client regulators, some of whom have limited understanding or appreciation of science but a strong say in ORD's budget and priorities. Activities in support of regulatory programs
often have a narrow focus and often compete to pre-empt long-term research programs. They tend to be disruptive and to consume the resources of a research organization disproportionately. Regulatory strategies, which are typically prescriptive and specific, tend to freeze concepts and methods in time, and the more closely that ORD is tied to the regulatory programs, the greater the risk that ORD will to some extent be working on outdated problems or with outdated approaches.
On the other hand, ORD's involvement in problem-driven research and technical assistance to regulatory and regional offices has important benefits for ORD 's own core research as well as for the agency's operating programs. Perhaps the greatest dividends are the resulting improvements in the scientific aspects of the agency's regulatory actions and the maintenance of an in-house scientific core group experienced in dealing with environmental risks and programs. An experienced scientific core group can be of great value in meeting emergency requirements for technical expertise. ORD's technical assistance provides the regulatory offices with competent scientific help and leadership, and it enables ORD's research scientists to keep abreast of regulatory and policy developments elsewhere in EPA.
In the 1997 NRC report Building a Foundation for Sound Environmental Decisions, our companion committee in this study concluded that ORD should maintain a balance between the problem-driven research and technical support for the agency's regulatory programs and the core research to better understand and anticipate environmental risks. Our committee agrees. Those two functions are not unrelated or incompatible; they are mutually reinforcing. Core research is the indispensable wellspring that prepares and enables ORD to provide better problem-specific research and technical assistance to the agency and the nation.
RESEARCH PARTNERSHIPS AND OUTREACH
Develop and implement a pro-active, structured, and visible strategy for stimulating, acquiring, and applying the results of research conducted or sponsored by other federal and state agencies, universities, and industry in this country and abroad.
From time to time the question arises whether EPA should have its own research program or rely on research results developed elsewhere.
Advocates of having the research conducted elsewhere often cite past criticisms of the agency's research program and point to excellent research programs of other agencies, such as NIH, NSF, and the Department of Energy, that collectively, and in some cases individually, dwarf that of EPA. And, of course, the academic community and the private sector conduct much of the research relevant to EPA's mission.
In the 1992 report Safeguarding the Future: Credible Science, Credible Decisions, a panel of four senior academicians, including two members of our committee, concluded that EPA needs its own strong science base to provide the background required for effective environmental protection programs. Similarly, Building a Foundation for Sound Environmental Decisions, the 1997 report of the NRC's Committee on Research Opportunities and Priorities for EPA – our companion committee in this study – concluded that EPA needs a strong in-house research program.
Our committee agrees that a vigorous research program should be maintained in EPA. Moving the research program out of the agency would most likely weaken, not strengthen, the scientific foundation of EPA' s decisions and actions. Although some abstract concept of scientific “quality” might be improved by reducing some kinds of ORD technical work that are unlikely to advance research frontiers, such work is often critically necessary to EPA's pursuit of its mission and statutory responsibilities. Overall, eliminating ORD or moving its functions out of EPA would be destructive, and the level of damage would increase with passing time as EPA became increasingly unable to pursue, apply, or even understand new research knowledge. An EPA devoid of a research program would not be likely to attract substantial scientific talent, and an EPA without scientific talent would be ineffective and potentially harmful to the nation.
However, even with a much larger budget, ORD could never meet all the vast and constantly expanding needs of EPA and the nation for scientific and technical knowledge to guide environmental protection efforts. ORD has had a first-rate research program in some important areas, such as aquatic toxicology and human inhalation toxicology, but it is not possible for ORD to be a leader across the full range of scientific knowledge required by EPA. The agency should recognize the limits of its research capabilities and develop an effective, structured, and visible strategy to acquire, use, and support research in areas where ORD cannot be pre-eminent. The strategy should include a pro-
gram of increased sabbatical assignments for ORD researchers to gain experience in other scientific organizations, and more visiting appointments of scientists and engineers from universities, other government agencies, and private organizations to work in ORD laboratories and centers.
Reassess the numbers, qualifications, and skill mix of the staff of ORD's National Center for Environmental Research to ensure they are consistent with the needs of the current program of research grants, centers, and fellowships.
ORD's expanded and strengthened competitive research grants, centers, and fellowships programs have greatly increased the number and activities of talented academic researchers across the nation who are engaged in research relevant to EPA's mission. ORD deserves to be commended for its excellent performance in developing and implementing these programs, as well as for the partnerships it has built with other agencies and funding organizations in joint grant solicitations. However, as discussed in Chapter 2, questions have been raised about the adequacy of the numbers, qualifications, and skill mix of the staff of ORD's National Center for Environmental Research to administer the grants, centers, and fellowships programs. Since 1995, ORD has increased the research-grant funds administered by the center by about 400% without substantially increasing the staff who administer this program. The increased grant-program activity has placed high demands on the staff who are responsible for the review process and the monitoring and dissemination of grantees' research products. And the staff who administer EPA's Science to Achieve Results (STAR) program grants must address mission-relevance and technology-transfer aspects of grantee's research that NIH and NSF grants administrators are not required to address.
Develop additional mechanisms to promote and facilitate research interactions among STAR grantees and ORD research staff.
The committee encourages strengthening the interactions between STAR grantees and research scientists and engineers in the ORD laboratories. At present, there are insufficient mechanisms for facilitating such interactions effectively. One possible mechanism is to ask grant
applicants to identify in their proposals how their research might be enhanced by interactions with EPA scientists and how their research might complement or supplement ongoing or planned research in the ORD laboratories. Reviewers of the proposals, as well as ORD scientists and the SAB, could also be asked for suggestions.
Increase EPA's efforts to disseminate actively ORD's research products, to explain their significance, and to assist others inside and outside the agency in applying them.
The 1992 Safeguarding the Future report concluded that the academic community, Congress, other federal agencies, industry, the public, and even many within EPA are generally unfamiliar with the work of EPA scientists. The 1992 panel emphasized that many officials involved in funding EPA science were uncertain about what science products EPA had produced, and whether the quality and quantity of its products were commensurate with the dollars expended. It noted that EPA's policy and regulatory work receives a great deal of public attention, but the agency's science typically receives a similar degree of attention only when the scientific basis for a decision is questioned. The panel concluded that EPA should strive to make more widely known the short-term and long-term scientific goals and achievements of its research laboratories, contractors, and grantees. It urged the agency to develop and implement a coherent communications, outreach, and education plan to publicize the activities and accomplishments of EPA scientists.
Even within the agency, many regulatory and regional program officials throughout EPA's history have been largely unaware and even dubious of any important benefits from ORD's research program, and consequently they have not been supportive of ORD's budget. Recently, the GAO reported that one of EPA's regulatory program offices so acutely needed information on ORD's work – information relevant to its program and well beyond the progress reports that ORD was providing – that the regulatory office found it necessary to pay for the development of a system to track ORD's projects. That kind of situation is not healthful for ORD or the agency. ORD's ongoing efforts to disseminate its research products and inform others about them have, with some exceptions, been meager and unimaginative.
Publication of original research articles is critically important, but it
is not sufficient. EPA should publish more individual research-topicarea summaries and comprehensive annual summaries of the results of in-house and extramural research and technical-support activities. The summaries should be planned and tailored for specific audiences and should emphasize the potential applications of ORD's work by other EPA offices, state agencies, industry, and others.
In addition, our committee concurs with the 1998 recommendation of ORD's BOSC that the National Center for Environmental Assessment should revise its mission to focus more on being an advisor, catalyst, and resource for the risk assessments performed by the rest of the agency, rather than trying itself to do individual risk assessments with its own limited resources. The center should focus on being a research organization dedicated to advancing the state of practice in risk assessment. It should reduce its role as a performer of individual risk assessments that could be done by EPA's regulatory offices.
RESEARCH ACCOUNTABILITY
Improve the documentation and transparency of the decision-making processes used by ORD for setting research and technical-assistance priorities, making intramural and extramural assignments, and allocating funds.
In commenting on ORD's fiscal year 2000 budget, the SAB indicated that the lack of transparency in the decision-making process used by ORD to set research priorities made it difficult to evaluate the adequacy of the proposed budget. Although the ORD strategic plan discusses the general processes and criteria by which decisions are made on research priorities and funding allocations, the plan describes the processes and criteria only in very broad terms.
During the committee's site visits and interviews, the staff of some EPA regulatory program offices expressed the belief that they have little influence on ORD 's research priorities through the Research Coordination Council or any other mechanism. They felt that they needed a stronger voice in the setting of ORD's priorities, and that ORD should be held more accountable to the agency's other offices for performing agreed-upon tasks.
Our committee concludes that ORD should continue to be respon-
sive to the agency's regulatory offices for the problem-driven and technical-assistance components of its program, and the agency's regulatory offices should continue to have a strong voice in decisions about the ORD plans and budget elements devoted to those components. For the core-research portion of its program, however, ORD should have greater freedom to set the agenda, without the need for specific concurrence of regulatory program offices that are focused on statutory requirements and regulatory goals. In the agency planning process, ORD should continue to consider the views and needs of the program offices in developing both components of its program, but it should maintain an adequate degree of independence in planning a core-research program that will successfully perform the leadership and anticipatory-research role that such a program can bring to the agency.
The process by which ORD decides whether a project or task is to be performed by in-house staff or through one or more extramural mechanisms is also of crucial importance to the quality of the work and the cost-effective management of resources. This decision-making process has not been sufficiently open or visible for persons outside ORD or EPA to reconstruct or assess how the decisions were made.
Expand upon the recently initiated agency-wide science inventory by conducting, documenting, and publishing a more comprehensive and detailed inventory of all scientific activities that are being conducted by offices throughout EPA.
ORD's research should not be the only scientific studies held accountable in EPA. A great deal of research-like activity, including many activities in scientific and technical data-gathering, analysis, and interpretation, are being conducted or funded by EPA offices outside ORD. Much of this work is not labeled “research.” The other offices of EPA do not have the kind of authorization that ORD has to conduct research per se, and full disclosure might risk the loss of control of such activities by the regulatory offices. Historically, many of the scientific studies and analysis performed or funded outside ORD were not fully coordinated across the agency or included in the ORD's research-planning and peer-review programs. Our committee is by no means opposed to scientific studies and analyses being conducted in parts of the agency outside ORD, but such activities require transpar-
ency, quality assurance, and accountability, just as ORD's program does.
ORD, with the help of others throughout the agency, recently initiated an inventory of science projects and programs across EPA. Our committee commends the agency for this important step. We recommend that the administrator direct the new deputy administrator for science and technology to expand on the preliminary-inventory by documenting a more comprehensive and detailed inventory of scientific activities conducted by all EPA offices. The inventory should include information well beyond the current scope – information such as goals and objectives of each project, milestones, schedules, principal investigators and project managers, and allocations of staff and financial resources. The results of the inventory should be used to ensure that such activities are properly coordinated through the agency-wide science-planning and budgeting process and are appropriately peer reviewed. The SAB should be engaged in assisting and overseeing this effort.
SCIENTIFIC PEER REVIEW
Change the agency's peer-review policy to more strictly separate the management of the development of a work product from the management of the peer review of that work product, thereby ensuring greater independence of peer reviews from the control of program managers, or the potential appearance of control by program managers, throughout the agency.
The committee congratulates EPA and its Science Policy Council for the excellent progress it has made in strengthening and expanding the agency's peer-review practices. The agency's 1998 peer-review handbook, discussed in Chapter 3, is a valuable resource and guidance document.
EPA's SAB has expressed concern about potential conflict of interest on the part of peer-review leaders – individuals assigned to manage reviews of agency work products – because current agency policy allows the same individual to be a project manager for the development of a particular work product and the peer-review leader for the same
work product. The SAB noted that such a manager might have a special interest in the outcome of the review and might therefore be unable to ensure the essential degree of independence. The SAB contrasted EPA's present policy with the agency's data-quality-assurance practices, whereby a staff officer is empowered to stop activity if there is a quality-assurance problem.
EPA has made excellent progress in expanding and strengthening its peer-review practices, but the agency should find a way to ensure a greater degree of independence in the management of its peer reviews. The committee acknowledges that the agency should have adequate flexibility to accommodate statutory and court deadlines and resource limitations. Nevertheless, independence is essential to the proper and credible functioning of the peer-review process, and EPA's current policies fail to ensure adequate independence. Our committee shares the SAB 's concern about the potential for conflicts of interest of EPA peer-review leaders and decision-makers. Despite good intentions, and even if the current policy works well much of the time, some of these individuals, under pressure to meet a deadline or implement a regulatory policy, might be tempted to compromise the integrity of the peer-review process for some work products by making convenient or improper decisions on the form of peer review, the selection of reviewers, the specification of charges to the reviewers, or the responses to reviewers' comments.
Our committee believes that the decision-maker and peer-review leader for a work product should never be the same person, and that wherever practicable, the peer-review leader should not report to the same organizational unit as the decision-maker. The committee recognizes that statutory and judicial deadlines can make it necessary that a program-office decision-maker retains the authority to proceed with an action on a provisional basis in the face of concerns or objections from a peer-review leader, the final decision being made by the EPA administrator. However, the independent decisions and any objections of a peer-review leader should be preserved and made a part of the agency decision package and public record for a work product. If such an independent assessment produces criticism of the adequacy or outcome of a peer review, EPA's policy should be to ensure that the criticism is clearly noted and that the reasons for proceeding despite the criticism are clearly explained.
The committee also recommends that the Science Policy Council's reviews of the agency's peer-review handbook and of experiences with its implementation include an explicit focus on promoting appropriate forms and levels of review for different types of work products and on reducing unnecessarily complex or inefficient requirements. The Science Policy Council should not necessarily wait the 5-year interval specified in the peer-review handbook; it should make changes as needed. The agency cannot afford to allow unnecessary or inefficient requirements to continue so long. The Science Policy Council's review should be ongoing. We also recommend that the Science Policy Council review a true random sample of peer-reviewed work products, examining the decisions made in structuring the review, the responses to review, and the cost, quality, timeliness, and impact of the review.
Finally, the committee wishes to emphasize that peer review must become accepted throughout EPA as a part of the agency's culture – a tool for improving quality – not merely a bureaucratic requirement. Measures such as periodic dissemination of the impacts and benefits of completed reviews might help to foster this cultural change in the agency.
