Research Management at EPA
THE ROLE OF ORD
EPA's Office of Research and Development (ORD) conducts research in its in-house laboratories, funds extramural research at academic institutions and other organizations, performs a variety of activities in the development and application of risk-assessment methods and regulatory criteria, and provides technical services in support of the agency 's mission and its regulatory and regional offices. In fiscal year 1999, ORD had 1,976 staff members at 12 geographically dispersed laboratory facilities, three field stations, three assessment offices, and the headquarters office in Washington, DC. ORD also had an extramural research budget of about $315 million – more than half of its $559 million total budget in fiscal year 1999 – for grants, cooperative and interagency agreements, contracts, and fellowships. Over the past 20 years, the resources of ORD have generally comprised about 7% of the agency's total budget (Figure 2-1).
A great deal of scientific activity is conducted or funded by EPA offices outside ORD. This work is typically labeled as something other than “research.” The other offices of EPA do not have the kind of authorization that ORD has to conduct research per se, and full disclosure might risk the loss of control of some of these activities by the regulatory offices. Perhaps the existence of a substantial amount of research-
like activity outside ORD is an indication that the agency's regulatory and regional offices believe that ORD is not fully meeting their scientific needs. In any case, it is problematic, because many scientific activities performed or funded outside ORD historically have not been well-coordinated across the agency and have not been included in the ORD research planning and peer-review programs.
In response to a recommendation from our companion committee in this NRC study (NRC 1997), ORD and the EPA Science Policy Council, with the assistance of other EPA offices, began to develop in 1998 an agency-wide “inventory of science activities.” The inventory is intended to become an “evergreen” interactive planning tool to integrate a variety of scientific efforts within a common strategy. Organized according to the agency's strategic goals under the Government Performance and Results Act, the inventory holds promise as a tool to help improve scientific collaboration across agency offices and to identify scientific gaps and opportunities for consolidation. The first draft of the inventory is sketchy, containing only general information about the various activities and no information about resources or milestones. It is not yet well-documented or published. Agency work groups are working under the direction of the Science Policy Council to develop recommendations for improvement of the scope, purposes, and form of the inventory, as well as standards for collecting the relevant information.
From time to time, the question arises whether EPA should have its own research program or rely on research results developed elsewhere. Advocates of having the research conducted elsewhere often cite past criticisms of the agency's research program and point to excellent research programs of other agencies and organizations, which collectively, and in some cases individually, dwarf that of EPA. Many other agencies and organizations certainly contribute much of the scientific and technical information that EPA requires. At the federal level, they include the Departments of Agriculture, Commerce, Defense, Energy, and Interior; the NIH; National Aeronautics and Space Administration; and NSF. NSF devotes about $600 million a year to environmental research – about the same as ORD's total budget – and recently, the National Science Board recommended a major expansion of NSF's environmental research, planning, education, and scientific assessment, with a funding target of an additional $1 billion over 5 years (NSF
1999). The Department of Energy spends about $500 million a year on environmental quality research. Many public-health, environmental, and natural-resource agencies at the state and local levels also support scientific activities. Internationally, the World Health Organization, the International Agency for Research on Cancer, and many nations have strong scientific review and risk assessment programs. And, of course, the academic community and the private sector conduct much of the research relevant to EPA's mission.
In Safeguarding the Future: Credible Science, Credible Decisions (EPA 1992), a panel of four senior academicians, including two members of our committee, was asked by EPA's then-administrator William Reilly for advice on how the agency could best meet the goal of using sound science for its decision-making. The panel concluded that “EPA needs its own strong science base to provide the background required for effective environmental protection programs.” The panel gave several reasons for this conclusion:
EPA decisions frequently are controversial and affect broad sectors of society and the economy. Controversial decisions demand a strong science base when decisions are made.
EPA cannot rely only on other government agencies to develop the scientific information it needs for decision-making.
The existence of its own science base allows EPA to tie science to long-term regulatory objectives and other environmental protection strategies.
Interaction between scientists and policy-makers is essential for sound decision-making.
Some scientific activities, such as controlled human exposure studies, require special facilities that are beyond the capability of most university-based research programs.
The NRC's Committee on Research Opportunities and Priorities for EPA – our companion committee in this study – also concluded that EPA needs a strong in-house research program (NRC 1997).
Based on the extensive experience of members of our committee with the research program and applications of science in EPA (see Chapter 1), our committee agrees that a vigorous research program should be maintained in EPA. Moving the research program out of the agency would most likely weaken, not strengthen, the scientific foun
dation of EPA's decisions and actions. Although some abstract concept of scientific “quality” might be improved by reducing some kinds of ORD technical work that are unlikely to advance research frontiers, such work is often critically necessary to EPA's pursuit of its mission and statutory responsibilities. Overall, eliminating ORD or moving its functions out of EPA would be destructive, and the level of damage would increase with passing time as EPA became increasingly unable to pursue, apply, or even understand new research knowledge. An EPA devoid of a research program would not be likely to attract substantial scientific talent, and an EPA without the kind of scientific talent that research attracts could be ineffective and potentially harmful to the nation.
However, ORD should recognize its limits and the need to depend on partnerships with other research organizations. ORD's resources are important, but only a small part of the total resources devoted to research relevant to environmental protection in the United States and abroad. Even with a much larger budget, ORD could never meet all of EPA's vast and constantly changing needs for scientific knowledge. ORD has had a first-rate research program in some important areas, such as aquatic toxicology and human inhalation toxicology, but it is not possible for ORD to conduct in-house research across the full range of scientific knowledge required by EPA. If ORD were to try to meet all of EPA's needs for scientific knowledge, it would dilute its research efforts on the most important problems and detract from critical core research needs. Careful choices need to be made in using ORD's important but limited resources to maximize the value it adds to the total reservoir of knowledge that is needed and disseminated within EPA, the scientific community, and other organizations. As recommended in Future Risk: Research Strategies for the 1990s (EPASAB 1988) and Building a Foundation for Sound Environmental Decisions (NRC 1997), EPA should focus on a few core research areas that it can handle well and rely on partnerships and outreach for other scientific needs.
The agency should strive to stimulate and coordinate research at national and international institutions that is responsive to its needs. EPA should be a leader in some areas of research, involved in others, and well informed in all relevant areas. To accomplish that, EPA needs a cadre of scientists responsible for maintaining a thorough awareness in specific research areas that are important to the mission of the agency. It should develop a strong, structured approach to the re-
trieval, synthesis, and application of the results of research conducted not only by ORD, but also by scientists not affiliated with or supported by EPA. This includes research conducted or sponsored by other federal and state agencies, universities, and industries, both in this country and abroad. The internet presents great opportunities for enhancing interagency and international research coordination.
Another important question asked from time to time pertains to the overall balance in EPA's research program. In a report accompanying EPA's appropriations for fiscal year 1993, the congressional conference committee requested EPA to review the balance between its basic and applied research, stating,
The Committee believes that, for a number of reasons, EPA has failed to sufficiently address the issue of basic research. Due to the large number of regulatory and statutory mandates, the Agency has focused on short-term applied research. Basic research can be used to identify and assess environmental problems which pose the greatest risk to human health and the environment.
ORD's strategic plan (EPA 1996a, 1997a) states, “While all of EPA uses science for policy and regulatory decision-making, and various EPA offices contribute to the scientific underpinnings of the Agency's decisions, the responsibility for leadership in science at EPA and for the bulk of EPA's research and development work resides in the Office of Research and Development.” Yet, EPA's statutory mandates and regulatory programs have historically been problem-driven, and consequently so has most of ORD's program. Strong demands are placed on ORD to meet the needs of the agency's regulatory programs.
The difficulties of serving multiple regulatory-office clients while trying to sustain a core-research program have posed challenges to ORD throughout its history. ORD has continually been torn between competing demands: on the one hand, to lead, and on the other hand, to serve the rest of EPA.
EPA acknowledged that difficulty in the preface to Fundamental and Applied Research at the Environmental Protection Agency (EPA 1994a):
Indeed, the difficulty for EPA as well as for other regulatory agencies has been meeting the needs of many research clients. EPA's research program must strike a balance between providing data and technical
support for ‘front-line' regulators solving environmental problems today and building a science knowledge base necessary to manage our ecological resources wisely in the coming decades; understand how pollutants affect our health; and prevent or reduce environmental risks in the future.
In Building a Foundation for Sound Environmental Decisions (NRC 1997), our companion committee in this NRC study recommended that EPA's research program maintain a balance of roughly equal proportions between problem-driven research, targeted at understanding and solving particular, identified environmental problems and reducing the uncertainties associated with them, and core research, which aims to provide broader, more generic information to help improve understanding relevant to environmental problems for the present and future. It described problem-driven research as the kind of research and technical support activity that ORD has pursued most in the past, efforts that are largely driven by current or anticipated regulatory efforts of other EPA offices. Problem-driven research is a means to understand single problems in depth and assess remedies. Core research is largely aimed at providing knowledge for the agency to anticipate and respond to current and future environmental problems.
The 1997 report described three components of core research:
Acquiring a systematic understanding of the physical, chemical, biological, geological, social, and economic processes that underlie and drive environmental systems, and the biochemical and physiological processes in humans that are affected by environmental agents.
Developing broadly applicable research tools and methods for understanding and managing environmental problems, including better techniques for measuring physical, chemical, biological, social, and economic variables of interest; more accurate models of complex systems and their interactions; and new methods for analyzing, displaying, and using environmental information for science-based decision-making.
Designing, implementing, and maintaining appropriate environmental monitoring programs and evaluating, synthesizing, and disseminating the data and results to improve understanding of the status of and changes in environmental resources
over time and retrospectively evaluating whether environmental policies are having the desired effects.
The distinction between research in problem-driven and core areas is not always clear-cut, and the categories might often overlap. Fundamental discoveries can be made during the search for solutions to narrowly defined problems, and breakthroughs in problem-solving sometimes occur as a result of core-research efforts. Feedback between the two types of research greatly enhances the overall research endeavor. The goals of core research tend not to vary much over time, so coreresearch priorities will remain relatively constant. Problem-driven research, on the other hand, should be responsive to regulatory program needs and changing priorities, so it should be re-evaluated and refocused regularly.
Our committee concurs with the 1997 NRC report and supports the increased priority and proportion of a core-research program in ORD. The core-research program should endeavor to emphasize the evaluation of potential environmental concerns and “over-the-horizon” possibilities (EPASAB 1995), as well as new approaches to managing current problems.
Safeguards will continue to be needed to ensure that the important scientific needs of EPA's regulatory programs and regional offices are not unduly compromised. A great burden has been placed on the agency-wide strategic-planning process, discussed later in this chapter, to ensure that such compromise does not occur. ORD's technical support role should be planned and conducted with clear understanding of the goals of such support, the appropriate degrees of interaction with program office staff, and the timing and channels of such interactions.
ORD's programs should address the needs of the agency in the context of a broad, comprehensive framework (e.g., Presidential/Congressional Commission on Risk Assessment and Risk Management 1997a,b). Our committee believes that ORD's overall program should
Identify and define the risks to human health and the environment and develop scientific and technical approaches to reduce such risks.
Demonstrate the feasibility of regulatory or nonregulatory risk-reduction actions that may be taken.
Support and facilitate the development and implementation of necessary regulations aimed at reducing risk.
Both the problem-driven and core components of EPA's research serve to support EPA's fundamental mission: to identify, assess, and abate risks to public health and the environment. Viewed in that context, the strategic mission of EPA's research program, in both the short and long run, is to develop and advance the scientific and technical basis for risk identification and assessment and to guide decision-makers in making risk-management judgments and selecting overall priorities.
Many scientists within and outside the agency believe that ORD should become more of a pro-active leader for the rest of the agency, giving greater emphasis to anticipatory research that develops the knowledge to lead EPA into new strategies and levels of understanding. Regulatory officials, on the other hand, often argue that EPA's limited resources, including ORD's, are provided to support the agency's existing statutory mandates and regulatory programs, so ORD should provide the regulatory offices with more technical assistance and short-term, quick-payoff, applied work.
Our committee is convinced, as was our companion committee (NRC 1997), that the core-research role is of crucial importance to EPA and the nation. The very nature of the problems faced by EPA has been changing dramatically, and surprises have become common. EPA was created in 1970 with the limited understanding of environmental issues available at that time, including some concepts that are now largely outdated and rapidly being subsumed in new concepts such as sustainable development and industrial ecology (EPASAB 1988; NAE 1994; NAPA 1994; OSTP 1994). These concepts envision the integration of environmental science and technology throughout the entire economy. They are not simply (or in many cases even primarily) concerned with reducing existing impacts or ensuring compliance with so-called “end-of-pipe” regulations. If ORD is to participate effectively in developing and implementing new concepts and policy directions, its scope of activities should be appropriately expansive. ORD should address not only the individual pollution-related problems that have traditionally concerned EPA, but also the research on complex topics such as sustainable development and biological diversity. Research should lead the activities of EPA and not just follow past policies or
respond to currently perceived needs. EPA's research should address future problems, not just past and present problems.
Some research problems are sufficiently broad or complex that they can be addressed effectively only by a long-term focused effort. In addition, a sustained program of anticipatory research would be expected to reduce the need for reactive projects in many cases. An effective over-the-horizon research program seeks to anticipate and address future scientific needs in support of environmental protection, thereby reducing future needs for reactive efforts that are often less efficient and more expensive.
Using ORD for short-term scientific assistance to regulatory and regional offices has some undeniably important benefits. Perhaps the greatest dividends are the resulting improvements in the scientific aspects of regulations, the maintenance of an in-house scientific core group experienced in dealing with environmental risks and programs, and the knowledge of agency issues that the research scientists obtain through such experience. An experienced scientific core group can be of great importance in meeting emergency requirements for scientific expertise. ORD's technical assistance provides the regulatory offices with competent scientific support, and it enables the ORD research scientists to keep abreast of regulatory and policy developments elsewhere in EPA.
ORD should meet the continuing challenge to lead the agency through research while continuing to assist its client regulators, who have variable levels of understanding and appreciation of science but a strong say in ORD's budget and priorities. ORD's regulatory assistance activities often have a narrow focus and compete with or pre-empt long-term research programs. Such activities tend to consume the resources of a research organization disruptively as well as disproportionately. There is also some inevitable risk to ORD's scientific credibility when it provides technical assistance for regulatory strategies that might be predetermined, or are perceived to be. In addition, regulatory strategies, which are typically prescriptive and specific, tend to freeze concepts and methods in time. The scientific components of regulations also tend to be frozen in time. The more closely that ORD is tied to the regulatory programs, the greater the risk that ORD will work to some extent on outdated problems or with outdated approaches – waging the last war instead of preparing for the next one.
THE 1995 REORGANIZATION OF ORD
By the early 1990s, the structure of ORD reflected its historical origins, the traditional areas of strength of its laboratories, the technical-assistance requirements of the agency's regulatory programs, directives from Congress, the agency's research-mission ambiguities, and a culture of entrepreneurship in some parts of ORD. In 1993, EPA estimated that approximately 29% of ORD's total resources were devoted to “fundamental” research, 43% to “application-directed” research, 19% to “development,” and 9% to “technical assistance,” although it expressed uncertainty about how to define those terms (EPA 1993).
As our committee began its work, EPA was making major changes in its research program. The process leading to those changes began in 1993 with a decision by the administrator, in response to a request of Congress, to evaluate all of EPA's laboratories in relation to the agency's scientific and technical needs. The first major step in the evaluation was a study by the MITRE Corporation, working with a team of EPA officials, and with assessments by EPA's Science Advisory Board (SAB) and the National Academy of Public Administration (NAPA). The MITRE (1994) report, A Comprehensive Study of EPA Scientific and Technical Laboratories and Their Facilities and Capabilities, provided extensive documentation on the laboratories and their functions, as well as an analysis of five principal options for reorganizing them. Among its findings, the MITRE report expressed concern about the lack of clear, agreed-upon mission statements for EPA, ORD, or the laboratories. It also commented on the excessive use of contract personnel at EPA laboratories, various facility and equipment problems, and the need for improvement in quality-assurance and research-planning practices. Regarding laboratory reorganization, the MITRE report concluded that the discipline-based organization of ORD's laboratories was not optimal to support the mission-based organization of the rest of the agency. It favored a functional reorganization as previously recommended in Environmental Research and Development - Strengthening the Federal Infrastructure (Carnegie Commission on Science, Technology, and Government 1992).
EPA's SAB and the NAPA reviewed the MITRE report (NAPA 1994; EPASAB 1994). The SAB report generally concurred with MITRE in endorsing a variation of the Carnegie Commission model for labora-
tory organization, but it argued that management improvements were needed in ORD before reorganization should occur. The SAB expressed concern about the extent to which EPA laboratory scientists were being required to serve as contract managers. It also expressed concern about the possibility that decreased ORD involvement in short-term, applied research and technical assistance to the regulatory and regional offices might lead to the expansion of separate research programs in the regulatory offices, potentially damaging the overall quality and effectiveness of ORD's research role in the agency. The NAPA review also generally supported the MITRE recommendations but emphasized that ORD's mission and goals should be clarified before any laboratory reorganization took place.
In response to the MITRE study and the SAB and NAPA reviews, an agency-wide, senior-level steering committee prepared final recommendations to the administrator. Its report, Research, Development, and Technical Services at EPA: A New Beginning (EPA 1994b), recommended the following changes in ORD's research program:
Strategic Planning: Initiate a new strategic research planning process in concert with other offices of EPA.
The Role(s) of ORD: Increase long-term research from about 30% to at least 50% of ORD 's total program budget.
ORD's Laboratories: Functionally consolidate ORD's 12 laboratories, three field stations, and four assessment centers into three national laboratories and a national assessment center, thereby delegating more research-management responsibility to the laboratories. Reduce Washington, DC headquarters staff of ORD by half through attrition and reassignment. Replace some contract personnel at EPA laboratories with federal personnel. Establish new scientific career-track opportunities and performance-evaluation procedures for EPA laboratory scientists.
Research Grants: Increase annual funding for extramural, investigator-initiated, peer-reviewed, competitive research grants and centers from $20 million to $100 million. Create a competitive, investigator-initiated research-grants program for in-house scientists at ORD's laboratories.
Peer Review: Strengthen and expand peer-review practices for proposals, publications, risk assessments, and laboratory programs.
Fellowships: Create a program of 300 graduate-student fellowships.
The EPA administrator accepted those recommendations, and the actions taken in response to them amounted to the most important changes in the history of ORD.
ORD stated four main goals of the 1995 reorganization (EPA 1996a):
To reorganize and refocus its laboratories and other organizational components using risk assessment and risk management as organizing principles.
To increase its interactions with the academic community by expanding its competitive extramural research grants and fellowship programs.
To expand and strengthen its peer-review practices.
To institute a new strategic planning process for setting research priorities.
In addition, ORD decided to delegate some laboratory-management and administrative authority from headquarters to the laboratories, giving more decision-making responsibility to laboratory managers and eliminating a layer of headquarters administrators. Although the missions of individual laboratories were adjusted, no ORD laboratory facility was abolished in the reorganization. Each former laboratory director simply began to report to the director of a newly designated national laboratory instead of an ORD suboffice director at headquarters. The directors of the national laboratories report directly to the assistant administrator for research and development. The headquarters suboffices to which the laboratory directors previously reported were abolished, and ORD reduced its headquarters staff substantially through reorganization and attrition.
ORD has stated (EPA 1996a) that the most important strategic principle guiding the 1995 reorganization and refocusing of its program was the risk-assessment and risk-management paradigm developed by committees of the National Academy of Sciences (NRC 1983, 1994c), as summarized in Figure 2-2. Figure 2-3 illustrates some of the most important ways in which scientific and technical activities potentially contribute to the process of risk management.
The new risk-based organization of ORD is summarized in Figure 2-4. In recommending this reorganization, the New Beginning report (EPA 1994b) identified the following benefits and limitations:
Aligns laboratory missions with EPA's unique research responsibilities for reducing uncertainty associated with risk assessment, with research priorities directly related to needs of the environmental management decision process.
Focuses missions to provide a clear framework for determining the appropriate composition of the scientific work force and supports development of required critical mass of key scientific and engineering disciplines.
Permits organization of ORD research on a multi-media, multistressor basis involving human and ecological risks.
Enables the agency to better identify where cooperation and joint activities with other federal agencies, industry, and academia can yield sufficient gains.
Moves laboratory leadership to the field and creates necessary conditions to empower laboratory directors and federal scientific teams.
Creates opportunities to build new working relationships between ORD and its clients.
Creates the opportunity for greater scientific career development in the staff.
Science leadership and innovation by national laboratories must be guided by a strengthened research planning and decision process that includes effective participation of program and regional offices in establishing needs, priorities, and accountability.
Processes must be established to support laboratory based science leaders, allowing them to interact effectively in national and international policy and science forums.
Restructuring the ORD laboratories according to the risk-assessment paradigm was intended to ensure that research programs within the laboratories would focus their strengths on reducing the uncertainties in identifying, understanding, and managing environmental risks. Such a focus for the in-house EPA laboratories was also recommended by the SAB (EPASAB 1988). Emphasis on reducing the uncertainties in risk assessment was previously recommended in Safeguarding the Future: Credible Science, Credible Decisions (EPA 1992), and a 1994 NRC report later defined such uncertainties and variability more fully (NRC 1994c). Although the missions and scientific requirements of EPA and ORD are far broader in scope than the risk-assessment paradigm, ORD chose to focus its in-house laboratories on areas of research related to the aspects of risk assessment and risk management (e.g., aquatic toxicology, inhalation toxicology, control technology) that historically have been of greatest interest to EPA and its laboratories and are not high priorities for research in other agencies. Other research areas would mainly be supported through partnerships with other agencies and organizations, including ORD's extramural grants program. With the restructuring of the ORD research enterprise, it was considered essential that ORD laboratory personnel be capable of performing state-of-the-art research techniques, that they take leadership roles in national and international environmental research initiatives, and that they interact effectively with those components of EPA that need technical information.
Before 1995, roughly 30% of ORD's resources were devoted to what ORD called fundamental and anticipatory research. Since 1995, ORD has increased that proportion to nearly 50%. That increase has resulted in a corresponding reduction in the short-term applied projects and technical assistance that ORD provides for the agency's regulatory offices. In our committee's site visits (see Chapter 1), some complaints were heard from regulatory offices that perceived some reductions in technical support from ORD.
In our 1995 interim report, this committee endorsed ORD's move to focus on reducing uncertainties in risk assessment (also see NRC 1994c and Presidential/Congressional Commission on Risk Assessment and Risk Management 1997a,b) but cautioned that its success would depend critically on the effectiveness of the strategic-planning process. To coordinate research planning and management in the new organization, ORD created several cross-cutting groups (Figure 2-5), includ
ing a Research Coordination Council that includes representatives from the agency's regulatory program and regional offices, an Executive Council of laboratory and center directors, a Management Council of deputy laboratory and center directors, a Science Council of assistant laboratory and center directors, and Research Coordination Teams involving assistant laboratory and center directors (Figure 2-5).
Our interim report also offered a preliminary endorsement of the general scope and direction of other changes then being made in ORD; a more confident assessment was not possible at that time, because the changes needed time to take root. Now, the committee views the 1995 reorganization of ORD and the agency's peer-review practices to be still a work in progress but sufficiently mature to be assessed in the following pages.
One of the principal findings in Safeguarding the Future: Credible Science, Credible Decisions (EPA 1992) was that “EPA does not have a coherent science agenda and operational plan to guide scientific efforts throughout the agency and support its focus on relatively high-risk environmental problems.” The authors of that report (including two members of this NRC committee) singled out two areas that especially suffered from the lack of an adequate research planning process and the dominance of the annual budget process in setting the agency's scientific agenda. First, the panel noted that today's increasingly complex environmental problems require greater emphasis on cross-media approaches, whereas the budget process was largely driven by media-specific statutes and priorities. Second, the panel expressed concern that the lack of an adequate scientific planning process had prevented EPA from carrying out prolonged research or anticipating future scientific information needs. It noted that the only certain prediction for the future is that an environmental issue of critical importance that no one has anticipated will appear, so EPA should have a strong science base and flexibility to deal with unanticipated problems. However, it observed that EPA's science programs were not structured to allow for the stable funding needed to pursue the necessary long-term, anticipatory research and scientific assessment.
The 1992 Credible Science panel judged the EPA budget process to be an “obstacle to formulating and carrying out a coherent science strategy, ” and it stated that EPA cannot present Congress with a rational argument for an appropriate funding level for science without first developing a coherent strategic plan for its science efforts. The panel recommended that EPA strengthen its strategic scientific planning process and that a comprehensive, long-term scientific strategy drive its annual budget decisions (not vice versa).
Stronger coordination and planning of environmental research among U.S. agencies and greater outreach and research collaboration with other organizations in this country and abroad were also among the principal recommendations of Environmental Research and Development - Strengthening the Federal Infrastructure (Carnegie Commission on Science, Technology, and Government 1992) and Research to Protect, Restore, and Manage the Environment (NRC 1993).
Over the years, ORD has published several research outlook reports, and just before the 1995 reorganization, ORD was engaged in an “ issue-based planning process” (EPA 1993), organized around a selected set of 38 environmental problems and scientific questions, grouped into 12 broad research themes. However, those efforts mainly consisted of describing selected research topics and did not contain the essential elements of a strategic-planning process.
In 1995, our committee's interim report (NRC 1995b) recommended the following in regard to scientific planning in EPA:
Most sorely needed are a coherent scientific and technical strategy statement for EPA, a strategic plan for ORD, and a strategic plan for each ORD laboratory and center that is consistent with the agency and ORD plans. Each strategic plan should consist at a minimum of a vision statement, a mission statement, and a plan for achieving them. These documents need not be lengthy or complex, but they should be robust and specific enough to enable the agency and this committee to evaluate the intended role of ORD and its organizational components in providing scientific and technical knowledge to support national environmental programs, policies, and decisions, as well as to identify unnecessary geographical and functional duplication and significant gaps in ORD activities. It is crucially important that the strategic-planning process drive the development of ORD's budget, instead of being driven by it.
In 1997, as part of its response to the 1993 Government Performance and Results Act, the agency published the EPA Strategic Plan (EPA 1997b), which included the broad scientific goals described in the first chapter of this report. In connection with that agency-wide effort, ORD undertook a major effort to institute a new strategic-planning process, as recommended in the New Beginning report (EPA 1994b) and the 1995 interim report from our committee (NRC 1995b). A new high-level position, deputy assistant administrator for science, was created to manage the planning process. ORD published a strategic plan in 1996 and an updated plan in 1997. To develop and update its strategic plan, ORD instituted a process involving extensive consultation with all components of ORD, the other offices of EPA, the SAB, the NRC, and others from the private sector and the academic community. Described as a “living” document, the strategic plan (Figure 2-6) defined ORD's vision, mission, strategic principles, long-term goals and objectives, high-priority research topics, and potential measures for judging the success of the program.
The ORD strategic plan (EPA 1996a, 1997a) set forth a simple vision: “ORD will provide the scientific foundation to support EPA's mission.” It defined ORD's mission in four statements:
Perform research and development to identify, understand, and solve current and future environmental problems.
Provide responsive technical support to EPA's mission.
Integrate the work of ORD's scientific partners (other agencies, nations, private sector organizations, and academia).
Provide leadership in addressing emerging environmental issues and in advancing the science and technology of risk assessment and risk management.
The plan set forth nine strategic principles for ORD:
Focus research and development on the greatest risks to people and the environment, taking into account their potential severity, magnitude, and uncertainty.
Focus research on reducing uncertainty in risk assessment and on cost-effective approaches for preventing and managing risks.
Balance human health and ecological research.
Infuse ORD's work with a customer/client ethic that breaks down organizational barriers and ensures responsiveness to ORD's internal and external customers.
Give priority to maintaining strong and viable scientific and engineering core capabilities that allow us to conduct an intramural research and technical support program in areas of highest risk and greatest importance to the Agency.
Through an innovative and effective human resources development program, nurture and support the development of outstanding scientists, engineers, and other environmental professionals at EPA.
Take advantage of the creativity of the nation's best research institutions by increasing competitively-awarded research grants to further EPA's critical environmental research mission.
Ensure the quality of the science that underlies our risk assessment and risk reduction efforts by requiring the very highest level of independent peer review and quality assurance for all our science products and programs.
Provide the infrastructure required for ORD to achieve and maintainan outstanding research and development program in environmental science.
And it proposed six long-term overarching goals for ORD:
To develop scientifically sound approaches to assessing and characterizing risks to human health and the environment.
To integrate human health and ecological assessment methods into a comprehensive multimedia assessment methodology.
To provide common sense and cost-effective approaches for preventing and managing risks.
To provide credible, state-of-the-art risk assessments, methods, models, and guidance.
To exchange reliable scientific, engineering, and risk assessment/risk management information among private and public stakeholders.
To provide leadership and encourage others to participate in identifying emerging environmental issues, characterizing the risks associated with these issues, and developing ways of preventing or reducing these risks.
More-detailed sets of objectives for each of those go als are described in the plan.
ORD's strategic plan recognizes that there will never be sufficient funding to investigate every identified environmental problem, and that there is a need for a systematic basis for organizing its research enterprise and setting priorities among the many research topics that ORD could address. The approach adopted was based on risk magnitude and risk-reduction opportunity.
The 1996 plan and its 1997 update described a research priority-setting process (Figure 2-7) and criteria (Figure 2-8). The process of selecting high-priority research topics involves consultation with sources
within and outside EPA, followed by assessments of the importance of various research areas to EPA's mission, the risk-assessment and risk-management criteria under consideration, and other factors.
Our committee agrees that EPA's research and development program should focus on scientific and technical areas where there is the
greatest potential for reducing adverse impacts on human health and the environment. This strategy is appropriate in terms of EPA's mission. It will help enable EPA to control or minimize risks as efficiently as possible while focusing the limited resources of the agency and the nation on situations where the pollution has the greatest impact and where the greatest reductions of risk can be expected.
Risk-based prioritization is an appropriate way for ORD to focus many of its in-house research efforts and set priorities among identified environmental problems to investigate. However, risk assessment has important limits as a tool for priority setting. It is still a relatively new concept, and its methodology is still maturing. It tends to be most useful for problems that are well defined and data-rich, and less useful for future or emerging problems that are not yet well defined. There is not as yet any objective way to compare fundamentally different outcomes, such as comparing cancer with non-cancer effects or adverse human-health effects with ecological or property damage. Comparing voluntary with involuntary risks and short-term impacts with future effects are also problematic. Further, present techniques for characterizing the adverse effects of chemical mixtures or other multiple stressors are primitive and inexact.
Applying the process and criteria in its 1996 strategic plan and 1997 update, ORD selected six high-priority research topics and six other important research areas to receive special attention over the following few years within the broader ORD program, and it described potential research tasks, products, and uses for each of the topics and areas in the strategic plan. Of the six highest-priority topics chosen, three are targeted at specific environmental problem areas – safe drinking water, with an initial focus on microbial pathogens, disinfection by-products, and arsenic; high-priority air pollutants, with an initial focus on particulate matter; and emerging environmental issues, with an initial focus on endocrine disruptors. The other three high-priority topics address broader questions of methodology and approaches – research to improve ecosystem risk assessment; research to improve health risk assessment; and research on pollution prevention and new technologies for environmental protection. In fiscal year 1999, ORD devoted about $343 million – a little more than 61% of its total resources of $559 million – to these six high-priority research categories:
Safe Drinking Water
Ecosystem Risk Assessmen
Health Risk Assessment
Additional areas discussed in the 1996 strategic plan and the 1997 update as being important, but not of the highest priority, included research on tropospheric ozone, airborne toxicants, and other air pollutants; indoor air quality; contaminated groundwater, soils, and sediments; exposures to pesticides and toxic substances; waste-site risk characterization; waste management and site remediation; and environmental monitoring. The strategic plan also emphasized the importance of anticipatory research, exploratory research, and other continuing efforts to identify emerging issues, as the SAB had urged in its 1995 report Beyond the Horizon: Using Foresight to Protect the Environmental Future (EPASAB 1995).
In its strategic plan, ORD described in general terms the steps involved in translating strategic-planning decisions into a research program. These steps involve the development of research plans; the determination of who should conduct the work (beginning with choices between in-house or extramural investigation); the development of budget operating plans and laboratory implementation plans for inhouse work or appropriate extramural mechanisms, such as grants, cooperative agreements, or contracts; and the process of planning for research-information management.
Our committee commends ORD for the progress it has made to date in developing its strategic and research plans. However, we believe that there can be further substantial improvements.
Although the ORD strategic plan discusses the processes and criteria by which decisions are made on research priorities, funding allocations, and who will do the work (in-house or outside the agency), the
plan describes the decision-making processes and criteria only in very broad terms. The plan lists several factors that are typically considered when ORD decides who will perform each research activity. These factors include the nature of the work, who has the appropriate expertise, how urgently the results are needed, the degree to which the work must be specified or can be made flexible, the available in-house capacity, the potential value of involving multiple institutions, and the opportunities for funding leverage. ORD has a number of mechanisms available for extramural funding, including interagency agreements, contracts, grants, cooperative agreements, and fellowships. Each of the mechanisms has its own features, advantages, and limitations. The process by which ORD decides whether a project or task is to be performed by in-house staff or through one or more of its extramural mechanisms is of crucial importance to the quality of the work and the cost-effective management of resources. Unfortunately, the descriptions of ORD's decision-making process are inadequate for most persons outside ORD to reconstruct or review the decision-making process. Our committee urges ORD to make processes of priority setting, resource allocation, and intramural or extramural decision-making more transparent and better documented to give the decisions greater credibility to the broad range of stakeholders within and outside the agency.
ORD's strategic plan also discusses the criteria that ORD will use in providing “technical support,” which ORD defines as “activities ORD conducts in response to specific requests by the Program Offices, Regions, or states to address well-defined needs that are not covered by ORD's research program.” Decisions to allocate ORD funds to technical support are generally based on the potential value of such work to the agency's regulatory programs, the extent to which ORD has unique scientific or technical capabilities to address the problem, the potential benefits to environmental quality and human health relative to the resource requirements for technical support, and the extent to which ORD judges it can help solve the problem. Again, ORD's description of the decision-making process for technical assistance is somewhat vague.
During the committee's site visits and interviews, the staff of some EPA regulatory program offices expressed concern that ORD's support of their programs appears to have diminished since the 1995 reorgani-
zation. Some of the program-office staff expressed the concern that, in practice, they have little influence on ORD's research priorities through the Research Coordination Council or any other mechanism. They felt that they need a stronger voice in the setting of ORD' s priorities, and that ORD should be held more accountable to the agency's other offices for performing agreed upon tasks. Some program-office staff members also expressed the concern that ORD's performance is often too slow to meet the needs of other EPA offices. As a result, they acknowledged to the committee that EPA regulatory offices engage in some research without ORD involvement, even though those other offices are unable to afford much research.
Dissemination and Technology Transfer
Safeguarding the Future: Credible Science, Credible Decisions (EPA 1992) concluded that the academic community, Congress, other federal agencies, industry, the public, and even many persons within EPA are generally unfamiliar with the work of EPA scientists. The 1992 panel emphasized that many officials involved in funding EPA science were uncertain about what science products EPA had produced, and whether the quality and quantity of its products were commensurate with the dollars expended. It noted that EPA's policy and regulatory work receives a great deal of public attention, but the agency's science typically receives a similar degree of attention only when the scientific basis for a decision is questioned. The panel concluded that EPA should strive to make more widely known the short-term and long-term scientific goals and achievements of its research laboratories, contractors, and grantees. It urged the agency to develop and implement a coherent communications, outreach, and education plan to publicize the activities and accomplishments of EPA scientists.
Even within the agency, many regulatory and regional program officials throughout EPA's history have been largely unaware and even dubious of any important benefits from ORD's research program, and consequently they have not been supportive of ORD's budget. ORD's ongoing efforts to disseminate its research products and inform others about them have, with some exceptions, been meager and unimaginative.
The committee recommends that ORD substantially increase its dissemination and technology-transfer activities. ORD should publish a comprehensive annual summary of its in-house and extramural research and technical-support activities, emphasizing the results and potential applications of its work by other EPA offices, state agencies, industry, and others. ORD should also take stronger measures to assist its stakeholders within and outside the agency to understand and apply the results of ORD's work. ORD should strive harder in demonstrating its accomplishments and anticipated accomplishments to its stakeholders both within and outside the agency.
At the same time, the committee noted during its site visits and interviews with managers and staff of EPA's regulatory offices that not a single individual was in favor of moving the research program outside the agency or even moving the scientific and technical-service functions and resources from ORD to the control of the regulatory program offices. Each group of regulatory officials was asked that question, and the predominant answer was that such moves would weaken, not strengthen, the scientific foundation of EPA's actions and decisions. They considered it highly unlikely that the regulatory offices could ever attract and maintain the high level of staff expertise that ORD has assembled, and even in their worst criticisms of ORD, they believed that a research program outside EPA would be even less helpful than ORD to their needs. Instead, the regulatory officials urged that ORD find a way to become more relevant, helpful, and accountable, at least with respect to the technical services and short-term, problem-driven research components of ORD's program.
Research Plans and Strategies
As promised, ORD has published and continues to publish peer-reviewed plans related to individual high-priority research topics and certain other key research areas. It has published research plans for microbial pathogens and disinfection by-products in drinking water (EPA 1997c), arsenic in drinking water (EPA 1998b), endocrine disruptors (EPA 1998c), and pollution prevention (EPA 1998e), as well as research strategies for particulate-matter research (EPA 1996b), ecological research (EPA 1998d) and waste-management research (EPA 1999a). Each plan has been externally peer reviewed.
Members of our committee reviewed and discussed the research plans and strategies cited above. In general, the committee concludes that these plans are a useful step in the development of ORD's research-planning process. The committee was especially encouraged to note that many of the research priorities described in ORD's research plans for arsenic in drinking water (EPA 1998b) and endocrine disruptors (EPA 1998c) were independently confirmed in similar recommendations published by the NRC in subsequent reports on those topics (NRC 1999a,b). In other words, before receiving the reports from the independent expert committees of the NRC, ORD and its peer-review process arrived at many of the same scientific conclusions. Our committee considers that to be a noteworthy validation of these ORD research-planning efforts.
The committee finds that these and future plans and strategies can be substantially improved, however. Some of the plans (e.g., endocrine disruptors, waste-management research) failed to consider explicitly a substantial amount of research conducted or funded by other agencies and organizations. The plans and strategies contained little information about the resource levels required and measurable results or time-tables associated with the anticipated research. The plans and strategies generally give little insight as to how the research activities will be orchestrated and coordinated among ORD laboratories and other agencies and organizations. The basis for setting research priorities is not clearly described in any plan. The plans do not devote much attention to workforce skill mix, facilities, equipment, or data management requirements.
However, the plans and strategies that our committee reviewed are a promising beginning. Other plans under development by ORD will address research on risks to children, mercury, global change, environmental monitoring and assessment, human health risk assessment, and the drinking-water contaminants candidate list.
In 1997, a draft of EPA's particulate-matter research-needs document was reviewed by the Clean Air Scientific Advisory Committee (CASAC) of EPA's SAB (EPASAB 1997). Although ORD's strategic plan (EPA 1996a) emphasized the importance of using research to reduce uncertainties in risk assessment, and the particulate-matter research document also listed that as ORD 's top criterion for identifying research needs, the CASAC review concluded that the document failed to identify – or in some cases even acknowledge – the many significant
uncertainties about the relationships between airborne particulate matter and health risks, and it failed to place the stated research needs in the context of such uncertainties. The CASAC also expressed doubt about the technical feasibility of some of the research proposed in the EPA document.
Shortly thereafter, at the request of Congress, the NRC convened the Committee on Research Priorities for Airborne Particulate Matter to assist EPA in developing its research strategy in this area. In its first report (NRC 1998), the committee provided a conceptual framework for an integrated national program of particulate-matter research, identified 10 critical research needs linked to key policy-related scientific uncertainties, and proposed a 13-year, integrated research strategy with recommended short-term and long-term timing and estimated costs. In its second report (NRC 1999c), the committee commended EPA on its responsive implementation of the committee's recommendations. Some of the concepts and practices recommended for the particulate-matter research area might help ORD improve its research-planning activities in other areas of research. For example, in 1999, ORD began a pilot effort to translate several of its research strategies and plans for certain areas into multiyear implementation plans, as the NRC had recommended and demonstrated for particulate-matter research. ORD's previous practice had been to plan and implement all research on a year-by-year basis. The multiyear planning should enable ORD and reviewers of the plans to better evaluate research activities, anticipated products, and critical paths over time scales more relevant to a research program than annual projections.
Recently, the U.S. General Accounting Office (GAO) criticized ORD 's performance in reporting the progress of some of the other research plans listed above, noting that one of the agency's regulatory program offices so acutely needed information on ORD's work, well beyond what ORD was providing, that the regulatory office paid for the development of a system to track ORD's work. The GAO (1999) report stated,
Because the program office needed better information to monitor the status of the work laid out in the research plan and to track project-level resource expenditures, the Office of Water developed its own tracking system for the research on microbial pathogens, disinfectants, and disinfection by-products. Since 1997, the Office of Water has paid a contractor over $148,000 to develop and maintain the tracking system and input data on the status of individual [ORD] projects.
Commenting further on the lack of transparency and progress reporting in ORD's research planning and budgeting process, the GAO report stated,
. . . in commenting on [ORD's] fiscal year 2000 budget, the [SAB's]Research Strategies Advisory Committee indicated that the lack of transparency in the process used to set research priorities made it difficult for the Committee to evaluate the adequacy of the proposed budget. The Committee recommended that EPA make available information on high-ranking programs that it entertained during the budget-making process but could not fund because of overall budget constraints and competition with other programs. In addition, the Committee found that the criteria that EPA used to emphasize or de-emphasize programs in the proposed budget were unclear and recommended that EPA develop explicit criteria that can be used for setting research priorities during the budget development process. The Committee concluded that such an exercise would not only improve communication and understanding of the budget process for those outside the agency, but would also assist EPA in making its internal decision process more efficient.
To address these concerns, the GAO report recommended
First, to improve the link between research needs and resources and to better ensure that limited research funds within EPA and other organizations are most efficiently targeted, we recommended that EPA (1)identify the specific research that must be accomplished, (2) establish time frames showing when the results must be available, (3) estimate the resources that will be required to support the needed research, and (4) use these data to develop budget requests and inform stakeholders about what research will be funded. Second, we recommended that EPA im-
prove the tracking of ongoing research in relation to existing research plans and communicate the agency's progress so that the Office of Research and Development's key customers, including the Office of Water and outside stakeholders, can obtain timely and accurate reports on thestatus, timing, and funding of individual research projects.
The Government Performance and Results Act requires federal agencies to update their strategic plans every 3 years, and EPA is preparing to update its plan. ORD is developing its own updated strategic plan, to be published later this year. It is not expected to change the previously identified high-priority research topics or to depart significantly from the contents of the 1996 plan and 1997 update, but it is expected to add three new high-priority research areas – children's health, safe food, and global change.
Three years after the 1995 reorganization of ORD, its Board of Scientific Counselors (BOSC) expressed concern about the lack of strategic plans at the level of the individual laboratories and centers (EPABOSC 1998a-e). Although ORD had developed an overall strategic plan (EPA 1996a, 1997a), and the ORD national laboratories and centers had each developed mission statements, the BOSC found that none of the laboratories and centers had developed its own strategic plan. Recently, EPA advised our committee that all the laboratories and centers have drafted such plans, but they were developed too late for consideration by the BOSC in its first program reviews, and they were not provided to our committee.
Our committee is not convinced that ORD has provided adequate delegation of opportunities for leadership and accountability throughout the organization. The absence of published strategic and management plans for the laboratories and centers, as noted in the BOSC reviews, is problematic. At a minimum, ORD should make it clear that the directors of the national laboratories, centers, and divisions are responsible for
Selecting, defining, and justifying the problem priorities for their part of the organization, based upon the overall ORD strategic
goals, the comprehensive problem list, and the mission and capabilities of their respective laboratory, center, or division.
Identifying and developing the research and technical support programs and projects of their laboratory, center, or division, together with an organizational upgrading program that reflects the agreed upon priorities for their part of the organization.
Effectively executing these programs within the approved budget – a responsibility implying delegated authority to triage various programs within the context of approved budgets and priorities.
Developing effective channels of communication to ensure timely transfers of information to all levels of the relevant program offices.
The committee also urges that the strategic plans of ORD's laboratories and centers place substantial emphasis on needs and strategies for maintaining and upgrading scientific capabilities, including staff-skill mix, facilities, and equipment.
Strategic planning in ORD has been predominantly a “top-down” and widely inclusive effort, but that is only partially effective. The top-down strategic-planning effort should be matched by and integrated with a “bottom-up” planning approach as research plans are developed pursuant to the strategic plan, especially with regard to specific research program and project proposals. Individual research program and project priorities should be developed by individual laboratories and their divisions in response to the overall strategic goals and missions defined from above. In other words, the mission and goals should largely be defined in a top-down process, but the laboratory programs and projects should be defined in a more bottom-up process by the researchers who will implement them. In the committee's site visits and the BOSC laboratory program reviews, a number of suggestions were heard from principal investigators about the need for the bench and field scientists to have a stronger role in planning the actual research activities in response to the strategic plan. Their well-informed and realistic views of costs, time, probable difficulties, and likely outcomes of research could do much to ensure that ORD's plans are feasible and push the limits of what can be done.
THE ORD LABORATORIES
In the 1995 reorganization, ORD's 12 geographically dispersed laboratories were consolidated into three mega-laboratories, called national laboratories: the National Health and Environmental Effects Research Laboratory, the National Exposure Research Laboratory, and the National Risk Management Research Laboratory.
The National Health and Environmental Effects Research Laboratory (the Effects Laboratory) is the largest ORD laboratory, combining all the former health and ecological research laboratories into one national laboratory with headquarters in Research Triangle Park, NC (see Figure 2-4). In fiscal year 1999 the Effects Laboratory had 708 employees, of which approximately 250 were principal investigators, and its total budget was $118 million, including a $40 million extramural budget. The laboratory is organized into nine divisions; five of them, located in North Carolina and Ohio, address health-effects research – environmental carcinogenesis, experimental toxicology, human studies, reproductive toxicology, and neurotoxicology – and four divisions, located in Florida, Minnesota, Oregon, and Rhode Island, pursue environmental research with, to some extent, a geographical (regional) orientation. As of 1997, approximately 33% of the laboratory 's total budget and personnel were in the health divisions, 55% in the environmental divisions, and the remainder in administration (EPABOSC 1998a).
The overall mission of the Effects Laboratory is to perform laboratory and field research to identify and understand the health and ecological effects of environmental stressors and the likelihood of such effects occurring under conditions of environmental exposure. In keeping with ORD's focus on the “risk paradigm,” the Effects Laboratory focuses on the first two components of the risk-assessment process, hazard identification and dose-response assessment.
The National Exposure Research Laboratory (the Exposure Laboratory) was formed by combining former ORD laboratories in Research Triangle Park, NC, Cincinnati, OH, Las Vegas, NV, and Athens, GA, with headquarters in Research Triangle Park, NC (see Figure 2-4). In fiscal year 1999, the Exposure Laboratory had 448 employees and a total budget of $109 million, including a $45 million extramural budget. The Exposure Laboratory's overall mission is to perform research and development to characterize, predict, and diagnose exposures to hu-
mans and ecosystems, giving priority to the research that most significantly reduces the uncertainty in risk assessment and most improves the tools to assess and manage risk and to characterize compliance with regulations.
The National Risk Management Research laboratory (the Risk Management Laboratory) was formed by combining former ORD laboratories in Cincinnati, OH; Research Triangle Park, NC; Ada, OK; and Washington, DC, with central administration in Cincinnati (see Figure 2-4). In fiscal year 1999, the Risk Management Laboratory had 393 employees and a total budget of $108 million, including a $62 million extramural budget. Of the laboratory's six divisions, four are basically organized according to environmental compartments or media: air, water, land, and subsurface. The other two divisions address sustainable technology and technology transfer.
The Risk Management Laboratory is responsible for developing the scientific basis for environmental risk management affecting both human health and ecosystems. The Risk Management Laboratory conducts research and development on source or problem characterization, prevention methods, control methods, remediation or restoration methods, performance and cost verification, and technology transfer.
In 1996, EPA's ORD requested and obtained under the Federal Advisory Committee Act a charter for a new body, the BOSC, to advise EPA's Assistant Administrator for Research Development. The BOSC is not part of the SAB. Established with 15 senior expert members from universities and other organizations serving on an ongoing basis, including one member of this NRC committee, and augmented with temporary members appointed to serve on ad hoc subcommittees as needed, including additional members of this NRC committee, the BOSC was charged to evaluate the management and operations of ORD's research programs and peer-review practices.
As one of its first tasks, the BOSC was asked to conduct program reviews of the ORD laboratories and centers, including the strategies and practices used by the laboratory and center directors to implement ORD's strategic plan (EPA 1996a, 1997a) and the mission of each laboratory and center. The BOSC accomplished that task through self-study questions and site visits in 1997, and it completed reports of the program reviews in 1998 (EPABOSC 1998a-e).
The BOSC found much that it liked. Overall, it judged the Effects
Laboratory to be “very solid with a high potential for being a national leader in a number of areas” (EPABOSC 1998a) It concluded that the Effects Laboratory “has a solid research foundation, and has made significant efforts to establish priorities and directions consistent with elements applicable to it in the ORD strategic plan.” In the health-effects divisions of the Effects Laboratory, the BOSC especially praised the programs in neurotoxicology, reproductive and developmental toxicology, and human-chamber studies. In the Effects Laboratory's environmental-effects divisions, it praised ecoregions work and landscape ecology at the Corvallis division, freshwater and estuarine toxicity-test methods development at Duluth, and other programs. It commented that staff morale seemed to have improved in the Effects Laboratory since the 1995 reorganization. Principal investigators at the laboratory reported to BOSC that the research environment within the Effects Laboratory had substantially improved in recent years.
The BOSC concluded that the Exposure Laboratory was “conducting some high-quality, peer-reviewed science in high-priority areas for EPA” (EPABOSC 1998b). It noted that the Exposure Laboratory had made significant scientific contributions in source-exposure research; chemical, physical, and biological process modeling, especially urban and regional air-pollution modeling; environmental characterization research; exposure analysis and assessment research; exposure-dose research; analytical measurements; environmental process research; and animal exposure studies.
At the Risk Management Laboratory, the BOSC noted its “excellent national and international reputation for applying sound and innovative engineering principles to identifying and controlling air and water pollutant emissions from a variety of sources” (EPABOSC 1998c). It singled out the Risk Management Laboratory's strong reputation in emission-source characterization, and it credited the laboratory with being “highly responsive to ORD in its attempts to reorient its research planning to conform to the ORD strategic plan.”
In addition to the concern noted previously that none of the reorganized laboratories had developed a strategic plan, the BOSC identified other problems. The administrative structure of the Effects Laboratory had not become well established by the time of the BOSC review. The BOSC found that the health-effects research divisions were generally organized by scientific disciplines, but the environmental research di-
visions were more geographically defined. In addition, the health-effects components of the laboratory had centralized administrative operations, reflecting pre-reorganization practices, while the ecological-effects divisions, each of which had laboratory-level status before the reorganization, retained their own administrative operations. These disparities were reduced somewhat after the BOSC review (Reiter 1999).
At the Exposure Laboratory, the 1995 reorganization of ORD and its immediate aftermath brought a 28% decrease in personnel and a 56% decrease in the laboratory's overall budget. The decreases were due to problems with the conversion of contract personnel to federal positions and major reductions in contract funds (EPABOSC 1998b). These cuts required reductions in technical support for EPA's program offices as the Exposure Laboratory tried to increase its proportional focus on research.
The BOSC concluded that personnel and funding were insufficient to carry out the Exposure Laboratory's mission (EPABOSC 1998b). It expressed concern that only 168 of the 407 staff at the Exposure Laboratory had research degrees at the masters or doctoral level, whereas 103 staff positions were in administrative jobs, and about 45 of those were strictly management. It identified redundant administrative structures at each of the Exposure Laboratory's four locations – Research Triangle Park, NC; Las Vegas, NV; Cincinnati, OH; and Athens, GA. It urged ORD to increase the number of research and technical-support personnel at the Exposure Laboratory and reduce the number of administrators. It urged ORD to emphasize the hiring of postdoctoral researchers and research technicians.
The BOSC also found that ORD's 1995 reorganization plan and 1996-1997 strategic plan had “not infused themselves into the scientific culture” at the Exposure Laboratory. It observed that workers at all levels at the Exposure Laboratory were still trying to understand what the reorganization meant. The BOSC concluded that the reorganization had “not created any scientific excitement among the employees or change in the way they are doing their research.” It urged the Exposure Laboratory to maintain its commitment to the ORD reorganization and strategic plan for several years to achieve success.
At the Risk Management Laboratory, the BOSC noted that ORD's 1995 reorganization required the laboratory to broaden its mission con-
siderably while reducing its staff by 70 positions and its extramural resources by 60%, and it expressed concern that the Risk Management Laboratory might not receive the resources minimally needed to fulfill its broadened mission (EPABOSC 1998c). Previously, the Risk Management Laboratory had mainly performed an engineering and technology role, but the laboratory's new risk-management mission posed a considerable challenge and required a fundamental transformation. The BOSC concluded that the available resources and staff of the Risk Management Laboratory were inadequate for its new, broader mission. Such a mission requires resources and expertise in areas such as economics, management science, social and behavioral sciences, microbiology, ecology, systems analysis, and risk communication – resources and expertise that the Risk Management Laboratory lacked. The BOSC expressed concern that such changes might diminish the traditional engineering strengths of the Risk Management Laboratory and noted that the laboratory's longstanding competency in wastewater technology was no longer being used effectively. The BOSC also judged the Risk Management Laboratory's infrastructure to be inadequate. If the necessary resources were not provided to the Risk Management Laboratory, the BOSC recommended that the laboratory 's strategic goals and mission be reformulated to be more in line with its staff and facilities. In fact, the BOSC questioned whether ORD had adequately understood the talents and capabilities of the Risk Management Laboratory when it developed its strategic plan.
Noting that the Risk Management Laboratory's research priorities are heavily influenced by statutory requirements or court orders, as well as ORD, the BOSC expressed concern about the vagueness and lack of clarity of the laboratory's understanding of its research scope, how it sets its research priorities, how much flexibility it has in setting such priorities, and how it makes decisions about the allocation of available resources to various research activities. The BOSC also questioned whether the Risk Management Laboratory was preparing adequately to meet the part of its research mission dealing with the management of ecological risks.
Before the 1995 reorganization, as much as 60% of the Risk Management Laboratory's budget had been devoted to outside contracted research; the laboratory's scientific and technical staff were heavily involved in managing extramural projects, and in-house research was
minimal (EPABOSC 1998c). When ORD shifted the emphasis from extramural management to in-house research, the shift had a major impact on the Risk Management Laboratory, despite the fact that the Risk Management Laboratory still had 111 active cooperative agreements with 75 academic institutions at the time of the BOSC review (EPABOSC 1998c). The BOSC urged the laboratory to devote high priority to identifying the needed skill mix to perform its broadened mission, acquiring such staff, retraining present staff, and improving the laboratory infrastructure, including laboratory space and equipment. The BOSC also emphasized the need to reduce the burdens of bureaucracy, red tape, unnecessary or redundant committees, and other overhead activities on professional staff (EPABOSC 1998c).
The BOSC emphasized that stability of resources and personnel is critical for maintaining a strong research laboratory (EPABOSC 1998a). It pointed out that regardless of the time frames of EPA's regulatory activities, ORD must operate in a research environment, which requires a stable planning process and longer timetables, and that rapidly shifting priorities and unstable funding are detrimental to a research environment. The BOSC suggested the exploration of ways to improve stability by making resource allocations more flexible.
In addition to the BOSC reviews, each of the ORD national laboratories has conducted its own divisional and programmatic reviews, using experts from outside the agency in a process organized by the laboratory peer-review coordinator (see Chapter 3). The divisional reviews address the scientific content and quality of the activities of each laboratory division. The program reviews focus on the research goals, approaches, progress, and results of large-scale research program areas. Like the BOSC reviews, these divisional and programmatic reviews entailed questions and informational materials prepared in advance, on-site visits, reports for laboratory management, and responses from the laboratory divisions and programs.
Our committee, two members of which participated in the BOSC reviews of the laboratories and centers and most of whom participated in additional site visits to each laboratory and center as described in Chapter 1, generally concurs with the results of the BOSC program reviews. Overall, the committee believes that the 1995 reorganization of the ORD program is still a work in progress, but it has begun to mature, and the committee strongly supports continued efforts to refine it.
During our committee's site visits to the laboratories, we were pleased to meet with many outstanding researchers, some of them with world-class reputations. For example, scientists at the Effects Laboratory, who consistently publish nearly 300 research articles each year in peer-reviewed journals, include the current president of the Society of Environmental Toxicology and Chemistry, the president-elect of the Society of Teratology, and many research scientists who serve on the editorial boards of journals and as adjunct members of the faculties of major universities.
The committee recommends that ORD continue to have its laboratories, laboratory divisions, and major research programs reviewed by panels of outside experts approximately every 5 years. The committee favors having as many of these reviews as possible performed by the BOSC, according to its now-established procedures, because the BOSC has done well in its initial round of reviews, and it advises the assistant administrator for research and development, so it is more independent of laboratory management, or at least perceived to be, than internally managed review groups. However, the committee cautions that excessive use of external review panels can undermine the sense of responsibility of laboratory managers for identifying and resolving problems
Each of the ORD laboratories (and the Assessment Center) has conducted a competitive internal research-grant program. The number of awards and the amounts awarded have been modest, and the programs are being re-evaluated. The committee encourages ORD to pursue them.
THE ORD CENTERS
The 1995 reorganization of ORD created two national centers. The National Center for Environmental Assessment was created from ORD 's former Office of Health and Environmental Assessment, including the former Environmental Criteria and Assessment Offices in Cincinnati and Research Triangle Park. The National Center for Environmental Research and Quality Assurance (recently renamed the National Center for Environmental Research) was created from ORD's former headquarters Office of Exploratory Research and charged to implement a greatly expanded research grants and centers program.
In fiscal year 1999, the National Center for Environmental Assess-
ment (the Assessment Center) had 177 employees and a total budget of $35 million, including a $16 million extramural budget. The center develops methodologies for performing risk assessments and reducing the uncertainties in current risk-assessment approaches; conducts assessments of contaminants and sites of national significance; and provides guidance and support to agency risk assessors. It also contributes to the planning of research relevant to those activities.
The National Center for Environmental Research (the Extramural Center) is the smallest of ORD's laboratories and centers in terms of staff. In fiscal year 1999, the center had 90 employees and a total budget of $151 million, $139 million of which was extramural. The Extramural Center is responsible for the programs that fund ORD's extramural research grants, centers, and fellowships. In response to a recommendation from the NRC (1977), ORD has conducted a competitive, peer-reviewed, extramural research-grant program since 1979. Until 1995, funding for the grants program fluctuated between $5 million and $25 million per year.
In Safeguarding the Future: Credible Science, Credible Decisions (EPA 1992), an independent panel of senior academic scientists (including two members of our committee) expressed concern that EPA lacked adequate mechanisms for acquiring the best available scientific information from other scientific organizations and the broader scientific community. In 1994, an agency-wide steering committee (EPA 1994b) recommended strengthening the grants program and increasing its size to $100 million a year. It envisioned funding approximately 400 new grants each year, with an average grant lasting 2 or 3 years at an average level of $100,000 per grant year.
Grant funding was dramatically increased in the 1995 reorganization of ORD, and since 1997, the Extramural Center has funded about $100 million per year in its Science to Achieve Results (STAR) program of research grants and fellowships (Figure 2-9). It currently supports about 700 competitively awarded research grants, 15 research centers, and 300 EPA fellowships each year, all awarded through national competition (Figure 2-10). Each year it receives about 3,000 to 3,500 proposals and awards about 200 new grants and 120 new graduate fellowships. The Extramural Center's programs are administered by two divisions, one devoted to environmental science and the other to environmental engineering.
There were two primary reasons why EPA decided to strengthen
and expand the extramural research-grant program. First, recognizing that the immense need for scientific and technical knowledge about environmental problems far exceeded any realistic assessment of the budgets and capabilities of EPA's in-house laboratories, the agency wanted to increase greatly the collaboration and potential contributions of the nation's academic community through an expanded and strengthened, competitive, investigator-initiated grants program. Second, as discussed in the next section of this chapter, the agency wanted to reverse the trend of previous years in which many in-house research scientists and engineers in ORD laboratories were increasingly spending their time as administrators of extramurally funded projects at the cost of doing less and less in-house research. The primary concern was that EPA's laboratories were losing their expertise and abilities to perform first-rate research. In addition, concerns were heard about favoritism and poor oversight in the administration of some externally funded projects by EPA laboratory personnel. To change that, the agency centralized much of its extramural research funding in a rigorously competitive STAR grants program, and it sent a strong message to in-house laboratory staff that they would be expected to do more research
and less administration, and that they would be judged mainly by research accomplishments in the future.
Topics for STAR program solicitations are developed in a process that begins with agency-wide research-coordination teams that include representatives from ORD's national laboratories and centers, as well as EPA regulatory and regional offices. After considering the ORD strategic plan, the needs of other EPA offices, and input from external groups, decisions about potential solicitation topics are made on the basis of such criteria as the need for improving risk assessment or risk management in a topic area, the suitability of the topic for the grants process, the extent to which the topic may complement or be per
formed instead by in-house research programs, the possibility of collaborative funding with other federal agencies, and the availability of resources. The proposed solicitation topics are reviewed by ORD 's Science Council and Executive Council. Solicitation topics of potential interest to other agencies are considered in interagency discussions and by the Committee on Environment and Natural Resources of the National Science and Technology Council.
Peer review, discussed in detail in the next chapter of this report, is a major activity and defining feature of the Extramural Center. EPA's longstanding regulations (40 CFR Part 40 Section 40.150) for peer review of research grants and cooperative agreements specify that new grant applications “will be reviewed for technical merit by at least one reviewer within EPA and at least two reviewers outside EPA.” The regulations also specify review by EPA staff for other criteria, such as relevancy to EPA's needs and priorities. In 1995, at the direction of Congress, ORD began awarding a substantial portion of its grants through a process conducted jointly with the National Science Foundation (NSF). Working in partnership, ORD and NSF developed joint program announcements, peer-review procedures, and funding for several areas of research solicitations of mutual interest.
Today, peer reviews for scientific merit in the EPA STAR grants program are conducted under policies and practices that are generally modeled after those of NSF (1998) and are also similar in many ways to the extramural grants programs of the National Institutes of Health (NIH) (42 CFR Part 52h). All three agencies use external expert reviewers to assess the scientific and technical merit of extramural research proposals in a competitive award process.
The EPA STAR program, with a budget of $96 million in fiscal year 1999, receives approximately 3,000 to 3,500 applications per year for research grants, centers, and fellowships, demonstrating considerable interest from the academic community. In each of the past few years, roughly 20% of these applications have been judged through peer review as meritorious for potential funding, and about half of the meritorious applications have received funding (i.e., approximately 10% of the applications received). In fiscal year 1998, the latest year for which complete data are available, the STAR program received 1,666 research-grant applications, judged 389 of them to merit funding, and awarded funding for 167; an additional 37 grants were funded by other
federal agencies participating with EPA in the solicitations. In the same year, the STAR program received 1,251 fellowship applications, judged 168 of them to be excellent, and awarded 126.
NSF, with a much larger research-grant budget of about $2.5 billion out of its $3.9 billion total agency budget, reviews about 30,000 proposals each year and funds about 10,000 new awards annually, funding approximately 27% of the proposals it receives. NIH, with an even larger budget of about $17.9 billion in fiscal year 2000, reviews about 40,000 grant applications annually through its Center for Scientific Review and funds about 30%. The EPA STAR grants are typically funded for up to 3 years, although there are exceptions. The NSF grants are commonly funded for 1 to 5 years. NIH restricts funding of new grants to 25% of its grants budget to ensure continued funding for ongoing grants.
One important difference between the EPA grants program and the NSF and NIH programs in terms of peer-review practices derives from the different agency missions. NSF and NIH are science agencies, so their reviews focus mainly on technical merit and potential impacts of the proposed research. EPA, with its environmental protection mission, subjects research proposals to separate reviews for technical merit and relevance to the agency's program needs. The relevancy review is performed on applications rated meritorious by the merit-review panels. EPA program offices are consulted, often through research coordination teams, in the relevancy reviews. The relevancy reviews focus on criteria such as relevance to ORD's overall research strategy, potential to reduce scientific, risk-related uncertainties in decision-making, and applicability to agency programs and priorities.
In the EPA grants process, the merit reviews are performed by panels of outside experts, mostly from the academic community, in a process organized and led by a science review administrator on the staff of the ORD Extramural Center. The Extramural Center maintains a computerized Peer Review Panelist Information System containing resumes and other information about potential reviewers to support this process. Typically, each research application is submitted to three principal reviewers, and each reviewer is assigned approximately eight applications. The reviewers are consulted in decisions about the applications to be assigned to them. They generally serve under a professional services contract. Each reviewer receives the applications at
least 4 weeks before the panel meeting, as well as information about the solicitation to which the applications are responding, the review process and criteria, the conflict-of-interest guidelines, the other applications being reviewed by other experts on the panel, and other materials. Each reviewer prepares a written evaluation of each assigned application and gives it an overall rating. In the panel meeting, the applications are sorted into two groups: those having the greatest merit and those having lesser merit. All applications in the first group are presented by the panelists who reviewed them and discussed by the full panel. The panel then rates them by vote. Anonymous summaries of panel evaluations and, if requested, anonymous evaluations are provided to applicants.
In a generally similar process, NSF grant proposals are reviewed by an NSF program officer plus several external experts chosen by the program officer. NIH grant proposals are reviewed by a grants referral officer and then by a scientific review group, typically about 20 active biomedical researchers, mostly from outside NIH. They are appointed to multiyear terms upon the recommendation of a scientific review administrator and meet or teleconference about three times a year, sometimes seeking additional outside opinions.
Merit review criteria in the EPA grants program address both the research proposal and the investigator. For the proposal, reviewers are asked to assess factors such as originality, creativity, potential scientific knowledge contributions, appropriateness and adequacy of methods, technical merit of the proposed approach, feasibility, and quality assurance plans. For the investigator, reviewers are asked to evaluate qualifications, demonstrated knowledge, publication record, and time commitment. In a similar process, merit review criteria in NSF emphasize the intellectual merit of proposed research and its potential contributions to education and other societal goals. Also similarly, NIH criteria emphasize the potential significance of proposed research, the proposed approach, innovation, the qualifications of investigators, and the research environment. EPA, NSF, and NIH also consider inclusion and recruitment of minorities and the protection of humans, animals, and the environment.
All EPA research grantees are required to provide annual and final progress reports. ORD's Extramural Center posts abstracts of these reports on its web site. The center's web site also contains all grant
solicitations, lists all funded grants and centers, and provides a keyword search function. The center conducts annual all-investigator meetings for individual solicitation categories and prepares special summary reports of results and research-in-progress reports in selected topic areas. The center seeks to involve ORD laboratory personnel and promote communication with regulatory program staff in the grants program through its web site, special reports, progress-review workshops, and other mechanisms, including informal communication.
In 1997-1998, ORD's BOSC conducted program reviews of the national centers, as they also did for the national laboratories (see previous section). Reviewers included 15 senior expert members from universities and other organizations serving on an ongoing basis, including one member of our committee, and augmented with temporary members appointed to serve on ad hoc subcommittees as needed, including additional members of our committee. The BOSC focused on the mission of each center and the strategies and practices used by the center directors to implement ORD's strategic plan (EPA 1996a, 1997a). As with the national laboratories, the BOSC accomplished the review through self-study questions and site visits in 1997 and completed reports of the program reviews in 1998 (EPABOSC 1998d-e).
As with the national laboratories, the BOSC found positive things to say about the centers. It judged the mission of the Assessment Center to be of “significant value to ORD if attained” and observed “many positive attributes and strengths” within the Assessment Center, including “high quality and productivity of its scientific expertise” (EPABOSC 1998d).
The BOSC had high praise for the Extramural Center, commenting that “Center management and staff have exhibited extraordinary creativity and hard work in initiating programs to accomplish [the center's] mission” (EPABOSC 1998e). It applauded the “quality and commitment” of the center's director and staff as “quite impressive.” It particularly commended the Extramural Center's efforts to improve the integration of research efforts and the communication of results to target audiences. It praised the center's joint solicitations with other agencies and organizations; Adopt-a-Grant program for ORD laboratory scientists; workshop, research-in-progress, and state-of-the-science reports; web site; and other efforts to communicate program results to other EPA offices and outside audiences.
As with the national laboratories, the BOSC found that neither of the national centers, created in 1995, had yet developed its own strategic plan (EPABOSC 1998d,e).
The BOSC judged that too much of the Assessment Center's effort was being devoted to high-profile issues, short-term problems, and “firefighting,” and the Assessment Center was often stretched too thin (EPABOSC 1998d). In view of the agency's vast needs for risk assessments, the many risk-assessment activities being performed by the agency 's regulatory program offices, and the limited resources available to the Assessment Center, the BOSC urged the Assessment Center to rethink its role and become more of a risk-assessment leader, catalyst, and resource service center for the other offices of the agency– a source of advice, guidance, and methodology – rather than a primary performer of individual risk assessments. The BOSC suggested that the Assessment Center could be most effective by focusing not on performing or trying to “own” or control individual risk assessments, but rather on improving and supporting the scientific underpinnings of the risk assessments performed across the agency through the development, acquisition, testing, and maintenance of state-of-the-art methods and information that support the agency 's risk-assessment activities. The BOSC urged the Assessment Center to promote the suggested role by working creatively and aggressively to strengthen its relationships with other agency programs and understand their needs and expectations for risk-assessment services. The BOSC also suggested that the Assessment Center strengthen its relationships with ORD's national laboratories and ORD-supported academic research centers to promote research relevant to the Assessment Center's mission.
The BOSC expressed concern about the adequacy of the size and skill mix of the Extramural Center's limited staff resources in view of the enormous recent growth of the grants program, which had not been accompanied by commensurate growth in staff size or other internal resources (EPABOSC 1998e). The BOSC was concerned about the ability of the Extramural Center's staff to manage effectively the administrative and technical aspects of the greatly expanded program. In particular, the BOSC was concerned about the growing workloads and skill mix of the center's project officers. It expressed concerns about the adequacy of time and resources to track progress on awarded grants, facilitate appropriate interactions with and among researchers,
and ensure that the results of grantees' research are communicated to the rest of the agency and other interested groups. The BOSC recommended greater streamlining of grant and fellowship management practices and greater prioritization of the target audiences for the communication of research results.
The BOSC also raised questions about the Extramural Center's funding for social-science research. EPA support for social-science research has largely been limited to economic topics such as resource valuation or regulatory compliance costs. The BOSC noted the dearth of social scientists other than economists in EPA and wondered how the Extramural Center would be able to deal with such research areas as human behavior, risk perception and communication, law, history, philosophy, and ethics. It suggested that the Extramural Center seek help from other agencies with more experience in these areas.
Our committee generally concurs with the BOSC assessments of the national centers (EPABOSC 1998d,e). It is clear that the Assessment Center performs an important service to the agency in its methodological development work and other activities, including its support of the agency-wide Risk Assessment Forum. We also note that the Assessment Center has often made good use of outside expert advice. For example, in its development of agency risk-assessment guidelines, the Assessment Center has been highly responsive to many of the recommendations of the National Research Council's 1994 report Science and Judgment in Risk Assessment (NRC 1994c). Our committee especially concurs with the BOSC recommendation that the Assessment Center focus on being an advisor, catalyst, and resource for risk assessments conducted by the rest of the agency, rather than trying to do many risk assessments with its own limited resources. The Assessment Center should focus on being a research organization dedicated to advancing the state of practice in risk assessment, not a performer of individual risk assessments that could be done by EPA's regulatory offices.
We commend the Extramural Center for developing and conducting the STAR research-grants program in an open, careful, and credible process of national competition and independent merit review. This program has become a valuable mechanism by which EPA engages outside scientific and engineering talent in the agency's research program. Our committee further believes that the Extramural Center merits strong praise for the steps it has taken to collaborate with other
funding agencies and organizations, including joint research-grant solicitations with
the National Science Foundation, on water and watersheds, technology for a sustainable environment, decision-making and valuation for environmental policy, and environmental statistics;
the National Institute of Environmental Health Sciences, on children 's environmental health and disease prevention, chemical mixtures in environmental health, and endocrine disruptors;
other components of the National Institutes of Health on genetic susceptibility and human malformations;
the National Aeronautics and Space Administration, on ecosystem restoration, hazardous algal blooms, and ecological effects of environmental stressors using coastal intensive sites;
the National Oceanic and Atmospheric Administration, on ecological effects of environmental stressors using coastal intensive sites;
the Department of Energy, on bioremediation; and
the Office of Naval Research, on bioremediation.
At a smaller level, the Extramural Center has collaborated in grants solicitations with the American Water Works Association Research Foundation, the Association of California Water Authorities, and the Chemical Manufacturers Association.
EPA can derive substantial benefits from these joint solicitation programs. They enable the participating agencies and organizations to learn from each other, pool and leverage resources, engage experts who normally work with other agencies, and expand the scope and use of the research results to more stakeholder audiences and users. These are win-win ventures.
The committee also commends the Extramural Center for developing and maintaining its excellent internet site, which makes available to everyone the abstracts of research applications funded by the Extramural Center and its interagency partners, the annual and final reports from grantees, and reports of workshops on the integration of research results and their relevance for decision-making.
The committee encourages the Extramural Center to continue its development of state-of-the-science reports on topics of grant solicita-
tions. Such reports can be of considerable value in communicating to others the results of research funded by the Extramural Center and its interagency partners, current knowledge in a given area, policy implications of what is known, and gaps in knowledge that can guide future research.
Although the committee supports the strengthened and expanded STAR grants program, it also recognizes that the funds used to increase the program approximately fourfold, from $23 million in 1993 to approximately $100 million today, came from reductions in other ORD programs. Much of it came from funds that the laboratories had previously used for interagency agreements, cooperative agreements, and contracts with other research organizations in government, the academic community, and the private sector. In its site visits to the laboratories, the members of this committee heard concerns from some ORD managers and staff about what they perceived to be the “loss” of these funds, and they lamented the damaging effects of the change on their laboratories ' relationships with the outside research community.
The committee strongly supports the Extramural Center's undergraduate and graduate fellowship programs, as discussed in the next section of this chapter, and commends the agency for establishing them.
The committee recommends that ORD continue to have both of its national centers, and major programs within them, reviewed by panels of outside experts approximately every 5 years. The committee favors having as many of these reviews as possible performed by the BOSC, according to its now-established procedures because the BOSC has done well in its initial round of reviews, and it advises the assistant administrator for research and development, so it is more independent of center management, or at least perceived to be, than internally managed review groups.
THE SCIENTIFIC WORK FORCE
For ORD and other offices of EPA, the ability to attract, retain, and support a capable and dedicated work force of scientists, engineers, technicians, managers, other professionals, and support staff is the most critical requirement for strong scientific and technical performance. Our committee is aware of many excellent scientists and engi-
neers in ORD who are highly qualified; perform first-rate research; publish in peer-reviewed journals; and participate actively in professional societies, advisory panels, and university faculties. The committee 's site visits to ORD laboratories and centers were especially encouraging in that regard, and the committee strongly concurs with the judgments expressed by ORD's BOSC (EPABOSC 1998a-e) concerning the substantial number of highly capable and productive staff members in the ORD laboratories and centers. Yet, the maintenance and proper support of a first-rate scientific and technical work force have always posed difficult challenges for EPA.
The ORD work force is aging. More than 47% of ORD's employees are 50 years old or older, and more than 550 ORD employees will be eligible to retire within the next 5 years. Periodic EPA hiring freezes, combined with intense scientific and technical job-market competition from the private sector and academic institutions, are making it extremely difficult for ORD to recruit the new talent needed to sustain and enhance its research work force.
Safeguarding the Future: Credible Science, Credible Decisions (EPA 1992) concluded that an inadequate infrastructure and lack of long-term support have limited EPA's ability to attract and retain outstanding scientists, and that EPA did not yet have the critical mass of such scientists needed to make EPA science generally credible to the broader scientific community. The 1992 report recommended continued attention to appropriate science and science-management career tracks – research career tracks for scientists in ORD, and career tracks for scientists in the agency's program and regional offices that are similar to those for agency attorneys. It emphasized that the criteria for scientific promotion in EPA should include evidence of continuing advancement in a scientific discipline, such as completion of coursework, receipt of board certification, publications in the scientific literature, and contributions to the work of scientific organizations. The report recommended that a panel of scientists from universities and other agencies regularly evaluate the productivity of EPA scientists as a requirement for promotion within the agency. It urged that compensation for EPA scientists be based on competition with the best of their peers. It emphasized the need to minimize the bureaucratic duties of scientists and to ensure that they spend a significant percentage of their time on scientific activities. The report also recommended increased contact and enhanced
rotational opportunities to enable EPA scientists to participate in the broader scientific community, including participation in academic organizations, professional society activities, industry, and other federal agencies. It also recommended rotational opportunities for non-EPA scientists to work in EPA science programs.
The 1992 report recommended that ORD recruit and make a long-term commitment of support for four to six research scientists and engineers with world-class reputations in areas vital to EPA's long-term strategy and direction. The panel envisioned that these eminent scientists and engineers would serve as examples and mentors for all scientists in EPA and would bring access to networks of world-class scientists to benefit the agency. The panel recommended that EPA's Science Advisory Board be asked to form a search committee.
In a self-study report prepared for ORD's BOSC, the Effects Laboratory identified expanded appointment authority to attract and retain high-quality professionals as one of its greatest needs (EPABOSC 1998a). It expressed concern about its ability to compete with industry and academia in some areas. The Effects Laboratory expressed support for ORD's postdoctoral program and suggested the creation of a mechanism similar to the Senior Scientific Leadership Corps created by Congress for the Department of Health and Human Services.
Because of rapid scientific advances, EPA has a continuing need to reassess its research skill base and increase its scientific and engineering capabilities in many areas, such as epidemiology, molecular toxicology, and industrial ecology. Economics and the social sciences are also of critical importance to EPA, especially in cost-benefit analysis and other decision-making aids. During the 1970s and early 1980s, ORD had an active in-house program of economic and social-science research, including both methodological development and applied studies. In 1983, this program and the resources supporting it were transferred to EPA's policy office; the budget for that program diminished considerably and was recently eliminated. ORD has continued to fund extramural economic and social science research at a modest level through its competitive grants program, and the policy office and EPA regulatory offices perform some economic studies, but EPA's inhouse program in economics and social-science research has diminished from approximately 30 ORD staff members in the 1970s to an almost entirely grants-based extramural program today. During the
1980s, EPA's policy office performed an economics oversight role for the agency through its regulatory impact-analysis function, but that role has since diminished.
ORD's graduate and postdoctoral fellowship programs have become an outstanding asset to the agency. This program is helping to develop the next generation of environmental scientists and provides valuable new talent to ORD's national laboratories. To prevent future shortages of environmental science and engineering personnel in critical disciplines, the federal investment in education programs should generally keep pace with overall research and development needs. Thus, EPA's fellowship program should emphasize specialities for which there is evidence of strong current and future demands. For example, in certain areas of specialization in toxicology, the current disparity between supply and demand is acute. Neurotoxicologists and genetic toxicologists who work on environmental problems are in short supply. The availability of social scientists adequately prepared for environmental research is also severely limited. EPA's fellowships program should emphasize disciplines such as these.
The committee recognizes that estimation of future research work force needs and projected resources can be difficult in disciplines related to environmental protection and environmental health. Estimates can be made, however. Trends in job placements, the number of people completing educational requirements in individual disciplines, and the public and private funding for environmental programs should be among the factors in determining the scope, emphasis, and priority specialization areas for educational support through EPA's graduate-student fellowship program.
In its 1995 reorganization, ORD sought to reverse the trend of previous years in which many in-house research scientists and engineers in ORD laboratories were increasingly spending their time as administrators of extramurally funded projects at the cost of doing less and less in-house research. The primary concern was that EPA's laboratories were losing their expertise and abilities to perform first-rate research.
As one ORD employee put it (Budde 1997),
In the early days of federal environmental research at EPA, government research scientists and engineers worked with their technicians in laboratories and pilot plants. It was hands-on R&D, and these people had the respect of their scientific colleagues in academia and industry.
As Congress passed one after another piece of environmental legislation, with daunting requirements and timetables, it provided EPA with ever growing quantities of extramural research money instead of authorization to hire more federal scientists and engineers.
EPA research scientists and engineers were offered promotions, power, and influence by becoming administrators of money instead of leaders of science and engineering. The few new hires were almost always managers or administrative support people. Gradually over the past 15 or more years, most EPA research scientists and engineers became money managers and administrators. EPA research by its own federal staff was ignored and even discouraged by management, and so it was effectively destroyed except in a few isolated pockets. Spending the big extramural bucks received all the attention.
In addition, concerns were heard about favoritism and poor oversight by ORD laboratory personnel in the administration of some externally funded projects. To change that, the agency centralized much of its extramural research funding in a rigorously competitive STAR grants program, and it sent a strong message to in-house laboratory staff that they would be expected to do more research and less administration, and that they would be judged mainly by research accomplishments in the future.
One of the nine strategic principles established by ORD states, “Through an innovative and effective human resources development program, nurture and support the development of outstanding scientists, engineers, and other environmental professionals at EPA.” In a 1996 workshop involving a cross-section of staff from ORD's laboratories, centers, and offices (EPA 1997a), participants identified the following general work-force-support needs to be of the highest priority:
Reduce red tape – Empower staff by reducing unnecessary paperwork.
Communications – Develop and implement a comprehensive communications plan to improve two-way communication and make electronic communications more effective within ORD.
Career advancement and development – Provide career enhancement opportunities for all employees.
Resources and infrastructure – Define “infrastructure” and provide adequate resources to support science.
To address those issues and others, ORD established a Human Resources Council in 1996. Chaired by an ORD laboratory director, the council has 25 members representing staff from each ORD office, center, and laboratory, with at least one representative from each geographical location, as well as representatives from the agency's human resources office, labor unions, and civil rights office.
One of the most important goals of the Human Resources Council should be to help managers within ORD identify ways to improve and maintain staff morale. Achieving and maintaining good morale are essential to EPA, whose future is inevitably affected by the zeal and confidence with which the staff carries out its work. Good morale is difficult if not impossible to define, however. It might be observed through the pride of employees in being identified with the agency and their pleasure in working for it, but perhaps the most obvious way to recognize morale, like health, is when it is damaged. Staff morale is a fragile thing, and motivational systems that are improperly devised or administered can damage it.
The committee was pleased to observe in its laboratory site visits that ORD has many competent and dedicated scientists, engineers, and other staff. At times, however, many ORD staff have been discouraged and pessimistic about the future of ORD and frustrated and uncertain about prospects for their own professional careers within ORD. The concerns heard by our committee were many: too much disruptive change in budgets, priorities, and policies, often crisis-driven; excessively bureaucratic procedural hurdles; too many scientifically underqualified administrators instead of research scientists managing laboratory programs; failure to replace departing scientists and technicians with new talent in a timely manner; lack of trust up and down the management chain; institutional faultfinding and paranoia; inadequate travel funds and other infrastructure support for nonmanagerial scientists; lack of explanations for decisions; unkept promises; criticism from Congress and others; pessimism that anyone will listen or be able to help. Undoubtedly, some of these complaints reflected individual problems, and ORD management has taken steps to address some of them. But feedback is a valuable commodity, and it pays to keep listening.
The committee offers the following observations and recommendations for developing and supporting the scientific work force.
Over EPA's 30-year history, the priorities, initiatives, and operating policies of ORD have often changed sporadically in response to shifting agency demands, the goals and priorities of different administrations, and congressional mandates. Although the much-decried “pollutant-of-the-week ” syndrome might be an overstatement, “priority-of-the-year” is close enough to the truth to be of concern in a research program. The sporadic character of research funding for major air pollutants is an example (Powell 1999). Too often, research on a particular pollutant becomes a high priority a year or two before a National Ambient Air Quality Standard is to be evaluated – usually too late for long-term studies. Then, after the agency has made a decision, administrative interest wanes and work is curtailed, despite the certainty, embodied in the Clean Air Act requirement for a re-evaluation every 5 years, that another cycle of interest will soon begin.
Research programs require the development of scientific and engineering talent, experience, and infrastructure. They cannot be turned off and on rapidly. Research requires a longer time scale than non-scientists often appreciate. A lack of stability in goals, priorities, practices, structure, or funding can be especially harmful to a research organization. ORD's historical lack of stability and sometimes disruptive changes have been attributed to growth in EPA's legislative mandates and priorities; specific directives from Congress in the appropriations process; changes in political administrations; changes in public attitudes; lawsuits and court decisions affecting regulatory programs and associated scientific needs; pressures from public groups or regulated parties; inadequate budgets to meet competing demands; recommendations from outside groups; and changes in the leadership of ORD.
The limited financial and human resources of ORD should be managed with a steady hand and a clear and persistent vision of how to maximize the gains in scientific understanding from ORD's budget and the creative time and energy of its staff. ORD should try to refrain from making abrupt shifts in research priorities or internal processes. It should seek feedback and consultation from staff at all levels and provide timely fore-warning when changes will be needed regarding in-house or extramural research budgets, responsibilities, organizational structure, or priorities for research projects, programs, and funding mechanisms.
ORD and EPA should make a special effort to resist a tendency commonly seen in large institutions to impose cumbersome bureaucratic procedures in response to management concerns. Institutional paranoia, fault-finding, and fear of possible fault-finding can paralyze and demoralize an organization. Excessively bureaucratic procedures are antithetical to a creative research program with high standards of quality, efficiency, and teamwork. ORD should frequently examine itself to identify and eliminate excessive bureaucratic safeguards, administrative hurdles, redundant requirements for approvals at multiple levels of management, and other bureaucratic impediments.
Research leaders at all levels in ORD should strive to minimize bureaucratic impediments, provide timely responses to requests from other organizations and from staff scientists and engineers, and place high priority on finding ways to increase flexibility in getting research done. ORD managers throughout the organization should be given the authority and resources to make decisions at the lowest appropriate level of management, provided that such decisions are compatible with EPA policies and ORD's strategic goals and budget priorities. Decisions that fit within this category include problem selection and program definition; acquisition of most equipment and supplies, personnel assignment; attendance at scientific meetings; inviting and supporting a visiting scientist; and granting permission for ORD scientists to work for a time in another laboratory in this country or abroad.
An organization that does not adequately aid the continuing improvement of its employees is remiss in its responsibilities and in the long run handicaps itself. Continuing career development for ORD's research staff is critical to the quality and productivity of their research. Opportunities for professional development are especially important for ORD scientists and engineers who, in the 1995 reorganization of ORD discussed earlier in this chapter, were asked to return to research after functioning as managers of extramurally funded projects.
In addition to in-service training, career development includes participation in professional society meetings and activities, as well as col-
laboration with scientists in other federal agencies, research centers, and universities. The Individual Development Plan (IDP) seems to receive wide support in ORD. The IDP is negotiated between each employee and supervisor. It addresses career paths, training, and rotational goals. The committee also recommends that ORD expand its programs for intellectual growth and exchange with other research organizations. Additional resources are needed for travel to scientific meetings and collaboration with scientists in distant laboratories inside and outside EPA. There is widespread dissatisfaction among ORD research staff with the lack of travel support to enable such interactions. The lack of such support inhibits their ability to share their results with the scientific community at large through participation in conferences and workshops and to develop and sustain meaningful scientific collaborations with scientists at other institutions.
Mentoring of junior staff – scientific and support staff alike – is another key element of a successful research program. The committee observed many elements of a mentoring program in ORD's laboratories. The committee recommends that ORD establish a more formal mentoring program to promote professional growth of all of its junior staff.
The committee recommends that ORD increase sabbatical assignments for ORD researchers to gain experience in other scientific organizations, and that ORD bring more scientists from universities, other government agencies, and private organizations to ORD laboratories and centers for visiting appointments.
The long-term success of EPA's research and development program depends on a staff of well-trained, creative scientists, engineers, and other professionals. Personnel policies affecting the recruitment, retention, and support of research personnel at ORD are critical. ORD often has not provided recruitment and retention packages for research scientists and engineers that are competitive with those of other research organizations in academe or industry. While federal personnel policies impose limits and difficulties, the difficulties can be overcome, as demonstrated by the strong in-house research programs of NIH and NIST.
In 1989, ORD instituted a new program for the recruitment and pro-
motion of a limited number of nonmanagerial, senior scientific and technical research and development staff at its laboratories and centers. ORD currently has eight scientists and engineers serving in such positions, which are called “ST” positions. Individuals in those positions are paid at Senior Executive Service levels, and recruitment bonuses up to 25% of annual salary are allowable. No managerial duties are required in these research positions. ORD instituted a special board and has additional experts on an ad hoc basis to review the scientific and technical qualifications of both in-house and external candidates and the performance of individuals in such positions. The board includes senior staff from the ORD laboratories and centers, as well as outside scientists and engineers who are typically at the level of full professor. In fiscal year 1999, ORD promoted four in-house research scientists to ST positions.
EPA has also established an excellent fellowships program, which is intended to revitalize the ORD work force through an infusion of young scientists and engineers. Fellows are selected through a merit-based, competitive process in targeted scientific and technical disciplines relevant to needs identified by ORD. They are generally appointed for 3-year terms, with salaries ranging from about $40,000 to $60,000 per year, full employee benefits, and relocation expenses. ORD advertises the program through professional scientific societies, university graduate departments, scientific periodicals, and EPA 's web site. In fiscal year 1999, ORD sought candidates in water resources engineering and management; urban and regional planning; environmental science; chemistry; biology (e.g., cell, developmental, molecular, reproductive, neurobiology, and animal and plant physiology); biochemistry; physical chemistry; human health sciences; endocrinology; epidemiology; pharmacology; toxicology, population and community modeling; geography; microbiology; hydrology; ecology (e.g., aquatic, coastal systems, coral, ecosystem, estuarine, landscape, marine, and microbial); ecotoxicology; meteorology; applied mathematics and statistics; systems analysis; computer science; geographic information science; geomorphology; geostatistics; genetics; immunology; environmental, chemical, and biomedical engineering; and other disciplines. In the first year of the program, ORD was swamped with nearly 2,500 applications for 100 available fellowships. In 1999, ORD received 1,061 applications for 50 available positions. ORD placed 47 applicants in the laboratories, including 15 at the Effects Laboratory, 21
at the Exposure Laboratory, and 11 at the Risk Management Laboratory; and 3 went to ORD's Assessment Center. The 50 included 28 women and 10 minority fellows. In fiscal year 2000, ORD received 798 applications but was only able to hire 16 candidates due to an agency hiring freeze.
Criteria for the selection and advancement of research managers in ORD should emphasize persons who are accomplished scientists in their own right and have the ability to select, inspire, lead, and otherwise encourage other scientists and engineers to succeed in meeting agency research needs by pushing back the frontiers of understanding in their fields of special competence.
The following criteria are suggested:
Accomplishments in original scientific research, demonstrated by publication in refereed scientific or engineering journals.
Demonstrated ability to develop and implement high quality scientific and engineering research projects and programs relevant to agency and national needs.
Credibility and reputation in the scientific community.
Ability to select, inspire, and lead scientists and engineers to further their professional development by increasing their scientific and technical competence and their ability to summarize and add to the policy-relevant scientific and engineering knowledge needed by EPA and the nation.
Ability to communicate research needs, plans, and results to policy-makers, Congress, the scientific community, stakeholder groups, and the public.
Research managers in ORD should be scientifically and technically accomplished, but they should also be capable administrators and personnel managers. The selection of capable people, their support and development, and sometimes their discharge are among the most important tasks any ORD supervisor should perform. Failure to take the time to choose wisely in the first place or to work as long and as patiently as it takes to help a miscast employee move into a better role
within the research program, or out of the office altogether, can result in more lost time and declining public support than virtually any other mistakes. Leaders in ORD should consult regularly with, and be perceived to seek and consider advice from, the scientists and engineers within ORD. Research managers in ORD should be selected on the basis of scientific competence and the personnel skills needed to lead and nurture professional development of their staff scientists and engineers.
Our committee's vision for the future of EPA's research program requires leaders who have technical competence; managerial abilities; communication skills; knowledge and skills in research planning and administration and in the public decision process, including its political dimensions; and the ability to marshal constituencies for an effective research program.
The issue of scientific leadership is discussed further in the last chapter of this report.