2
Overview of the STAR Program1

The Science to Achieve Results (STAR) program is the primary funding mechanism of the Environmental Protection Agency (EPA) for supporting extramural research grants and graduate fellowships in engineering and the environmental sciences. The program was established to augment EPA’s research and scientific activities by funding independent but coordinated research efforts at academic and nonprofit research institutions.

Before the establishment of the STAR program, EPA supported its regulatory mission through research conducted at or sponsored by an array of laboratory and other technical facilities across the nation. The Office of Research and Development (ORD) operated 12 research laboratories, four assessment offices, and four field stations (EPA 2003a).

The ORD laboratories were headed by managers who had extensive resources within their control, including funds for supporting intramural and extramural research. There was the perception by many people that deci

1  

The information in this chapter was obtained from presentations to the Na-tional Research Council committee, interviews with various EPA staff, the experience of committee members who have worked with EPA’s Office of Research and Development and the STAR program, and informal communication with many persons associated with the STAR program, both in and outside the agency. Much of the information presented here is a composite of multiple observations from those diverse sources and cannot be attributed to any specific source. The committee has attempted to verify the information, but committee members have not observed most of the processes and procedures described, which generally have not been recorded in published documents.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 20
2 Overview of the STAR Program1 The Science to Achieve Results (STAR) program is the primary funding mechanism of the Environmental Protection Agency (EPA) for supporting extramural research grants and graduate fellowships in engineering and the environmental sciences. The program was established to augment EPA’s research and scientific activities by funding independent but coordinated research efforts at academic and nonprofit research institutions. Before the establishment of the STAR program, EPA supported its regulatory mission through research conducted at or sponsored by an array of laboratory and other technical facilities across the nation. The Office of Research and Development (ORD) operated 12 research laboratories, four assessment offices, and four field stations (EPA 2003a). The ORD laboratories were headed by managers who had extensive resources within their control, including funds for supporting intramural and extramural research. There was the perception by many people that deci 1   The information in this chapter was obtained from presentations to the Na-tional Research Council committee, interviews with various EPA staff, the experience of committee members who have worked with EPA’s Office of Research and Development and the STAR program, and informal communication with many persons associated with the STAR program, both in and outside the agency. Much of the information presented here is a composite of multiple observations from those diverse sources and cannot be attributed to any specific source. The committee has attempted to verify the information, but committee members have not observed most of the processes and procedures described, which generally have not been recorded in published documents.

OCR for page 20
sion making was highly decentralized; laboratory managers had substantial local autonomy and control over funding decisions. There was no coherent and transparent policy for judging and selecting proposals or cooperative agreements, and peer review, as it is commonly used in the scientific community, often was not used (Johnson 1996). The growing U.S. environmental agenda placed an increasingly heavy burden on ORD for new research results; it was increasingly difficult for ORD to respond in a timely manner, and laboratory managers relied more heavily on contracts and cooperative agreements for meeting the demands. Before 1992, ORD funding for contracts was roughly $160 million, for cooperative agreements $100 million, and for research grants $40 million. Those funding divisions created problems related to the proper management of the research and to ensuring that the work was responsive to the needs of the program offices (Johnson 1996). This chapter reviews the evolution of the STAR program; the components of the current program, including the research fields it covers; and the procedures for selecting research topics and awarding grants. EVOLUTION OF THE PROGRAM Robert Huggett, the assistant administrator of EPA for ORD, reorganized ORD and initiated the STAR program in 1995 by reallocating $57 million in funds from other ORD-sponsored research efforts (primarily the “exploratory research” program). The STAR program was assigned to one of the agency’s newly established research centers, the National Center for Environmental Research and Quality Assurance, now known as the National Center for Environmental Research (NCER) (see Figure 2-1). The program’s research focus has been developed specifically to meet the research needs of EPA and is run in accordance with the ORD Strategic Plan (EPA 2001). Although the STAR program apparently was not established with a defined mission or set of goals, EPA has developed a set of 6 goals for the program (P. Preuss, EPA, Washington, D.C., personal commun., August 5, 2002): Achieve excellence in research. Focus on the highest-priority environmental science and engineer-ing needs to assist EPA in its mission.

OCR for page 20
FIGURE 2-1 Location of the STAR program in the EPA hierarchy.

OCR for page 20
Develop the next generation of environmental scientists. Achieve high levels of accountability and integrity. Form partnerships and leverage resources. Communicate and integrate research results. The program began with three components: focused requests for grant applications, an exploratory research grants program (which invited grant applications to conduct exploratory research in environmental physics, chemistry, and biology without designating particular program foci), and a graduate fellowship program (EPA 1996a). During the intervening years, the components and management of the program have changed in response to changing agency needs, experience gained in operating the program, and external reviews. The program grew rapidly during its first 4 years, but its funding has since remained relatively constant, with the total STAR budget fluctuating around $100 million per year, as indicated in Figure 2-2. In its initial year, the STAR program accounted for 11% of total R&D expenditures. Over the intervening years, total expenditures for ORD have fluctuated between $500 and $600 million, and the STAR program now accounts for about 18% percent of the ORD total. Accounting for changes in prices, of course, the value of the STAR funds has increased less than the expenditures. For instance, when deflated by the consumer price index, the value of the STAR grants was about the same in 2002 as in 1997, 2 years after the program began. The problem of increasing costs is most noticeable in the fellowship component of the program. Tuition costs have typically been increasing at twice the rate of inflation over the last decade and in some regions have almost doubled (College Board 2001). EPA pays for the increases as long as tuition does not exceed $12,000 (including fees), but it has not adjusted the cap to reflect increasing tuition costs since the program began.2 At high-cost institutions, the fellowships may therefore be much less adequate than they were when the program began (although still larger than those offered by many other fellowship programs). During its 8-year life, the STAR program has evolved in several ways. Some of the changes have accompanied changes in the program’s funding. The average size of the individual investigator and center grants has in 2   As this report was being prepared, EPA reported that it is reviewing this issue (J. Puzak, EPA, personal commun., October 4, 2002).

OCR for page 20
FIGURE 2-2 STAR program appropriations. Source: P. Preuss, EPA, presentation to National Research Council committee, March 18, 2002. creased from $289,000 in FY 1995 to $743,000 in FY 2001 (J. Puzak, EPA, Washington, D.C., unpublished material, 2002). As EPA allocated more funds to the STAR program, it was able to induce other agencies with similar interests to enter into partnerships and provide supplementary funds (Figure 2-3). The agency has increased the funding of research fellowships approximately in proportion to the total funding for the STAR program (see Figure 2-4), and this has resulted in an increase in the number of fellowships awarded. However, because the size of EPA’s grants has increased, the increased funding has not resulted in an increase in the number of grants awarded. The program has evolved in other respects. Part of the evolution has been in response to changing agency research priorities. The amount of money allocated to the exploratory grants program has diminished; the emphasis has shifted to “focused” research solicitations, although some of these solicitations may also support some very basic, or “core,” research efforts. EPA has modified, improved, and strengthened important elements of the program as it has gained experience in managing a peer-reviewed, competitive research program. For instance, the improved quality of the peer

OCR for page 20
FIGURE 2-3 Funding for STAR partnerships. NIEHS, National Institute of Envi-ronmental Health Sciences; NSF, National Science Foundation. Source: P. Preuss, EPA, presentation to National Research Council committee, March 18, 2002. review panels reflects the agency’s experience in selecting panel members, identifying possible conflicts of interest, and managing an independent peer-review process. The agency has also substantially improved the information provided to the public on the substance and progress of individual grants. Similarly, many of the early requests for applications (RFAs) were quite general, and as a result the proposals submitted were not always well focused on the agency’s specific research needs. Where the agency considered this to be a problem, it has made the RFAs more focused. That also was apparently one of the reasons for reducing the exploratory research grants program.The agency recognized that some research topics could be

OCR for page 20
FIGURE 2-4 STAR graduate fellowship program obligations. Source: J. Puzak, EPA, Washington, D.C., unpublished material, 2002. addressed better by interdisciplinary teams of researchers than by individual researchers working under uncoordinated grants. For that reason, the STAR program began to provide more funding to interdisciplinary “research centers.”3 Modifications have occurred as the program has matured. In the first years, the focus understandably was on establishing and improving the grant-making process. The agency has since increased emphasis on reviewing the progress of individual research efforts, encouraging coordination among researchers, and stimulating cooperation between intramural and extramural research efforts. The primary mechanism for accomplishing those ends is the “progress review” meetings, which include all the principal investigators working on a particular topic. Similarly, now that some of the early research projects have been completed, the agency is increasing its emphasis on developing effective ways of communicating research results to potential users. Some of the specific communication efforts are described later in this chapter. 3   EPA had funded some research centers before the STAR program was begun, and those were incorporated into the STAR program in FY 1997 (EPA 1996a).

OCR for page 20
Finally, the agency has modified the STAR program in response to its many external reviews. Those reviews and the agency’s responses to them are summarized in Appendix B of this report. COMPONENTS OF THE PROGRAM The STAR program has three main components: individual investigator awards, research centers, and student fellowships. Individual Investigator Awards Individual investigator awards provide funding to individual investigators or small teams of cooperating investigators who propose to conduct research on topics identified by the agency. The proposals are investigatorinitiated through universities, colleges, and nonprofit research institutions. Awards are generally for 3 years and for about $50,000 to $1,000,000. In most years, the STAR program has funded 170-200 individual investigator awards (EPA 2003b). In some cases, the RFAs are issued jointly by EPA and one or more other federal agencies or organizations—most significantly, the National Science Foundation (NSF) and the Department of Health and Human Services—interested in similar research issues.4 In any year, an additional 30-50 grants are awarded by the other agencies and organizations for the joint RFAs; these awards supplement the STAR awards and are not included in the STAR program statistics. Research Centers The research centers fund multidisciplinary efforts involving a number of scientists working in complementary fields. The multidisciplinary aspect 4   Cooperating agencies have included the Department of Energy, the Office of Naval Research, the National Aeronautics and Space Administration, the National Oceanic and Atmospheric Administration, the U.S. Department of Agriculture, the National Institute for Occupational Safety and Health, the National Institute of Environmental Health Sciences, the Department of the Interior, the American Waterworks Research Foundation, and the Association of California Water Agencies.

OCR for page 20
of the centers allows research programs to incorporate, for instance, exposure assessment and health-effects research with validation of riskmanagement and health-prevention strategies. Several research organizations or institutions may be involved in one center. Most centers are funded for 5 years, and the amount of the award typically exceeds $5 million over the life of the grant. Centers can also be jointly funded. For instance, the National Institute of Environmental Health Sciences (NIEHS) and EPA jointly fund the Centers of Excellence in Children’s Environmental Health and Disease Prevention Research. Figure 2-5 shows the amount of funding obligated for research centers and the shift from individual investigator awards to research centers. In FY 1995, all the research funds were for individual investigator awards; in FY 2001, about one-third of the STAR grant obligations went to support research centers. Fellowships The STAR Graduate Fellowship Program was established to “encourage promising students to obtain advanced degrees and pursue careers in environmentally related fields” (EPA 2002a). About 125 fellowships are awarded each year, although the Office of Management and Budget (OMB) proposed eliminating funding for this program in FY 2002 (P. Preuss, EPA, personal commun., March 18, 2002). The program is the only federal fellowship program designed exclusively for students pursuing advanced degrees in environmental sciences.5 The STAR fellowships provide more financial support than most other fellowships (Hogue 2002). For instance, NSF fellowships offer a total of $27,300 annually to doctoral students for 3 years. STAR fellowships provide a total of up to $34,000 per year and are available to both master’s and doctoral students. Of the total amount, $17,000 can be used for a stipend, 5   Fellowships are available to graduate students in environmental engineering, atmospheric sciences, chemistry and materials science, geology (including geochemistry and geophysics), economics (including market incentives and health and ecosystem valuation), geography, genetics (including genomics, proteomics, and bioinformatics), microbiology, public-health sciences (including epidemiology, exposure assessment, biostatistics, and health risk assessment), toxicology, aquatic ecology and ecosystems, oceanography and coastal processes, entomology, forestry, zoology, terrestrial ecology and ecosystems, and ecologic risk assessment (EPA 2002f).

OCR for page 20
FIGURE 2-5 Grant obligations to individual investigators and research centers. Source: J. Puzak, EPA, Washington, D.C., unpublished material, 2002. $12,000 for tuition and fees, and $5,000 for other authorized expenses. Students enrolled in master’s programs may receive up to 2 years of support and doctoral students may receive up to 3 years of support (EPA 2003c). In addition to contributing to the STAR goal of “develop[ing] the next generation of environmental scientists,” EPA hopes that these fellowships will assist the agency in its goal of “recruiting and retaining the next generation of well trained and highly qualified scientists” (EPA 2001; Preuss 2002a). That is an increasingly important issue for the agency in that it foresees the retirement of more than the average number of its scientific staff in the near future. STAR fellowships are highly competitive: only 10% of applicants receive funding (NCSE 2003). Prospective applicants are evaluated on the basis of rigorous peer review, academic and employment records, and potential. Fellowships have remained constant at about 10% of STAR funding. However, the program solicited no fellowship applications in FY 2003, because OMB eliminated STAR’s funding for this program. RESEARCH FIELDS The STAR program has been involved in a wide variety of research since its inception. Table 2-1 indicates the numbers of awards and amounts of support the STAR program has “committed” to different topics during its first 7 years of operation.

OCR for page 20
TABLE 2-1 Annual Commitments by Topica (in Millions of Dollars)   FY 1995 FY 1996 FY 1997 FY 1998 FY 1999 FY 2000 FY 2001   No. $ No. $ No. $ No. $ No. $ No. $ No. $ AIR Chemistry and physics 11 3.1 11 3.7     10 4.0             Ozone 1 0.4 12 5.1 7 10.5                 Air toxics     6 3.0 3 1.4 5 2.3 4 1.7         Particulates 2 1.0 6 1.8 9 3.7 14 44.9 8 4.5     4 3.8 Miscellaneous 9 4.1 3 1.5         5 1.7         Subtotal 23 8.6 38 15.1 19 15.6 29 51.2 17 7.9 0 0.0 4 3.8 WATER AND WATERSHEDS Water and watersheds 23 9.0 11 8.8 10 8.1 9 6.5 8 6.6         Drinking water     6 2.0 8 3.6 10 3.5 12 5.2 12 6.0 3 3.2 Contaminated sediments     3 1.7 5 1.8                 Algae blooms         4 1.0 7 3.0 2 0.9     9 3.1 Miscellaneous 14 4.4 6 1.9         9 7.0 2 0.7     Subtotal 37 13.4 26 14.4 27 14.5 26 13.0 31 19.7 14 6.7 12 6.3

OCR for page 20
to provide complete funding as long as Congress appropriates sufficient money and the grantee satisfies federal administrative requirements and submits the required annual progress reports. Commitments that have not been fully funded are given first priority for funding in later budget years (J. Puzak, EPA, Washington, D.C., personal commun., November 6, 2002). Implementation and Evaluation During the 3-5 years of a grant period (many grantees also receive, at their request, a 1-year, no-cost extension to the grant period), the STAR project officer is expected to monitor grant performance, including the submission of the annual progress reports and the grantee’s compliance with federal requirements, such as the OMB data-quality guidelines. Project officers also attempt to visit all the research centers and institutions that receive large individual grants to check on research progress. A major mechanism for evaluating research in progress is the scheduling of progress-review workshops, which are focused on specific topics, as determined by the project officer, and bring together all the STAR researchers and ORD staff working on the topic in question to discuss their progress and issues that have appeared in the course of the research. The key EPAfunded researchers and representatives of ORD laboratories doing associated research are expected to attend these meetings. Representatives of other EPA offices and federal agencies cooperating in the research effort are also invited, and the meetings are open to the public. In addition to providing an opportunity for peer review of research in progress, the meetings allow researchers to interact with one another. Although the greatest value of the workshops is probably the information exchange that occurs, EPA may also make the workshop proceedings available to the public. EPA reports that progress-review “workshops have been held in every program area, although there is not an annual workshop for every program area” (Preuss 2002b). With those monitoring mechanisms, the agency believes that it tracks grantees “more closely than other agencies” (Preuss 2002b). Efforts to coordinate STAR-sponsored research with the other units of ORD and with the program offices continue while the research is being conducted. If strong coordination is desired, the funding mechanism can be made a “cooperative agreement,” which allows EPA staff to work cooperatively with the researchers, rather than a grant, which requires the researchers to work independently. The progress-review meetings are another mech

OCR for page 20
anism for coordination. At a minimum, EPA staff and the public have access to the annual progress reports, which are posted on the NCER Web site for STAR research grants, and the program and regional offices can request the preparation of STAR “research capsule reports” (described below). Dissemination of Results The other activity that occurs after grants are awarded is the dissemination of information about the research efforts to potential users and the public. Many research programs pay little attention to dissemination, relying primarily on the researchers themselves, using such normal academic communication channels as conferences and journals, to spread the word. However, efforts through the STAR program are more aggressive. The office provides information concerning each grant that it awards on its Web site shortly after the award is made. It then requires grantees to prepare annual progress reports, and these too are made available to the public through the STAR Web site. A recently improved search engine allows users to quickly obtain a list of all STAR projects and reports dealing with a particular subject. In addition to the individual project reports, the STAR program prepares and publishes the following reports on its Web site: STAR reports, typically 4-6 pages long, which provide summary descriptions of research in progress on selected research topics for the general public. As of November 2002, STAR had released 10 of these reports (EPA 2002c). STAR research capsules, prepared at the request of EPA program and regional offices, which provide brief summaries of all the individual research projects that STAR is supporting on specific scientific issues. As of November 2002, STAR had released 18 of these reports (EPA 2002d). Progress-review workshop proceedings, which contain the presenta-tions made at selected progress-review workshops. As of November 2002, NCER had released nine of these proceedings, although it had sponsored many more workshops than that (EPA 2002e). State-of-the-science reports, which will summarize all the current scientific information related to selected environmental issues, regardless of who sponsored the research that provided the information. These reports are being prepared by contractors; none had been released by November 2002.

OCR for page 20
In addition to making that information available to those who have sufficient interest to visit and search the STAR Web site, STAR program officers apparently spend a substantial amount of time, including weekly conference calls, communicating research progress to the EPA program and other ORD offices (EPA/BOSC 2002). Communication to other potential user groups is more difficult, but the STAR program has experimented with some unusually aggressive mechanisms. In the mid-Atlantic region, for instance, a special pilot project was established to communicate STAR efforts to state and local agencies and public-interest groups and to determine the apparent relevance of the STAR research to the missions of the agencies and groups (Bradley 2002). In November 2002, STAR and EPA Region 1 (New England) sponsored a workshop at which STAR grantees in the region discussed their research with a similarly diverse group of interested parties. The workshop was well attended, and the participants strongly supported the program’s effort to communicate its research findings more quickly and efficiently (E. Abt, NRC staff, Washington, D.C., personal commun., Dec. 11, 2002). Apparently, the workshop was considered so successful that similar efforts are being considered in other regions. In spite of those efforts, the effective dissemination of research results to the diverse potentially interested audiences remains a challenge and is one of the issues highlighted in several reviews of the STAR program (for example, EPA/BOSC 2002). FELLOWSHIPS The process for awarding STAR fellowships is similar to but simpler than the process of awarding research grants. A notice indicating the availability of the grants, the eligibility criteria, the submission requirements, and the deadline for submissions is published in the Federal Register and on the STAR Web site, and copies are sent to graduate schools that have programs in environmental sciences and to individuals and organizations that have requested notification. The fellowship announcement is generally posted in mid-August and remains open for about 90 days, closing in midNovember. Potential applicants can sign up on the Web site to receive email notifications about fellowships. The fellowship applications are submitted to the Peer Review Division of NCER, which conducts the peer review for the applications. That division first reviews the applications to ensure that they are complete and satisfy the eligibility requirements. It then establishes a fellowship review panel composed of academics. The panel members are subject to similar

OCR for page 20
conflict-of-interest requirements as members of the peer-review panels for grants. Each fellowship application is assigned to three panel members designated as principal reviewers. They are responsible for reviewing, ranking, and preparing written comments on each of the applications they are assigned. The panelists receive the applications 4 weeks before the peer review meeting and assign each application to one of five categories—“excellent,” “very good,” “good,” “fair,” or “poor” on the basis of the following criteria (J. Gentry, EPA, Washington, D.C., personal commun., August 9, 2002): “Goals and objectives (all applicants). Comment on the serious-ness of the applicant’s dedication to the stated career goals and objectives. Comment on the student’s organizational, analytical, and written skills. Entering master’s student (applicants who at the time of submis-sion are applying for or enrolled in a master’s program and have completed less than 1 year toward this degree). Comment on the strength of the applicant’s planned course of study and probability of success of any proposed project. Entering doctoral student without another graduate degree (applicants who at the time of submission are applying for or are enrolled in a doctoral program, have completed less than 1 year toward this degree, and have no other graduate or other professional degree [MS, DVM, or JD]). Comment on the strength of the applicant’s planned course of study and probability of success of any proposed project. Beginning doctoral student with another graduate degree (ap-plicants who at the time of submission are applying for or enrolled in a doctoral program, have completed less than 1 year toward this degree, but have completed another graduate or professional degree [such as MS, DVM, MD]). Comment on how the applicant’s proposed doctoral program builds on his/her previous education or research projects. Why and how will any proposed research project advance the applicant’s academic or career goals? Continuing doctoral student (applicants who at the time of sub-mission are enrolled in a doctoral program and have completed more than 1 year, but less than 4 years toward this degree). Comment on the applicant’s research project as to its technical and social application, potential for success, and benefits expected.” During the panel meeting, only applications that have been rated excellent or very good by the principal reviewers typically are discussed, al

OCR for page 20
though a principal reviewer can ask that a particular application be discussed even if it has not received a sufficiently high ranking by all three principal reviewers. After the panel reaches agreement on the ranking of an application, a panel summary is prepared that contains a summary of the review comments and panel ranking; the summary is provided to the applicant. Typically, more applications are ranked excellent by the peer-review panel than EPA can afford to fund. The final decision about which of the “excellent” applications receive funding is made by the NCER staff according to such criteria as achieving a balance of fellowships among universities, filling identified shortfalls in particular disciplines, achieving a rough proportion among disciplines between the number of “excellent” applications and the number of fellowships awarded, and emphasizing applications in disciplines that EPA considers particularly important to fulfilling its science mission. The emphasis is on disciplines rather than on the specific research that applicants intend to undertake and whether it is relevant to EPA’s mission. The entire review process is completed by March. Applicants are notified and first-year funding provided to successful applicants in June. The fellowship awards are for 2 years (for master’s candidates) or 3 years (for Ph.D. candidates). EPA issues each successful applicant a letter committing the agency to the full term of funding contingent on the applicant’s remaining a student in good standing as demonstrated by an annual report provided by the student’s academic institution and academic adviser. Fellowships may be terminated if the EPA project officer determines that the fellow is not performing up to the standards of the program. PREVIOUS EVALUATIONS OF THE PROGRAM Given its relatively short life, the STAR program has been subject to an unusual number of reviews and evaluations by advisory committees and other internal and external groups (Table 2-7). The present review by a committee of the National Research Council is the most recent, and the committee has been informed that the EPA Office of the Inspector General is in the process of, or at least considering, undertaking another review (Harris 2002). NCER invited some of the reviews (such as this one) to obtain independent evaluations of its success in implementing an effective research program and suggestions on how the program could be improved. Others have been imposed on the organization.

OCR for page 20
TABLE 2-7 Summary of Reviews of the STAR Program Year Author Title of Report 1998 Board of Scientific Counselors (BOSC) of EPA’s ORD Program Review of the National Center for Environmental Research and Quality Assurance (NCERQA). Final Report of the Ad Hoc Subcommittee on the Review of NCERQA (EPA/BOSC 1998). 2000 EPA’s Science Advisory Board and ORD’s BOSC A Joint SAB/BOSC Report: Review of the Science to Achieve Results (STAR) Program (EPA/SAB/BOSC 2000). 2000 General Accounting Office (U.S. Congress) Environmental Research: STAR Grants Focus on Agency Priorities, but Management Enhancements are Possible (GAO 2000). 2001 EPA’s Science Advisory Board The Science to Achieve Results (STAR) Water and Watersheds Grants Program: An EPA Science Advisory Board Review (EPA/SAB 2001b). 2001 EPA and National Science Foundation Evaluation Report: A Decision Making and Valuation for Environmental Policy Interim Assessment (EPA/NSF 2000). 2002 EPA’s Science Advisory Board Interim Review of the Particulate Matter (PM) Research Centers of USEPA: An SAB Report (EPA/SAB 2002a). 2002 ORD’s BOSC Program Review of NCER (EPA/BOSC 2002). ? EPA’s Office of the Inspector General EPA’s Office of the Inspector General is considering conducting another review of the STAR program (Harris 2002).

OCR for page 20
The recommendations from those reviews and EPA’s initial responses to them are summarized in Appendix B. Almost all the reviews have been generally supportive of the program and have focused on how it might be even more effective. The STAR program appears to have implemented many of the recommendations, and these recommendations have contributed to the modifications of the program described in this chapter. Although the committee believes that objective, independent reviews of government programs can have substantial benefits, it also recognizes that such reviews can be expensive in the resources devoted to them and in the disruption that they can cause in the organization being reviewed. At the very least, anyone considering another review should carefully consider the results of previous efforts to ensure that the new review will truly be valuable. CONCLUSIONS AND RECOMMENDATIONS • The STAR program is a crucial element of EPA’s research efforts. It allows the agency to fill information gaps that are not addressed completely by its intramural program and to respond to new issues that EPA laboratories are not able to address. In addition to those primary purposes of the program, it provides the agency access to independent information, analyses, and perspectives. It helps to maintain environmental research and analysis capabilities in many of the nation’s academic and nonprofit research institutions. Finally, the program provides for the education of future leaders in environmental science and engineering. For these reasons, the STAR program should continue to be an important part of EPA’s research program. • As the STAR program has evolved, it has developed a grant-award process that in many ways exceeds those in place at other organizations that have extramural research programs. The agency has an aggressive planning process to identify the specific research that should be supported. The scientific peer-review process has been well established, and the proper mechanisms are in place to avoid conflicts of interest and to ensure independent reviews. The agency also puts an unusual amount of effort into preparing research solicitations and funding projects that have high relevance to its mission and program needs. • As the STAR program has developed, it has been able to induce other agencies with similar interests to enter into partnerships and provide supplementary funds. STAR should continue to partner with other govern

OCR for page 20
ment and nongovernment organizations to support research of mutual interest and of relevance to EPA’s mission, explore innovative approaches for carrying out this research, and sponsor a diverse portfolio of research that alerts the agency to emerging issues and provides independent analyses of issues that the agency is currently addressing. • When projects are under way, the STAR program actively monitors their progress and coordinates the efforts of the independent researchers with one another as appropriate and with the conduct of related research by EPA staff. • The STAR program has begun to emphasize communication of research results to potential users and has taken some remarkably aggressive steps to promote it. REFERENCES Bradley, P. 2002. Mid-Atlantic Integrated Assessment (MAIA) Pilot Program Overview. Presentation at the First Meeting on Review of EPA’s Research Grants Program, March 19, 2002, Washington, DC. Bryan, E. 2002. Peer Review Process for EPA’s National Center for Environmen-tal Research’s Science to Achieve Results Program. Presentation at the Third Meeting on Review of EPA’s Research Grants Program, June 6, 2002, Wash-ington, DC. College Board. 2001. Trends in College Pricing 2001. College Board Publica-tions, New York. EPA (U.S. Environmental Protection Agency). 1996a. Report to Congress. The Science to Achieve Results (STAR) Program. EPA/600/R-96/064. Office of Research and Development, U.S. Environmental Protection Agency, Washing-ton, DC [Online]. Available: http://es.epa.gov/ncer/publications/archive/ reportcong.html [accessed Jan. 13, 2003]. EPA (U.S. Environmental Protection Agency). 1996b. Research Opportunities. U.S. Environmental Protection Agency Announces the Availability of 1996 Grants for Research on Endocrine Disruptors Role of Interindividual Variation in Human Susceptibility to Cancer RiskBased Decisions for Contaminated Sediments. National Center for Environmental Research, Office of Research and Development, U.S. Environmental Protection Agency [Online]. Available: http://es.epa.gov/ncer/rfa/archive/grants/96/rfa2.html [accessed Jan. 13, 2003]. EPA (U.S. Environmental Protection Agency). 1997. Ecosystem Indicators. Research Opportunities. National Center for Environmental Research, Office of Research and Development, U.S. Environmental Protection Agency [On-line]. Available: http://es.epa.gov/ncer/rfa/archive/grants/97/ecosystem.html [accessed Jan. 13, 2003].

OCR for page 20
EPA (U.S. Environmental Protection Agency). 1998. Ecological Research Strat-egy. EPA/600/R-98/086. Office of Research and Development, U.S. Environ-mental Protection Agency, Washington, DC [Online]. Available: http://www.epa.gov/ordntrnt/ORD/WebPubs/final/eco.pdf [accessed Jan. 13, 2003]. EPA (U.S. Environmental Protection Agency). 1999. 1999 STAR Grants for Research: Ecological Indicators. Research Opportunities. National Center for Environmental Research, Office of Research and Development, U.S. Environ-mental Protection Agency [Online] . Available: http://es.epa.gov/ncer/rfa/ archive/grants/99/batch.html [accessed Jan. 13, 2003]. EPA (U.S. Environmental Protection Agency). 2001. Office of Research and Development Strategic Plan. Office of Research and Development, U.S. Environmental Protection Agency [Online]. Available: http://www.epa.gov/ ospinter/strtplan/documents/final.pdf [accessed Jan. 13 2003]. EPA (U.S. Environmental Protection Agency). 2002a. Welcome to EPA’s Na-tional Center for Environmental Research Homepage. National Center for Environmental Research, Office of Research and Development, U.S. Environ-mental Protection Agency [Online]. Available: http://es.epa.gov/ncer/about/ [accessed Jan. 13, 2003]. EPA (U.S. Environmental Protection Agency). 2002b. Airborne Particulate Matter (PM) Research Centers. National Center for Environmental Research, Office of Research and Development, U.S. Environmental Protection Agency [On-line]. Available: http://es.epa.gov/ncer/centers/airpm/ [accessed Jan. 13, 2003]. EPA (U.S. Environmental Protection Agency). 2002c. STAR Reports. National Center for Environmental Research, Office of Research and Development, U.S. Environmental Protection Agency [Online]. Available: http://es.epa.gov/ ncer/publications/starreport/index.html [accessed Jan. 13, 2003]. EPA (U.S. Environmental Protection Agency). 2002d. STAR Research Capsules by Topic. National Center for Environmental Research, Office of Research and Development, U.S. Environmental Protection Agency [Online]. Available: http://es.epa.gov/ncer/publications/topical/topic.html [accessed Jan. 13, 2003]. EPA (U.S. Environmental Protection Agency). 2002e. Progress-Review Workshop Proceedings. National Center for Environmental Research, Office of Research and Development, U.S. Environmental Protection Agency [Online]. Available: http://es.epa.gov/ncer/publications/workshop/ [accessed Jan. 13, 2003]. EPA (U.S. Environmental Protection Agency). 2002f. Fall 2002 Science to Achieve Results Fellowships for Graduate Environmental Study. 2002 RFA. National Center for Environmental Research, Officer of Research and Devel-opment, U.S. Environmental Protection Agency. EPA (U.S. Environmental Protection Agency). 2003a. ORD Lab & Office Loca-tions. Office of Research and Development, U.S. Environmental Protection Agency [Online]. Available: http://www.epa.gov/ord/htm/map.htm [accessed Jan. 13, 2003].

OCR for page 20
EPA (U.S. Environmental Protection Agency). 2003b. Research Opportunities -Environmental Research Grant Announcements. National Center for Environ-mental Research, Office of Research and Development, U.S. Environmental Protection Agency [Online]. Available: http://es.epa.gov/ncer/rfa/ [accessed Jan. 13, 2003]. EPA (U.S. Environmental Protection Agency). 2003c. STAR Fellowships. Na-tional Center for Environmental Research, Office of Research and Develop-ment, U.S. Environmental Protection Agency [Online]. Available: http:// es.epa.gov/ncer/fellow/ [accessed Jan. 13, 2003]. EPA/BOSC (U.S. Environmental Protection Agency Board of Scientific Counsel-ors). 1998. Program Review of the National Center for Environmental Re-search and Quality Assurance (NCERQA). Final Report of the Ad Hoc Sub-committee on the Review of NCERQA. Board of Scientific Counselors, Office of Research and Development, U.S. Environmental Protection Agency, Wash-ington, DC. April 30, 1998. EPA/BOSC (U.S. Environmental Protection Agency Board of Scientific Counsel-ors). 2002. Program Review of NCER. Board of Scientific Counselors, Of-fice of Research and Development, U.S. Environmental Protection Agency, Washington, DC. October 25, 2002. EPA/NSF (U.S. Environmental Protection Agency and National Science Founda-tion). 2000. Interim Assessment for the Decision Making and Valuation for Environmental Policy Grants Program. Final Report. Prepared for National Science Foundation and U.S. Environmental Protection Agency, by Aspen Systems Corporation. April 17, 2000. EPA/SAB (U.S. Environmental Protection Agency Science Advisory Board). 2001a. Water Quality and Pollution Prevention Multiyear Plans: An SAB Review. A Review by the Research Strategies Advisory Committee (RSAC) of the EPA Science Advisory Board (SAB). EPA-SAB-RSAC-02-003. Sci-ence Advisory Board, U.S. Environmental Protection Agency, Washington, DC [Online]. Available: http://www.epa.gov/science1/fiscal02.htm [accessed Jan. 13, 2003]. EPA/SAB (U.S. Environmental Protection Agency Science Advisory Board). 2001b. The Science to Achieve Results (STAR) Water and Watersheds Grants Program: An EPA Science Advisory Board Review. A Review by the Ecolog-ical Processes and Effects Committee (EPEC) of the EPA Science Advisory Board. EPA-SAB-EPEC-02-001. Science Advisory Board, U.S. Environmen-tal Protection Agency, Washington, DC [Online]. Available: http://www.epa. gov/science1/fiscal02.htm [accessed Jan. 13, 2003]. EPA/SAB (U.S. Environmental Protection Agency Science Advisory Board). 2002. Interim Review of the Particulate Matter (PM) Research Centers of the USEPA: An EPA Science Advisory Report. A Review by the PM Research Centers Interim Review Panel of the Executive Committee of the U.S. EPA Science Advisory Board (SAB). EPA-SAB-EC-02-008. Science Advisory Board, U.S. Environmental Protection Agency, Washington, DC. May 2002

OCR for page 20
[Online]. Available: http://www.epa.gov/science1/fiscal02.htm [accessed Jan. 13, 2003]. EPA/SAB/BOSC (U.S. Environmental Protection Agency Science Advisory Board and Board of Scientific Counselors). 2000. A Joint SAB/BOSC Report: Review of the Science to Achieve Results (STAR) Program. EPA-SAB-EC-00-008. Science Advisory Board, Board of Scientific Counselors, U.S. Envi-ronmental Protection Agency. [Online]. Available: http://www.epa.gov/ sab/pdf/ec0008.pdf [accessed Jan. 13, 2003]. GAO (U.S. General Accounting Office). 2000. Environmental Research: STAR Grants Focus on Agency Priorities, But Management Enhancements Are Possi-ble: Report to the Chairman, Subcommittee on VA, HUD, and Independent Agencies, Committee on Appropriations, House of Representatives. GAO/RCED-00-170/B-142370. U.S. General Accounting Office, Washington, DC. Harris, J.K. 2002. Use of Science to Achieve Results (STAR) Grants in Achieving EPA’s Strategic Goals. Memorandum to H.L. Longest II, Acting Assistant Administrator for Research and Development, from J.K. Harris, Director for Cross-Media Issues, Office of Program Evaluation, Office of the Inspector General, Washington, DC. March 13, 2002. Hogue, C. 2002. Graduate program cut from budget. Chem. Eng. News 80(17): 22. Johnson, J. 1996. Rebuilding EPA science. Environ. Sci. Technol. 30(11):492A-497A. NCSE (National Council for Science and the Environment). 2003. Description of the EPA STAR Fellowship Program. National Council for Science and the Environment, Washington, DC [Online]. Available: http://www.cnie.org/ NCSE/SciencePolicy/? [accessed Jan. 13, 2003]. NRC (National Research Council). 1998. Research Priorities for Airborne Particu-late Matter. 1. Immediate Priorities and a Long-Range Research Portfolio. Washington, DC: National Academy Press. Preuss, P.W. 2002a. National Center for Environmental Research, History, Goals, and Operation of the STAR Program. Presentation at the First Meeting on Review of EPA’s Research Grant Program, March 18, 2002, Washington, DC. Preuss, P.W. 2002b. Response to NAS Committee Queries. Presentation at the Third Meeting on Review of EPA’s Research Grant Program, June 6, 2002, Washington, DC.