A Review of the Environmental Protection Agency's Science to Achieve Results Research Program (2017)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Summary Environmental research has driven landmark improvements that led to the protection of human and ecosystem health. Recognizing the value of knowledge generated by environmental research and the ingenuity within academic and nonprofit institutions, the US Environmental Protection Agency (EPA) created a program known as Science to Achieve Results, or STAR, in 1995. STAR is EPA’s primary competitive extramural grants program. This report shows that through STAR, EPA has created a vehicle that fosters collaboration and knowledge-sharing, which have produced research that has supported interven- tions that may reduce the cost of regulations, protect public health, and save lives. STAR is managed by EPA’s National Center for Environmental Research and integrated into the Officer of Research and Development’s (ORD’s) overall research program through planning and coordination with EPA’s laboratories, offices, and centers. STAR research support consists of three main types: grants to individual investigators; larger multidisciplinary center grants, usually to groups of institutions; and a recently discontinued fellowship program, which supported master’s and doctoral students. In 2003, a National Research Council committee reviewed STAR and strongly endorsed it as an integral part of EPA’s research program. That com- mittee believed that STAR provided the agency access to external and inde- pendent information, analyses, and perspectives. However, the STAR program was too young to fully evaluate its effects. Since that review, there have been changes in the program. For example, funding has fluctuated, with a peak of around $138 million (2016 dollars) in 2001 and 2002, which represented 18% of ORD’s total budget; a median of $75 million (2016 dollars) in 2007, which represented 12% of ORD’s total budget; and a minimum of $39 million in 2016, which represented 8% of ORD’s total budget. Funding for the STAR fellowship program was eliminated for FY2016 to centralize graduate fellowships in the National Science Foundation (NSF). THE COMMITTEE’S REVIEW EPA asked the National Academies of Sciences, Engineering, and Medi- cine to conduct an independent assessment of the STAR program. The commit- tee established in response to the request was charged with assessing the pro- 1 

2 A Review of the EPA’s Science to Achieve Results Research Program gram’s scientific merit, public benefits, and overall contributions in the context of other relevant research and with recommending ways to enhance those as- pects of the program. The committee was also asked to consider the conclusions and recom- mendations of the prior National Research Council review of the STAR program (2003), the STAR program’s research priorities in light of the nation’s environ- mental challenges, and the effects of recent STAR funding trends on obtaining scientific information needed to protect public health and the environment.1 The committee’s approach is detailed in Box S-1. SCIENTIFIC MERIT The committee compared STAR’s procedures for priority setting, solicit- ing, awarding, and administering grants, with those of research programs of the California Air Resources Board, the Health Effects Institute, the National Oce- anic and Atmospheric Administration’s National Centers for Coastal Ocean Science, the US Department of Agriculture’s National Institute of Food and Ag- riculture, the NSF Division of Earth Sciences, and the National Institute of En- vironmental Health Sciences (NIEHS) Division of Extramural Research and Training (Chapter 2). BOX S-1 The Committee’s Approach to the Evaluation • To assess whether STAR procedures sponsor research of high scientific merit the committee compared STAR’s operating procedures with those of other extramural research programs that fund research in similar fields and read all STAR requests for applications released in 2003-2015 (see Chapter 2). • To assess the program’s public benefits, the committee created a logic model and then evaluated the program at various points along the model (see Chapter 3). • To assess STAR’s research priorities in light of the nation’s environ- mental challenges, the committee considered whether STAR research supports scientific fields that are important for addressing the challenges. It also considered how STAR has been used to address the nation’s changing environmental priorities (see Chapter 4). 1 According to the president’s budget blueprint submitted to Congress on March 16, 2017, “ORD would prioritize activities that support decision-making related to core en- vironmental statutory requirements, as opposed to extramural activities, such as providing STAR grants.” 

Summary 3 The committee found no major deficiencies in STAR’s procedures. STAR priority-setting procedures are integrated within four of ORD’s national pro- grams; this allows STAR to be flexible in light of the nation’s changing research priorities and avoids duplication of EPA’s internal research. STAR was the only research program included in this comparison that allowed neither submission of research topic ideas by the general public nor unsolicited proposals; this charac- teristic may limit the creativity of the program and merits consideration. Having reviewed the RFAs, the committee noted that the STAR program’s RFAs are generally of good quality and address a wide variety of topics. STAR has strong peer-review procedures, and it is a highly competitive program, with a median grant application award rate of 16% in 2003-2014. After peer review, EPA staff review grant applications for relevance to the intent of the RFA; it is unclear whether investigators receive comments on their applications’ relevance reviews. • Finding 1. EPA has high-quality procedures for priority-setting that allow STAR to be integrated within EPA’s research program. Ø Recommendation 1. EPA should continue to use its procedures for strategic planning and for setting priorities for STAR re- search. However, EPA should consider developing a mecha- nism to allow for public input to the STAR research agenda or the submission of unsolicited proposals. • Finding 2: STAR’s procedures to develop funding announcements and award grants ensure that the program sponsors research of high scientific merit. Ø Recommendation 2. The STAR program should maintain the procedures that it has in place. However, it should provide comments to applicants whose applications were not awarded because of lack of relevance so that they can improve their ability to prepare future grant proposals. PUBLIC BENEFITS The STAR program is productive. In 2003-2015, STAR awarded 541 in- dividual-investigator grants, 53 center grants, and 800 fellowships (Chapter 1). In October 2002-April 2017, there were 5,760 STAR journal publications (Chapter 3). The committee found that results of STAR-funded research are used by many different kinds of organizations, for example, in federal, state, and local government documents; in international guidelines; and in other docu- ments of academic or nonprofit organizations, such as National Research Coun- cil reports and American Public Health Association guidelines. In 2012, at least 104 STAR-funded papers were cited in those types of documents. The commit- tee found that those outputs and outcomes have led to numerous public benefits (Chapter 3). Some examples are the development of an environmental-science 

4 A Review of the EPA’s Science to Achieve Results Research Program workforce, the development of human-resources and research infrastructure across the nation, a potential reduction in the costs of compliance with environ- mental regulation, provision of the scientific basis of decisions required to pro- tect public health and the environment, and the study of methods for improving environmental management. Support of Public-Health Decisions STAR research has supported numerous public-health decisions. The STAR program implemented several large initiatives that address the human health effects of air pollution, such as the Particulate Matter Centers, the Clean Air Research Centers, and the Air, Climate, and Energy Centers. Studies sup- ported by the centers showed that increased air-pollution exposure leads to a decrease in life expectancy; they include a followup of the Harvard Six Cities Study published in 2006 and a large epidemiologic study of PM2.5 exposure and mortality in 51 US cities published in 2009. Those findings supported earlier research and led to the development of a more scientifically justified PM2.5 na- tional ambient air quality standard (NAAQS) which may have saved lives and reduced healthcare costs nationwide. Another effective initiative is the Children’s Environmental Health and Disease Prevention Research Centers, which are supported by STAR center grants. The grants are funded in partnership with the National Institute of Envi- ronmental Health Sciences (NIEHS) and aim to evaluate the effects of environ- mental exposures on child health and development. In 2016, a research project partially supported by a STAR grant found that infants could be exposed to ar- senic through rice cereal, and this led the Food and Drug Administration to pro- pose regulations to protect infant health. Another example is the discovery by the University of Washington Children’s Center that farmworker children had increased exposure to the pesticide ingredient azinphos-methyl which is a neu- rotoxin; this finding informed EPA’s decision to phase out the use of the az- inphos-methyl. Examples of STAR research to improve environmental management in- clude experiments in market-based incentives to lower emissions and studies that evaluated the potential reduction in the cost of pollution abatement and auc- tions in which landowners and land sellers compete to obtain part of a fixed budget allocated by the regulator to subsidize pollution abatement. Potentially Reducing the Cost of Compliance with Regulation STAR research has led to potential reductions in the cost of complying with environmental regulations. The reductions would benefit regulated indus- tries and states and localities that need to comply with environmental regula- tions. An example of STAR research that may benefit industry is the develop- ment of a tissue-based method for evaluating the thyroid effects of chemical exposures. The method may substantially reduce the costs of chemical testing 

Summary 5 compared with animal-based approaches. STAR research has also expanded the capability of climate and air-pollution models, and this may reduce the costs of compliance with NAAQSs. Another research project supported by STAR dis- covered a cost-effective way to remove nitrate from drinking water. Workforce Development In 2003-2015, STAR awarded 800 graduate fellowships. Many former STAR fellows continued in environmental and environmental health sciences careers. Among former STAR fellows who reported on their careers’ trajectory to EPA, 34% were in postdoctoral positions; 21% in teaching positions; 16% in research; 12% in the federal government; 5% in consulting firms; 4% in state, local, or tribal government; 4% in private industry; and 3% in nonprofits. The committee also found evidence that STAR fellows produced high-quality sci- ence; for example, a search for frequently cited STAR publications in Google Scholar found that about one-fourth were at least partially supported by a STAR fellowship. Infrastructure Development In FY 2014, the STAR program had grantees or fellows in all but two states (Vermont and South Dakota) (Figure S-1). Engagement with EPA in in- stitutions throughout the United States has created communities of scientists and engineers working in the human health and environmental sciences that might not have existed without support from STAR grants. In addition, research grants help to improve facilities for data collection and analysis within the supported grantees’ institutions. Tracking of Public Benefits of STAR Research Tracking of the public benefits of research is difficult; all research pro- grams struggle with tracking public benefits and attributing them to single re- search projects. One issue that made it difficult for STAR is that the EPA grant- ee project-results Web site was not up to date. There were many examples of grants long completed or at least in operation for a number of years on which annual or final reports were unavailable (see Chapter 2). STAR has made efforts to translate research to a broader audience and synthesize information on a given RFA topic by having investigators from dif- ferent grantee institutions collaborate on summary reports, but the efforts have been inconsistent (see Chapter 3). The overall benefits of the science could be strengthened if grantees consistently created synthesis reports and held more public webinars to discuss research. 

6 Alaska Hawaii FIGURE S-1 Geographic distribution of STAR grants, centers, and fellowships in FY 2014. Yellow indicates 1-5, green 6-10, purple 11-15, blue 16-20, and red over 20 active grantees in a state in 2014. In FY 2014, there were a total of 506 active STAR individual-researcher grants, centers, and fellowships. 

Summary 7 The committee acknowledges that many other research programs struggle with such challenges (see Chapter 3). Evaluations like the present one would be improved if there were more robust electronic databases that could be easily searched to detect linkages between grants, fellowships, and public benefits. There have been advances throughout the federal government to mine existing data in reports, academic literature, administrative records, and so forth, to iden- tify intermediate outcomes more effectively, to link federally funded projects to long-term effects, and to track career outcomes of graduate students supported by fellowships or graduate research assistantships. The National Institutes of Health (NIH), for example, has created the High Impacts Tracking System. The system loads progress reports and program officers’ notes about grants into a searchable system and allows structured tagging of outputs and effects. Another NIH example is RePARS, which permits automatic retrieval of sources of NIH funding of publications in any list, such as the bibliography of a National Academies report. Those efforts have recently been used to evaluate the Nation- al Toxicology Program’s effects on a water-quality standard for hexavalent chromium in California. EPA could make strides in this regard by collaborating with other organizations that are linking public benefits to research. EPA would benefit from working with other federal agencies that are ad- vancing ways in which such benefits are communicated to the public. NIH has found that the links between research studies and benefits to human health are described best in stories or case studies that resonate with those outside the re- search community. EPA should consider reporting stories more prominently on its Web site and blogs. STAR should also consider requiring grantees to report the potential influence and public benefits of their awards as part of their final reports and even 5-10 years after their research has been completed. The Fellowship Program As discussed previously, the STAR fellowship program supported stu- dents who continued careers in environmental and environmental health scienc- es. The STAR fellowship program was distinctive in that it covered both envi- ronmental and environmental health research. The two other agencies that support predoctoral fellows will not fill this gap: NSF training programs do not cover environmental health effects, and NIH training programs are geared to- ward overall health sciences. In addition, it appears that the move to centralize graduate fellowships in NSF has led to a large reduction in the support of stu- dents interested in environmental research. In 2015, there were 168 NSF fellows in environmental sciences and ecologic research and 51 STAR fellows. In 2017, after the STAR fellowship program was terminated, there were 176 NSF fellows in environmental sciences and ecologic research; thus, there are indeed fewer fellowships in environmental and environmental health sciences. The need for federally supported fellowship programs in the environmental arena is important in that the United States is projected to have considerable human-resources needs in the science and engineering policy fields. 

8 A Review of the EPA’s Science to Achieve Results Research Program • Finding 3. The STAR program has generated research that has many public benefits. However, these public benefits are not con- sistently tracked and synthesized. Ø Recommendation 3. The STAR program should partner with other federal agency efforts to improve communication of the benefits of its research to the public. In addition, EPA should update the grantee project results Web site. • Finding 4. The STAR fellowship program was critical for training future generations of scientists who pursue environmental careers. Ø Recommendation 4. The STAR fellowship program should be restored to EPA given the continued and growing need for sci- entists in environmental research and management. ADDRESSING EPA’S PRIORITY SCIENTIFIC QUESTIONS Does the STAR program contribute to fields that will help to improve hu- man health and the environment? To answer that question, the committee first considered what scientific disciplines and fields of study are needed to produce knowledge and capacity to protect human health and the environment. It then considered how STAR has engaged the various disciplines, which range from basic sciences—such as the earth sciences, atmospheric sciences, life sciences, ecology, and toxicology—to applied domains, such as environmental engineer- ing, sustainable energy, human exposure and health effects, and human behav- ioral studies. The committee categorized the RFAs released by STAR in 2003-2015 and the STAR research papers that it identified as having been cited more than 100 times in a Google Scholar search according to the fields of knowledge that will help to improve human health and the environment. Through its assessment, the committee found that STAR supports work in almost every field identified that contributes to environmental knowledge and capacity. The most common fields identified were the atmospheric sciences, climate sciences, ecology, environmental economics, environmental engineer- ing, human exposure and health effects, risk analysis, systems modeling and decision support, and innovative risk management. Many other federal research programs support scientific study in those fields. What distinguishes STAR from the other programs is not specifically the research topics that it supports but that its RFAs cover subjects that are important to EPA’s mission and that it address- es knowledge gaps which will protect human health and the environment. Ex- amples of how EPA has used STAR to address knowledge gaps or to respond strategically to emerging challenges are numerous (Chapter 4). The committee found that STAR has been called on to address human health and environmental concerns related to new technology, to address knowledge gaps identified in connection with environmental disasters, and to evaluate potential consequences of resource-conservation technologies. Some recent examples are the release of 

Summary 9 RFAs that cover the health effects of engineered nanoparticles, the environmen- tal effects and mitigation of oil spills after the Deepwater Horizon incident, and human and ecologic effects associated with water reuse and conservation prac- tice (Chapter 4 and Appendix C). The ability of EPA to use STAR to strategically address knowledge gaps has weakened in recent years; STAR has not had the ability to release as many RFAs. In 2003, it released 12 individual-investigator grant RFAs and one center RFA. In 2013 and 2014, it released five individual-investigator RFAs and two center RFAs a year. In 2015, it released only one individual-investigator RFA. The change limits the number of topics in which the STAR program is investing. • Finding 5. STAR plays a distinctive role in the nation’s overall en- vironmental-research portfolio. Ø Recommendation 5. The committee recommends that EPA continue to use STAR to respond to the nation’s emerging en- vironmental challenges. CONCLUSIONS Environmental research has led to technologic advances and to policies that have resulted in enormous improvements in human health and the environ- ment. However, many persistent environmental challenges remain, and complex challenges with unknown effects on human health and the environment are emerging. For example, increased energy demands have led to advanced ap- proaches to oil and gas extraction that have unknown environmental effects. Increasing urbanization has led to changes in land-use patterns, which may have adverse effects on the quality of air, land, and water and on human health. Ag- riculture and food production change as technology advances. Environmental research supplies the critical knowledge needed to address such challenges. The committee found that STAR has been integral to EPA’s efforts to address evolving environmental research priorities and that these efforts have benefited the public. The committee recommends that EPA continue to use the STAR program to address our nation’s evolving environmental research priorities.