“…EPSCoR is critically important and it is important not just because of its basic function of bolstering research and supporting graduate education across the nation but also because of the values it represents.”
John Holdren, Assistant to the President for Science and Technology, Director of White House Office of Science and Technology Policy, and Co-Chair of the President’s Council of Advisors on Science and Technology (PCAST), in EPSCoR 2030: A Report to the National Science Foundation (Arlington, VA: NSF, www.nsf.gov/od/oia/programs/epscor/2030%20Report.pdf, 2012).
“Smart people are everywhere.”
Ann Zulkosky, member of the professional staff at the U.S. Senate Subcommittee on Science and Space, First Meeting of the NAS EPSCoR Evolution Meeting, May 24–25, 2012.
Congressional demands for more equitable geographic distribution of federal research spending led to the creation of the first Experimental Program to Stimulate Competitive Research (EPSCoR) program. Aiming to expand opportunities for less advantaged researchers without compromising meritocratic principles, EPSCoR’s designers envisioned a short-term initiative that would improve the research competence of eligible states and enable them to successfully compete for federal dollars.
Today’s EPSCoR is much different than the EPSCoR of 30, 20, or even 10 years ago. Changes in the program’s membership and focus reflect changes both in stakeholder needs and in the broader research environment. The definition of success has shifted to focus largely on institutional rather than statewide research capabilities. All of this has caused EPSCoR to drift from its original mandate while pursuing goals that are in line with its core
responsibilities. This assessment of EPSCoR has sought to evaluate the program in terms of its original mandate as well as in its accomplishments that lie both within and beyond what it was initially designed to achieve.
FINDINGS AND RECOMMENDATIONS
The committee supports the continuation of programs that support the proposition stated in the America COMPETES Act:
“The Nation requires the talent, expertise, and research capabilities of all States in order to prepare sufficient numbers of scientists and engineers, remain globally competitive and support economic development.”
America COMPETES Reauthorization Act of 2010 (111th Congress, 2009–2010, April 22, 2010), http://www.govtrack.us/congress/bills/111/hr5116.
• The talent necessary to succeed in science and engineering resides in all states. Thus, it is in the national interest for the federal government to support efforts to develop and utilize this talent to enhance national research capacity.
• EPSCoR programs are a part of a broader national and global research enterprise.
• Congressional changes in state eligibility requirements and congressional mandates to agencies to create EPSCoR-like programs have resulted in multiple and often competing objectives and policy directives by participating agencies.
oCurrent eligibility criteria have led to more than half the states being included, blurring the programs’ objectives and reducing the likelihood of their success.
oPatterns of eligibility do not align well with other indicators of capacity, such as state population or number of research-intensive universities. As a result, outcomes are difficult to assess, especially on a comparative basis.
• EPSCoR programs have enhanced the nation’s human capital by strengthening research infrastructure and by training many future scientists and engineers in states where, in some cases, training opportunities had been scarce and largely inadequate prior to the program’s arrival.
• There is some evidence that the EPSCoR programs have not been a good fit for the mission agencies. For example, EPA and DOD terminated their EPSCoR programs. However, the mission agencies are the major
source of engineering research funding and therefore critical to engineering education.
• State-level commitments to enhancing research capacity are uneven across the participating states. The effectiveness of state committees in NSF EPSCoR states is also uneven.
• There is considerable variation in agency programs, review processes, and the role and composition of state committees. Further, the NIH IDeA program does not formally involve the state committee in its implementation, although informal interactions do occur.
• The aggregate share of federal R&D to eligible states has not changed significantly over the course of the program. There is also considerable variation among states in their progress toward a more competitive posture. In the aggregate, eligible states continue to be less successful in garnering NSF funding than are other states.
• Nearly all participating states report positive cultural change in attitudes toward science and engineering as a consequence, at least in part, of EPSCoR programs. Similarly, they also report positive organizational, policy, and program changes that have enhanced their research environment. Further, there is evidence that research capacity in eligible states has increased (although not enough in most cases to change their relative standings). There is anecdotal evidence that EPSCoR programs have contributed to this result, but the magnitude of their contribution is difficult to determine.
• The evaluation efforts of the EPSCoR-type programs leave much to be desired. To date, such efforts have relied on incomplete and inconsistent assessment of program designs and on metrics that do not allow for comparisons of effectiveness.
The committee recommends that the federal government continue to promote the development of research capacity in every state so that all citizens across the nation have the opportunity to acquire the postsecondary education, skills, and experience they need to pursue productive and successful careers in science, technology, engineering, and mathematics (STEM) fields and to contribute fully to the nation’s research enterprise.
With that in mind, the committee recommends the following actions to create a more focused program with greater impact.
• EPSCoR programs should concentrate on the programs’ core elements:
oTo enhance research excellence through competitive processes.
oTo enhance capacity for postsecondary training in STEM fields.
• EPSCoR programs should be restructured to combine beneficial aspects of current programs:
oThe NIH and NSF EPSCoR programs should pursue a “blended” funding strategy with two tracks:
A competitive-grant track that provides fewer and larger grants that are evaluated first for scientific merit and that are intended to produce focal points of research excellence and research opportunities for junior as well as senior faculty.
A smaller-scale, infrastructure investment or statewide investment track that works with state committees to ensure that every state has the capacity to provide advanced education and research experience.
oDOE, NASA, and USDA should develop strategies to help meet the mandate laid out in the America COMPETES Act that all mission agencies support postsecondary education in STEM disciplines.
• The EPSCoR programs, working through the EPSCoR Interagency Coordinating Committee (EICC), should develop and enforce a realistic framework for state eligibility and graduation from the program:
oThe 0.75 percent criterion fails to account for population and other critical aspects of research capacity and competitiveness. New graduation and eligibility criteria should be developed and implemented that could consider:
Proposal success rates per research-university faculty member.
Total research funding.
Progress to date and future opportunities for progress.
• The committee recommends that the agencies, cooperating through the EICC, reset the guidelines and that all states must reapply for eligibility after the expiration of their current EPSCoR grants.
• The proposal review for prospective EPSCoR projects should be made more rigorous to:
oEnsure that reviews of the scientific merit of the proposals are conducted by the most highly qualified panels of experts in the field of study. Scientific merit should be the first consideration in any assessment of a proposal’s strength and value. Specifically, all proposals should be reviewed in a two-step, sequential process.
First, a review of the proposal’s scientific merit—a “science score.”
Second, a review of the proposal’s potential (state, agency, societal) impacts—a “program score.”
oRequire some level of matching contribution for all research awards to ensure that the state is involved and committed to the project.
Sources dedicated as matching funds can be from the state, the university, the private sector, or other sources.
• The evaluation process conducted during and after an EPSCoR project’s implementation should be made more rigorous by:
oDeveloping and implementing an effective third-party evaluation design that is reliable and valid and that is consistent with other federal evaluation approaches, such as those developed by the Office of Management and Budget.
In conclusion, the committee recommends that the newly refocused federal programs be renamed to better reflect their mission and to remove “experimental,” which is now a misnomer.