A system as vast and complex as the biomedical research enterprise requires careful and evidence-based policy solutions. Without a culture of innovation, experimentation, and rigorous assessment, institutions will feel less comfortable adopting policy recommendations, the recommendations that are adopted may be less effective or produce unforeseen consequences, and even the successful ones may not be broadly adopted by institutions over time. And yet, the committee was surprised by the small number of publicly reported, evidence-based studies of the many ideas to address the issues confronting the nation’s young investigators, as well as the lack of a site analogous to clinicaltrials.gov to register experiments and pilots of the type mentioned throughout this report. If the nation hopes to design and sustain effective policies to support the next generation of investigators, we need to hardwire into the biomedical research enterprise1 a greater capacity for experimentation in policy and funding changes, followed by assessment, publication, and adaptation or replication if successful. This is, in essence, how science itself proceeds.
Indeed, many of the responses to the committee’s Dear Colleague Letter (see Appendix C) expressed a desire for more experimentation and assessment with public disclosure on approaches to address the challenges identified in this report. For example, the Public Affairs Advisory Committee of the American Society for Biochemistry and Molecular Biology noted that concrete data on the effect of supporting more staff scientists are not readily available and that “understanding the true financial and productivity effects of employing staff scientists would
1 In this report, “biomedical” refers to the full range of biological, biomedical, behavioral, and health sciences supported by the National Institutes of Health.
guide the community on how best to incorporate staff scientists” into their research enterprise. The Association of American Medical Colleges urged that any changes to the scope of grant award or review “be based on ample data, and as possible, pilot testing, given the potential for unintended consequences for any system wide changes.”
In recent years, the National Institutes of Health (NIH) has established initiatives that encourage impact assessments and examine programs or policies to support the biomedical workforce. For example, the NIH Common Fund funded a Coordination and Evaluation Center (CEC) to undertake longitudinal, cross-institution assessments of diversity programs such as the National Research Mentoring Network (NRMN) and the Building Infrastructure Leading to Diversity (BUILD) initiative (Estrada et al., 2016; National Academy of Sciences et al., 2011). NIH has also undertaken comprehensive evaluations of the Director’s New Innovator Award (DP2) (Tinkle et al., 2016), the Director’s Pioneer Award (DP1), mentored career awards, and the Institutional Research and Academic Career Development Award (IRACDA) program (K12), among others. In some cases, NIH has undertaken experimental designs to test the effect of policy changes, as is currently being done to examine anonymized peer review and implicit bias modules. For example, the NIH Scientific Workforce Diversity Office has launched an anonymized review to examine biases in peer review. That office is also assessing the efficacy of implicit bias modules and investments in research on workforce diversity. Assessments are also planned for other pilot projects for career development and mentoring, notably the Broadening Experiences in Scientific Training (BEST) program. In addition, the NIH Next Generation Researchers Initiative will be accompanied by a new effort to “encourage independent analyses of metrics that can be used to assess the impact of the NIH portfolio.”
Too often, however, there is an absence of published, evidence-based studies that demonstrate the effectiveness of potential reforms on the issues that will affect the next generation of investigators. Even within NIH some successful experiments have not been considered for broader implementation, because no formal mechanism for “registering” experiments and sharing analyses exists.
This is not to say that all attempts to improve the system are equally amenable to experimentation, and if every policy change were held to the scientific method, improvements would be slow in coming. Even so, the shortage of experimentation, analyses, and sharing slows the development of much needed changes in policy that will break down the barriers confronting today’s young scientists and those of future generations.
These evidence-based studies cannot be expected to emerge on their own; their conduct will require institutions, norms, and conditions that are conducive to innovation and experimentation. Fortunately, much of the institutional bedrock for such an approach already exists in the form of the embedded research capacity of our nation’s graduate research programs and in the individual NIH ICs. In the recommendations that follow, the committee proposes to harness these institu-
tions to serve as “policy laboratories” whose experiences can be used to benefit the next generation of researchers.
Congress and the National Institutes of Health should promote innovative pilot projects on the part of research institutions and other stakeholders that seek to improve and accelerate transitions into independent careers. A Next Generation Researcher Innovation Fund should be created to support these experimental projects.
Universities and other research institutions have been slow to adopt systemic change in how they train and support the next generation of researchers for several reasons, including a paucity of evidence that the recommended reforms of the past are effective and workable and the absence of ready resources to experiment with innovative approaches to the operation of the research enterprise. NIH has recognized the need to step in and support experimentation at the institutional level in certain areas. The BEST program, for example, supports 17 universities across the country to implement career development programs to improve access to a range of biomedical career options. The NRMN seeks to fund, implement, and evaluate innovative university approaches to research training and mentoring practices for individuals from diverse backgrounds.
These programs are laudable, but integration of sustained change into the biomedical research enterprise requires a more consistent effort to support regular experimentation and innovation that consider the full range of barriers affecting the next generation of researchers. Therefore, the committee recommends a new pilot program, perhaps run through the NIH Common Fund, to support novel experiments to improve the training and research landscape for the next generation of investigators as they seek to transition into independent research careers. The Biomedical Research Enterprise Council (BREC) described in Recommendation 3.1 could serve as a repository for all experiments proposed in Recommendations 6.1 and 6.2 (below). This pilot program could support approaches to, for example, increase the diversity of new faculty, improve support for staff scientists, and accelerate transitions to research positions in various sectors, such as academia, industry, or government. Proposals for pilot projects would be subject to NIH peer review and would address issues within or overlapping with the province of this report. The proposed experiments should be designed to include rigorous assessment, and those receiving funding under this program should be required to make results widely available at timely intervals, given that the final outcomes will take years to assess. In addition, institutions should be invited to make joint proposals or assemble consortia to submit a single proposal so that the pilots can proceed at multiple institutions where appropriate. One model for such a pilot program could be the Burroughs Wellcome Fund’s Physician--
Scientist Institutional Program,2 which provides awards of $500,000 per year over 5 years to create innovative programs to increase opportunities for physician-scientists to pursue careers in research, with careful evaluations of the funded programs.
Consistent with the need for broader stakeholder ownership of the biomedical research system, the experiments proposed in Recommendations 6.1 and 6.2 could be financed through a cost-sharing arrangement that includes NIH, universities, and industry. This approach creates incentives for stakeholders to mount programs that assist in reorienting the system, rather than the present tradition of focusing much of the responsibility for new policy reform on the shoulders of NIH. Ideally, applicants would be required to identify at least partially matching funds to ensure their commitment to the enterprise and to strengthen the impact of any initiatives designed to improve the biomedical research system.
The National Institutes of Health should enhance the use of its Institutes and Centers as vehicles to pilot new mechanisms designed to support the independence of early-career researchers and thereby strengthen its capacity for innovation more broadly. The Biomedical Research Enterprise Council proposed in Recommendation 3.1 should monitor and evaluate those efforts.
At NIH, policy reform involving the next generation of researchers tends to follow two broad approaches. In the first approach, a single Institute or Center develops an initiative that meet the needs of its research mission, which may later be adopted by other Institutes. For example, the National Cancer Institute (NCI)’s Research Specialist Award is designed to encourage the development of stable career opportunities for exceptional scientists who want to pursue research within the context of an existing cancer research program, but not serve as independent investigators. Another example is the National Institute of General Medical Sciences (NIGMS)’s recent decision to support early-career investigators through the Maximizing Investigators’ Research Award (MIRA). Created to support established investigators, MIRA was recently expanded to include early-stage investigators and awards money to the investigator instead of a specific project, thus providing more flexibility to pursue new ideas and opportunities as they arise.3
The second approach to developing new policies emerges out of the NIH Office of the Director and takes several forms:
- General policies for the ICs for the administration of their grant programs, such as the early-stage and early established investigator policies through
2 See https://www.bwfund.org/grant-programs/biomedical-sciences/physician-scientist-institutional-program (accessed December 6, 2017).
3 See https://grants.nih.gov/grants/guide/pa-files/PAR-17-190.html (accessed December 6, 2017).
- which the Office of the Director issued a general set of expectations for supporting certain early-career investigators;
- Parent announcements for specific grant programs that provide individual Institutes with discretion to decide the extent of their participation, such as the Pathway to Independence (K99/R00) award introduced more than a decade ago to support a timely transition from a mentored postdoctoral research position to a stable independent research position; or
- New funding programs, generally out of the Common Fund, such as the Director’s New Innovator (DP2) and Early Independence Awards (DP5).
Less common is an approach that integrates these two approaches for policy change. Although programs emerge from individual institutes and centers (ICs), as well as from the NIH Office of the Director, no central and independent mechanism is charged with evaluating the performance of these experiments, and, where successful, promoting their adoption across other NIH ICs and the broader biomedical research community. An integrated approach would use the dynamic and decentralized structure of NIH ICs as natural sites for policy experimentation regarding the multi-dimensional challenges confronting young investigators. NIH should champion the rapid adoption (and tailoring, as appropriate) by all ICs of experiments proven successful through rigorous evaluation. Conversely, when evaluation provides evidence that a policy is ineffective or counterproductive, the practice or policy should be abandoned.
Estrada, M., M. Burnett, A. G. Campbell, P. B. Campbell, W. F. Denetclaw, C. G. Gutiérrez, S. Hurtado, G. H. John, J. Matsui, R. McGee, C. M. Okpodu, T. J. Robinson, M. F. Summers, M. Werner-Washburne, and M. Zavala. 2016. Improving underrepresented minority student persistence in STEM. CBE Life Sciences Education 15(3):es5.
National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2011. Expanding underrepresented minority participation: America’s science and technology talent at the crossroads. Washington, DC: The National Academies Press.
Tinkle, S. S., J. C. Mary, J. E. Snavely, C. A. Pomeroy-Carter, and C. K. Tokita. 2016. An outcome evaluation of the National Institutes of Health Director’s New Innovator Award program for fiscal years 2007–2009. Washington, DC: IDA Science & Technology Policy Institute.
This page intentionally left blank.