Enhancing Philanthropy's Support of Biomedical Scientists: Proceedings of a Workshop on Evaluation (2006)

Jessica C. Fanzo and Elaine K. Gallin

The goal of the Doris Duke Charitable Foundation’s Medical Research Program (MRP) is to support and strengthen clinical research¹ in order to speed the translation of basic research findings into new cures, preventions, and therapies for human disease. Since 1998, when it awarded its first grants, the MRP has supported a number of recurring competitive grant programs. Three of these programs fund physician-scientists at different stages of their careers—the medical student level, the junior faculty or postdoctoral fellow level, and the midcareer level. These three programs were created because of the decreasing number of physician-scientists in comparison to the pool of physicians over the past decade (Zemlo et al., 2000; Nathan, 1998, 2002). This decrease has been particularly discouraging since it has occurred during a period of unprecedented scientific opportunities and a growth in research funding.

The first grants program that the MRP launched was the Clinical Scientist Development Award (CSDA) program. Established in 1998, it supported junior-level physician-scientists conducting clinical research in cancer, cardiovascular diseases, AIDS, and sickle cell anemia or other blood diseases. The transition from a postdoctoral fellow or a junior fac-

ulty member to an established investigator with his or her own trainees can be more difficult for physician-scientists, who must balance the demands of seeing patients with those of conducting research. By providing up to five years of support to these junior investigators to protect their time and support their research, it was expected that they would be more likely to compete successfully for subsequent grants and to remain in clinical research.

Between 1998 and 2001, five CSDA grant competitions were held that awarded grants to junior physician-scientists with an M.D. or M.D.-Ph.D. who either completed one or more years of a full-time clinical research fellowship or were faculty members at the assistant professor level or below for three years or less. Eligibility requirements included the need to be (1) a physician (M.D. or M.D.-Ph.D.) conducting translational clinical research; (2) working at and nominated by a U.S. institution; (3) devoting at least 75 percent of one’s time to clinical research; (4) mentored by a senior clinical investigator; and (5) at the point in their career path where they have not yet received a National Institutes of Health (NIH)–type R01 grant.

The nominee’s research proposals needed to have direct application to the prevention, diagnosis, or treatment of cardiovascular diseases, cancer, AIDS, or sickle cell anemia and other blood disorders. The proposed research could include (1) studies on the etiology and pathogenesis of these diseases in humans; (2) therapeutic interventions; (3) clinical trials; (4) disease control research that investigates how scientifically obtained information on prevention, early detection, and early diagnosis can be efficiently applied; (5) epidemiological studies; and (6) health outcomes research that attempts either to determine systematically the risks/benefits and costs of various medical practices or to utilize these results in defining more effective medical practice guidelines.

CSDA grantees were selected using a two-stage process. Institutions were invited to nominate several candidates in each disease area, and then nominees submitted research proposals and letters of recommendation, which were reviewed by an expert review panel.

Table 1 summarizes the number of applicants and grants awarded from 1998 to 2002. Junior-faculty-level grants were awarded during each of the five CSDA competitions. These grants provided $100,000 annually plus $8,000 per year in indirect costs. Faculty-level grantees were reviewed during the third year of their grants to determine if they would receive funding for years 4 and 5. CSDA grants also were made to support fellows in 2000, 2001, and 2002. Fellows received grants of $65,000 per year

for up to two years, at which time they were expected to transition into faculty-level positions. Fellows successfully transitioning received an additional three years of faculty-level funding. During the three competitions in which both fellow-level applicants and faculty-level applicants were considered, institutions could nominate candidates at each level.

The applicant success rate ranged from 10 to 17 percent. When awards were offered to faculty-level researchers and fellow-level researchers (between 2000 and 2002), all applicants competed in the same pool. Nevertheless, at least 30 percent of the top-ranked applications in those three competitions were from fellows. Out of 501 applicants, 155 were females (31 percent female applicants). Overall, 48 men and 23 (32 percent) women received CSDA grants.

While it is too early to conduct an in-depth evaluation of the CSDA program, the accomplishments of the CSDA grantees were monitored to begin to track the effectiveness of the program. These early results are outlined below:

Grantees were required to submit annual progress reports using a Web-based reporting system. Reports included information on research progress, financial expenditures, and future-year budgets. Information was also requested on their percentage effort spent conducting research; promotions and honors, publications; new grant applications; and new grants received. These data, which are part of a relational database, will

be used for a long-term evaluation of the program. The data have also been reviewed annually to ensure that grantees were fulfilling the program requirements, to track grantees’ progress, and to review their financial expenditures. Frequently, grantees requested to carry over unspent funds into future years. These requests may partly reflect the fact that it can take longer than expected to initiate a clinical research project because of issues such as patient recruitment. Regardless of the reason, the foundation’s flexibility in approving most requests to carry over unspent funds appeared to be important to CSDA grantees.

Of the 71 CSDA grantees, 57 were eligible to apply for years 4 and 5 renewal funding.² The renewal process was intended to provide the grantees with both an incentive to keep up their productivity and feedback from experts that would help them obtain additional grant support in the future. Six CSDA grantees did not apply for renewals because they relinquished their grants before their three years of support ended. The grantees who surrendered their grants early did so because they took research jobs outside the country or at NIH, or they received research grants that precluded them from keeping their CSDA grants. It is noteworthy that all six of these grantees remained in research.

To obtain funding for years 4 and 5, grantees submitted continuation applications that included detailed research plans for years 4 and 5 and their accomplishments during their grants. Three scientific experts evaluated the applications. Each renewal application was considered on its own merits. The success rate for the renewals was not predetermined.

As summarized in Table 2, 12 of the 57 grantees considered for renewal were not recommended for additional funding. Five of the 45 grantees who received renewal funding were funded for only one additional year. The renewal success rate was the same for both men and women. The primary reason for not receiving a renewal was low productivity, although occasionally a grantee’s time available for clinical research or commitment to clinical research played a part in the decision. While it was disappointing that 19 percent of the grantees were not recommended for continued funding, it is important to emphasis that subsequent survey data (see later section of this paper) indicate that most of these 12 grantees obtained additional research funding and appear to be successfully pursing clinical research careers.

Between 2000 and 2002, 14 grants were awarded to fellows. Fellows were required to transition into faculty-level positions within two years of receiving the CSDA grant. Unexpectedly, 50 percent (7 out of 14) of the fellows transitioned early—within the first year of their award. The seven fellow-level grantees not transitioning to faculty positions early submitted a transition application midway into their second year. Their applications were reviewed by scientific experts and evaluated for (1) evidence of promotion and institutional commitment (laboratory space and dedicated research time), (2) research productivity, (3) quality of the proposed research for the next three years, and (4) commitment to clinical research. All but one of the 14 CSDA fellows transitioned to faculty-level appointments. Currently, 11 CSDA fellows are assistant professors, one fellow is an instructor, one is a senior scientist at a private research institute, and one, now working in France, is a tenured junior faculty member.

No CSDA grants were awarded by the foundation in 2003 and 2004 because of budgetary issues. In consideration of reinstating the program in 2005, the foundation surveyed the 71 CSDA grantees who received grants from 1998 through 2002. The purpose of the survey was to collect information that would facilitate a quick assessment of whether the CSDA grantees were progressing in establishing themselves as independent, productive clinical researchers. The survey also asked the grantees four questions³ on their perception of the influence of the grant on their ca-

reers. It was decided that a more complete evaluation of the program should wait until all 71 grantees had completed their grants (in 2007). Thus, the survey did not collect in-depth information (such as the journals in which grantees published and their impact factor, whether their trainees were Ph.D.s or M.D.s, or the size and titles of the grants they received since the start of their CSDA grant). The following sections (one through seven) summarize the salient findings of the survey:

1. Survey respondents. Eighty-nine percent (63 out of 71) responded to the survey. Of the 63 respondents, 13 were fellows and 50 were faculty members at the time of their initial award. Table 3 includes the number of grantees responding to the survey by the year of their award. The survey data indicate that the average age for CSDA recipients was 36 (range: 29 to 41) for faculty-level grants and 34 (range: 31 to 40) for fellow-level grants. There was no difference in the age of award for men and women.

2. Promotions. At the time of the survey, 61 out of 63 grantees were still in academia and two grantees worked in industry. Forty-seven respondents (75 percent) reported being promoted since the start of their CSDA grants. As shown in Table 3, 20 grantees reported being at the associate professor level or higher. As expected, the 1998 class of CSDA grantees had the highest percentage (62 percent) of grantees at the associate professor level or above. Sixteen percent of the respondents were tenured, and most of these were from the classes of 1998 and 1999.

3. Publications and Service on Editorial Boards. The survey asked grantees for the total number of papers they published in peer-reviewed journals since first receiving their grants. Grants were awarded in July, and the survey data were obtained in January 2005. Thus, the publication data covered a period of 6.5 years for the class of grantees receiving their

awards in 1998 and 2.5 years for the class receiving grants in 2002. Table 4 contains the self-reported⁴ number of publications for the CSDA grantees broken down by grant year and whether they received a faculty-level or fellow-level grant. When the numbers of years post-award are taken into account, each class of faculty-level grantees published a mean of 2.8 to 5 papers in peer-reviewed journals per year. The mean number of papers published by fellow-level grantees was less than the mean number of papers published by the faculty-level grantees. Forty-six percent of the respondents reported serving on editorial boards or on peer review panels, and 75 percent of respondents reported serving on professional committees.

4. Time Commitment to Research and Patient Care. Grantees were asked to report the approximate time they spent conducting both basic and clinical research and the time they spent on patient care, teaching, and administration. It should be noted that one of the CSDA requirements is that grantees spend at least 75 percent of their time conducting research. Table 5 shows the mean percent effort spent on activities during a typical work week from the five classes of CSDA recipients. When comparing the cumulative means from the 1998 and 1999 classes (grantees who have completed their grants) to the most recent three classes (2000, 2001, and 2002), the 1998–1999 grantees reported spending 64 percent of their time conducting research, with approximately 60 percent of that time spent on clinical research. They also reported that 15 percent of their time was

spent on patient care not related to their research and 9 percent of their time on patient care related to their research. In contrast, grantees from the three most recent CSDA classes (grantees still receiving and/or spending CSDA funds) reported spending a mean of 75 percent of their time on research and a mean of 64 percent of their research time focused on clinical research. Therefore, it appears that after completing their CSDA grants, grantees decreased the time spent conducting research by about 10 percent. This decrease is accompanied by an increase in the time spent on patient care not related to research and teaching.

5. New Grants. A critical point in the career path of a clinical investigator is obtaining grant funding from NIH and other sources. The survey asked grantees if they had become the principal investigator of a new stand-alone grant or a project within a program project grant since receiving their CSDA grant. Ninety percent of survey respondents reported being the principal investigator on a new stand-alone grant or a project within a program project grant since the start of their CSDA

funding. Table 6 breaks down the respondents’ data by the date of the award and by whether they received years 4 and 5 renewal funding. Ninety-two percent of CSDA grantees receiving renewal funding and 83 percent of those receiving only three years of CDSA funding reported being the principal investigator on a stand-alone grant or on a project within a program project grant. CSDA grantees reported receiving R01, R21, K08, and K23 grants from NIH, with some grantees receiving more than one type of NIH grant. The awardees receiving years 4 and 5 renewal CSDA funding were more successful in obtaining NIH grants than those not receiving renewal funding (80 percent compared to 67 percent). Awardees also reported receiving grants from the following private sources: Burroughs Wellcome Fund Training Research Award, Robert Wood Johnson Development Award, AACR Bristol Myers Squib award, Aplastic Anemia Foundation of America fellowship grant, American Foundation for Urologic Disease Fellowship, Dermatology Foundation Clinical Career Development award, HHMI Postdoctoral Fellowship for Physicians, American Society of Hematology Fellow Scholars award, several American Heart Association awards, Leukemia Society of America Translational Research grant, and the James S. McDonnell Foundation grant. In addition, 35 of the 63 respondents reported serving as a coprincipal investigator on a grant.

6. Perceived Influence of the CSDA Grants. Table 7 presents the questions and responses obtained to four survey questions relating to the grantees’ perceptions of the influence of the grant. These questions were

adapted from a Burroughs Wellcome Fund survey (Pion and Ionescu-Pioggia, 2003). The vast majority of grantees believed that receiving a CSDA grant influenced their clinical research careers.

7. Commitment to Clinical Research Career. When asked if they plan to spend the majority of their career conducting clinical research, 100 percent of the grantee respondents answered yes. This ensures that the goal of the program—to foster the development of physician-scientists—is on target.

The first Clinical Scientist Award Program grant recipients received their awards only seven years ago, and the most recent recipients began their fourth year of the grant cycle in July 2005. The lack of sufficient elapsed time since making these grants and the relatively small number of CSDA grantees (only 71) argue against attempting to conduct a rigorous evaluation of the program at this time. Nevertheless, information garnered from annual grantee progress reports, a year 3 renewal review of faculty-level grantees, transition reviews of fellow-level grantees to the faculty level, and a January 2005 survey of grantees has been used to monitor the progress of the CSDA grantees and to determine if the program is on track to meet its goals. This information indicates that, with few exceptions, CSDA grantees have made significant progress toward establishing themselves as productive clinical investigators. The CSDA program appears to be accomplishing its goal of fostering the development of future clinical research leaders. Based on these findings, the Doris Duke Charitable Foundation reinstated the program, and awards were announced in the fall of 2005. The foundation will continue to collect data on its CSDA grantees and, when appropriate, hopes to collaborate with other foundations and philanthropies to do comparative studies.

Nathan, D. G. 1998. Clinical research: Perceptions, reality, and proposed solutions. National Institutes of Health Directors Panel on Clinical Research. JAMA 280:1427.

Nathan, D. G. 2002. Careers in translational clinical research-historical perspectives, future challenges. JAMA 287:832.

Pion, G., and M. Ionescu-Pioggia. 2003. Bridging postdoctoral training and a faculty position: Initial outcomes of the Burroughs Wellcome Fund career awards in the biomedical sciences. Academic Medicine 78:177–186.

Zemlo, T., H. H. Garrison, N. C. Partridge, and T. J. Levy. 2000. The physician-scientist: Career issues and challenges at the year 2000. FASEB Journal 14:221–230.