4
Outcomes for the Markey Scholars

ANALYSES OF CVs, CRISP, AND CITATION DATA

For the analysis of CVs, CRISP information, and citations, data were available for a total of 240 individuals in the study: 113 Scholars, 63 top-ranked candidates, and 64 competitive candidates. Of these, 80 percent were employed in academic institutions 10 years after completing their postdoctorate. For the results of the analyses of CVs, CRISP, and citation data presented in this report, the .05 level of significance is used.

Career Progression

Interestingly, Markey Scholars had held significantly more postdoctoral fellowships than top-ranked or competitive candidates. The Scholars had an average of 1.4 postdoctoral fellowships, compared to 1.1 and 1.2 for top-ranked and competitive candidates. Across all three groups, however, there was no significant difference in the time spent as a postdoctoral fellow (mean = 4.6 years). A number of Scholars considered the requirement by the Markey Trustees for at least one additional postdoctoral year to be an unreasonable burden and suggested that this requirement be dropped for any future funding vehicle for biomedical researchers.

Just under a quarter (n = 26) of the Markey Scholars were not employed in academic institutions at the time of their interview. Most of the Markey Scholars not in academia were employed in the biotech industry (n = 11), research institutes (n = 11), or at NIH (n = 2). One Scholar was practicing law and another was not in the labor force.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 30
Evaluation of the Markey Scholars Program 4 Outcomes for the Markey Scholars ANALYSES OF CVs, CRISP, AND CITATION DATA For the analysis of CVs, CRISP information, and citations, data were available for a total of 240 individuals in the study: 113 Scholars, 63 top-ranked candidates, and 64 competitive candidates. Of these, 80 percent were employed in academic institutions 10 years after completing their postdoctorate. For the results of the analyses of CVs, CRISP, and citation data presented in this report, the .05 level of significance is used. Career Progression Interestingly, Markey Scholars had held significantly more postdoctoral fellowships than top-ranked or competitive candidates. The Scholars had an average of 1.4 postdoctoral fellowships, compared to 1.1 and 1.2 for top-ranked and competitive candidates. Across all three groups, however, there was no significant difference in the time spent as a postdoctoral fellow (mean = 4.6 years). A number of Scholars considered the requirement by the Markey Trustees for at least one additional postdoctoral year to be an unreasonable burden and suggested that this requirement be dropped for any future funding vehicle for biomedical researchers. Just under a quarter (n = 26) of the Markey Scholars were not employed in academic institutions at the time of their interview. Most of the Markey Scholars not in academia were employed in the biotech industry (n = 11), research institutes (n = 11), or at NIH (n = 2). One Scholar was practicing law and another was not in the labor force.

OCR for page 30
Evaluation of the Markey Scholars Program TABLE 4-1 Differences Among Markey Scholars, Top-Ranked, and Competitive Candidates in Academia on Selected Outcome Measures Outcome Measure Markey Scholar Top-Ranked Candidate Competitive Candidate Significant Difference Number 87 55 49 — Percentage Promoted and Tenured 100 63 57 Yesa Percentage in Top-Tier Universities 60 24 10 Yesa aMarkey Scholars are significantly different from both top-ranked and competitive candidates. TABLE 4-2 Number and Percentage of Markey Scholars and Top-Ranked and Competitive Candidates in Academia by Faculty Rank Academic Rank Markey Scholars Top-Ranked Candidates Competitive Candidates Total Assistant Professor 0 17 11 28   (0%) (31%) (22%) (15%) Associate Professor 43 29 22 94   (49%) (53%) (45%) (49%) Professor 44 6 6 56   (51%) (11%) (12%) (29%) Other Positiona 0 3 10 13   (0%) (5%) (20%) (7%) Total 87 55 49 191   (100%) (100%) (100%) (100%) aOther positions include adjunct professor, instructor, research professor, visiting professor, clinical professor, etc. As shown in Table 4-1, of the Markey Scholars employed in academia, all were employed in tenure-track positions and all were tenured. Moreover, as shown in Table 4-2, half had also been promoted to full professor. For top-ranked and competitive candidates, not all of the academicians were in tenure-track positions, about one-quarter had not been tenured, and only a few had been promoted to full professor. Markey Scholars in academia were also granted tenure in significantly less time than either top-ranked or competitive candidates. Markey Scholars reached tenure significantly faster with an average of 5.4 years in the professoriate, compared to 7.1 and 7.8 years for top-ranked and competitive candidates.

OCR for page 30
Evaluation of the Markey Scholars Program As also shown in Table 4-1, of the 87 Markey Scholars employed in academia, significantly more, 60 percent, were employed in “top-tier” institutions, compared to top-ranked candidates employed in academia, only 34 percent, and competitive candidates, 26 percent. We operationally defined top-tier institutions as the top-ten highest ranked institutions in the fields of biochemistry and molecular biology, cell and developmental biology, molecular and general genetics, and neurosciences as listed in Research-Doctorate Program in the United States.1 Publications and Citations To examine the productivity of Scholars relative to the comparison groups, the committee calculated the number of journal articles each individual published for a 14-year period beginning with year the individual became or would have become a Markey Scholar. The 14-year period was chosen because it was assumed that all individuals would have obtained their first professional position (i.e., an assistant professorship if in academia) within four years and the focus of the assessment was on productivity from the time of the award through the period 10 years after the first professional position. Thus, for example, journal articles for those in the first cycle were counted for the period 1985-1998. Reviews in journals were not counted and neither were book chapters. The source of the data for the number of journal articles were CVs obtained from each individual included in the study (cycles 6 and 7 were excluded from this analysis due to an insufficient number of up-to-date CVs for individuals in the comparison groups). Mean and median number of articles were calculated for each group within a cycle and overall. As shown in Table 4-3, Scholars were slightly more productive than the individuals in the two comparison groups. While top-ranked and competitive candidates had roughly the same number of mean journal articles at 36.5 and 34.8 respectively over the 14-year period, Scholars had a higher mean number of journal articles during the period at 44.1, or better than 3.1 articles per year. There tended to be several Scholars in each cycle who were highly productive. To adjust for this, the committee also calculated the median number of journal articles per individual. As also shown in Table 4-3, top-ranked candidates had the lowest median number of journal articles, 1 These institutions include, in alphabetical order: California Technological Institute; Columbia University; Harvard University; Johns Hopkins University; Massachusetts Institute of Technology; Rockefeller University; Stanford University; University of California, Berkeley; University of California, San Diego; University of California, San Francisco; University of Washington; University of Wisconsin; Washington University; and Yale University.

OCR for page 30
Evaluation of the Markey Scholars Program TABLE 4-3 Mean and Median Number of Journal Articles for Markey Scholars, Top-Ranked Candidates, and Competitive Candidates, by Cycle and Overall Cycle Years of Articles Number with Article Data Markey Scholars Top-Ranked Candidates Competitive Candidates 1 1985-1998 16 2 10 2 1986-1999 14 7 12 3 1987-2000 16 5 5 4 1988-2001 16 9 6 5 1989-2002 15 9 4 ALL   77 32 37     Mean Number of Articles Cycle Years of Articles Scholars Top-Ranked Competitive 1 1985-1998 60.4 23.0 39.4 2 1986-1999 30.7 28.9 31.6 3 1987-2000 40.8 40.0 28.4 4 1988-2001 45.9 27.6 46.0 5 1989-2002 40.9 52.4 24.5 ALL   44.1 36.5 34.8     Median Number of Articles Cycle Years of Articles Scholars Top-Ranked Competitive 1 1985-1998 44.5 23.0 40.0 2 1986-1999 31.5 27.0 33.0 3 1987-2000 43.0 29.0 25.0 4 1988-2001 37.0 24.0 51.5 5 1989-2002 39.0 54.0 24.5 ALL   36.0 30.0 34.0 with competitive candidates higher and Scholars just slightly higher than them. In sum, the committee concludes that, overall, Scholars were slightly more productive than individuals in the comparison groups as measured by number of journal articles. However, much of the greater productivity was generated by Scholars in particular cycles, (e.g., 1 and 4) where several Scholars were so productive as to elevate the mean number of articles above the typical number (the median) for Scholars in that cycle. To assess the impact of these journal articles, the committee also counted the number of citations for them through 2004. So, for cycle 1 individuals, for example, the committee tallied the citations for the period

OCR for page 30
Evaluation of the Markey Scholars Program 1985-2004 for the journal articles published by this group in the period 1985-1998. For cycle 5, to provide another example, the committee counted the citations for 1989-2004 for journal articles published by individuals in this cycle during the period 1989-2002. The source of the data for the number of journal articles were CVs obtained from each individual included in the study (cycles 6 and 7 were excluded from this analysis due to an insufficient number of up-to-date CVs for individuals in the comparison groups) and citation data for each of the articles obtained from a custom database provided by the Institute for Scientific Information (ISI). Mean and median citations per individual and mean citations per article were calculated for each group within each cycle. In addition to having higher numbers of journal articles, as shown in Table 4-4, Scholars had both higher numbers of mean and median citations per individual and higher numbers of citations per article than the individuals in the two comparison groups for almost every cycle and overall. Top-ranked candidates had higher mean and median citations than the candidates in the competitive group in three out of the five cycles and overall and higher citations per article in four of the five cycles. Table 4-4 also shows that, as the median number of citations per individual tended to be lower than the mean number for almost every group, there were individuals, particularly among the Scholars, who elevated the mean because their articles were so highly cited. That is, the impact of some Scholars was significantly greater even than other Scholars. Extramural Funding One of the most important indicators of independence in the sciences is the ability to attract extramural funding. For biomedical scientists, the National Institutes of Health (NIH) is the largest and most significant provider of extramural funding. In 1996, for example, NIH awarded more than 30,000 grants totaling in excess of $8 billion. Of the awards available from the NIH, the traditional research project (R01) grant is widely accepted as the most important indicator of scientific independence. During the five-year interval from 1992 to 1996, NIH averaged 18,000 R01 awards totaling about $4 billion annually (NIH, 2006a). Using the NIH’s CRISP database (National Institutes of Health, 2006b), we examined the grant productivity for all Scholars and candidate groups, regardless of their current position. We undertook this analysis both including and excluding individuals not currently employed in academia as NIH research grants are not limited to university-based researchers. We examined grant productivity from four perspectives. First, we tallied the total number of grants obtained during the 10-year interval of interest. Second, we calculated the interval between obtaining faculty status and the first NIH grant. Third, we tallied the

OCR for page 30
Evaluation of the Markey Scholars Program TABLE 4-4 Mean and Median Number of Citations per Individual and Mean Citations per Article for Markey Scholars, Top-Ranked Candidates, and Competitive Candidates Cycle Years of Articles Years of Citations Number with Citation Data Scholars Top-Ranked Competitive 1 1985-1998 1985-2004 16 2 10 2 1986-1999 1986-2004 12 7 10 3 1987-2000 1987-2004 16 5 5 4 1988-2001 1988-2004 15 7 6 5 1989-2002 1989-2004 15 8 3 ALL     74 29 34 Cycle Years of Articles Years of Citations Mean Citations per Individual Scholars Top-Ranked Competitive 1 1985-1998 1985-2004 4,141 401 1,095 2 1986-1999 1986-2004 2,593 1,961 1,822 3 1987-2000 1987-2004 5,133 2,448 1,114 4 1988-2001 1988-2004 3,596 1,210 2,311 5 1989-2002 1989-2004 3,223 2,869 828 ALL     3,808 2,007 1,503 Cycle Years of Articles Years of Citations Median Citations Scholars Top-Ranked Competitive 1 1985-1998 1985-2004 3,200 401 1,004 2 1986-1999 1986-2004 2,261 549 1,643 3 1987-2000 1987-2004 4,003 2,176 584 4 1988-2001 1988-2004 1,733 1,299 922 5 1989-2002 1989-2004 2,381 2,486 586 ALL     2,475 1,426 980 Cycle Years of Articles Years of Citations Mean Citations Per Article Scholars Top-Ranked Competitive 1 1985-1998 1985-2004 69 17 28 2 1986-1999 1986-2004 85 68 59 3 1987-2000 1987-2004 126 61 39 4 1988-2001 1988-2004 79 51 50 5 1989-2002 1989-2004 79 53 38 ALL     86 55 43

OCR for page 30
Evaluation of the Markey Scholars Program number of R01 grants. Fourth we calculated the interval between obtaining faculty status and getting the R01. The results are shown in Tables 4-5 and 4-6. The results were generally the same whether the analysis was of all individuals or limited to those in academia. During the ten years following their first professional position, Scholars obtained significantly more NIH grants than competitive candidates but about the same number as top-ranked candidates. There was little difference in the time to first NIH grant among the three groups. Similarly, Scholars received significantly more R01 grants than competitive candidates, though only slightly more than top-ranked candidates. Importantly, they took significantly less time to get their first R01 grant than either top-ranked or competitive candidates. TABLE 4-5 Differences in Grant Awards Among Markey Scholars, Top-Ranked Candidates, and Competitive Candidates Outcome Measure Markey Scholar Top-Ranked Candidate Competitive Candidate Significant Difference Number of NIH Grants 3.3 3.2 1.9 Yesa Years to First NIH Grant 3.5 4.1 3.9 No Number of R01 Grants 1.8 1.5 0.9 Yesa Years to First R01 Grant 4.0 5.6 5.6 Yesb aMarkey Scholars are significantly different from competitive candidates only. bMarkey Scholars are significantly different from both top-ranked and competitive candidates. TABLE 4-6 Differences in Grant Awards Among Markey Scholars, Top-Ranked Candidates, and Competitive Candidates in Academia Outcome Measure Markey Scholar Top-Ranked Candidate Competitive Candidate Significant Difference Number of NIH Grants 3.5 3.4 2.2 Yesa Years to First NIH Grant 3.4 3.9 3.8 No Number of R01 Grants 2.0 1.6 1.1 Yesb Years to First R01 Grant 4.0 5.4 5.5 Yesa aMarkey Scholars are significantly different from competitive candidates only. bMarkey Scholars are significantly different from both top-ranked and competitive candidates.

OCR for page 30
Evaluation of the Markey Scholars Program Summary In conclusion, Scholars were significantly more likely to be promoted, tenured and work at a top-tier university. They had a higher mean number of journal articles and a slightly higher median number of journal articles. The Scholars’ citations, on average, were higher both per article and per individual. While Scholars obtained significantly more NIH grants, and R01s in particular, than competitive candidates, there was no statistical difference when comparing Scholars and top-ranked candidates. However, Scholars were able to obtain R01 grants faster than both top-ranked and competitive candidates. The committee believes that the high level of productivity, especially of the top-ranked candidates, is a testament to the effectiveness of the selection process in identifying highly competitive candidates. INTERVIEWS OF SCHOLARS AND TOP-RANKED AND COMPETITIVE CANDIDATES We conducted 35 to 45 minute phone interviews with each of the Scholars and comparison group members approximately 10 to 12 years after they received or applied for the Markey Award. The topics in the survey instrument specifically probed the Scholar’s decision-making process over those years. The ethnographic interview schedule was modified for use with the top-ranked and competitive candidates and is included in Appendix F. The committee elected to treat the interview data as descriptive; therefore, these data were not analyzed statistically. Specifically, the interview was designed to address the following areas: Was the flexibility of Markey funding important? Did it enable Scholars to change locations when appropriate? The ability of Scholars to venture into a “risky” research agenda. The impact of Trustee negotiations with employing institutions to ensure that Scholars were not burdened with teaching and administrative responsibilities. The length of the Scholars award and the value of mandating some postdoctoral experience. The importance of (up to) seven years of stable funding on the family formation of biomedical scholars and their (frequently also professional) spouses.

OCR for page 30
Evaluation of the Markey Scholars Program Nomination for Markey Award All candidates had to go through an internal institutional review process before being nominated as only six nominations per institution were permitted each year (except during the initial year, when the limit was four nominations per institutions). Sixty-four percent of the Scholars (76.2 percent of females and 61.4 percent of males) remembered being selected by their advisor or department chair to apply for the Markey Award. The remainder remembered being self-motivated in submitting their application or did not remember the process at all. Some Scholars reported occasional “tension” between them and other postdoctoral fellows or others in the laboratory who did not get such a prestigious award, but in general the Scholars were coming from well-supported laboratories, so jealousy was rarely an issue. Independence Approximately 60 percent of the Scholars considered themselves independent at the time they received the Markey award in terms of their capacity to devise their own experiments prior to starting their postdoctoral or fellow position. The remaining Scholars felt that their sense of independence developed during this period. Differences in the self-report of independence, however, were dependent on the group, the academic degree, and the gender of the respondents. Among Ph.D.s, Scholars claimed independence at a higher rate than top-ranked and competitive candidates. Scholars who were Ph.D.s reported feeling more independent than M.D.s or M.D./Ph.D.s. in the other two groups (Table 4-7). Female Scholars were less likely to report being independent at the start of their postdoctorate than were the males and Ph.D. (Table 4-8). Females in the top-ranked candidate pool, however, reported a greater sense of independence than the female Markey Scholars, but the sample sizes were too small to provide conclusive results. More males among competitive candidates reported a developing period of independence than Markey Scholar males. TABLE 4-7 Percentage Claiming Independence, by Degree and Group   Ph.D. M.D., & M.D./Ph.D. Total Scholars 67 (n = 42) 40 (n = 27) 61 (n = 69) Top-ranked 46 (n = 17) 71 (n = 10) 53 (n = 17) Competitive 42 (n = 13) 32 (n = 6) 38 (n = 19) Total 55 (n = 72) 50 (n = 43) 53 (n = 115)

OCR for page 30
Evaluation of the Markey Scholars Program TABLE 4-8 Percentage Claiming Independence, by Gender and Group   Male Female Total Scholars 65 (n = 59) 42 (n = 10) 61 (n = 69) Top-ranked 50 (n = 21) 47 (n = 6) 53 (n = 27) Competitive 34 (n = 13) 50 (n = 6) 38 (n = 19) Total 54 (n = 93) 50 (n = 22) 53 (n = 115) Several Scholars mentioned that they appreciated having the extra time in their postdoctoral period (as stipulated by the Markey Award) to develop new lines of investigation and to cement their independence from their postdoctoral mentor. For many Scholars, having their own supply money and salary meant that those Scholars were able to function as a sort of “lab within a lab” in the latter days of their postdoctoral appointments. The top-ranked and competitive candidates used a variety of alternate support mechanisms to fund their postdoctoral training period. The majority of top-ranked candidates (66 percent) had support from either HHMI or private non-profit foundations. The majority of competitive candidates (48 percent), in contrast, had support through government training grants (either NRSAs or medical school residency/fellowship support). Other forms of support for both groups included using the PI’s research grant funds or international government support. In all cases, the support was usually for salary only or with at best very small allotments for supplies. Ninety percent of the top-ranked candidates and 84 percent of the competitive candidates, however, reported they did not have to cut short their postdoctoral training due to a lack of funding. Very few Scholars commented that they changed their research direction after receiving the award, but many mentioned the award (and the time that came with it) gave them the confidence to pursue “riskier” lines of research. I think the Markey money encouraged the fellows to take more risks and thus impacted on research productivity - many of us had a poor production rate with papers during our postdoctoral fellowships because we were trying new things. However, most of the Markeys were very creative and spontaneous in their work, which I see as the most important thing. (Scholar) The Markey gave me a lot of bullets and I could just start shooting and some of the bullets hit! All in all, the Markey Award was wonderful because it allowed me to be less methodological, less reliant on validation, and I had very little

OCR for page 30
Evaluation of the Markey Scholars Program hindrance in terms of scientific questions. I had less pressure to produce, but ended up producing a lot as a result! (Scholar) The strength of the program was that they supported true innovation, not just variations on a theme; I have never found that since. (Scholar) I do believe, given the nature and abilities of the Markey scholars who were selected, they would have done fine regardless. These were the blue chip investments. Still, maybe in this day and age, we should also think about the mavericks out there, the ones willing to do things at even a higher risk level. (Scholar) Knowing that I had five years of support made it easy for me to decide to pursue research that would otherwise have been very difficult to fund. I think that aspect of the Markey is something that is so valuable. My science career would have been very different without it, and would have made much less impact. (Scholar) Comments from the comparison group members emphasize the ability of Scholars to be innovative, by highlighting how conservative one might be otherwise. There are a “whole group of guys, like myself, who were raised in a time when you had to be really conservative and grant proposals had to be backed with a lot of data. It carries over into our approaches now; we are really still very conservative in our proposals to get NIH money because of the hard times in our past. We got shell-shocked. (Top-Ranked Candidate) NIH should set aside 10-15 percent of their funds for high-risk experiments (as NCI does). NIH should fund research where if it works, it would really pay off. Look at Maya Lin (architect for the Vietnam Memorial), if she had been submitting a grant to NIH she wouldn’t have a chance because of her lack of experience. NIH should be funding the idea, not the work record. (Competitive Candidate) Awareness came for me when I tried to write the Markey application. I was supposed to write something innovative, and I realized I couldn’t do it. I realized I was a product of a myopic way of doing science and I couldn’t break out of it. (Competitive Candidate) The attitude of many who were among the top-ranked or competitive candidates was highlighted by one who stated: The separation between being a success and being a disaster is razor thin. I am fortunate that I have a good position, a lab, am funded. I can see how a different fork in the path could have led to very different circumstances. I would like

OCR for page 30
Evaluation of the Markey Scholars Program females. One hundred percent of the competitive candidate M.D./Ph.D.s (11 people) were consulting for profit, (compared with 64 percent of the M.D./Ph.D.s in the Scholars group (n = 14). As noted above, this is probably an “underreporting” of the incidence of commercial involvement, as individuals in the first two cohorts were not specifically asked this question. It was of special interest to note that in all the categories queried, female Scholars had fewer reports of commercial interests than male Scholars. Whether the seeming under-representation of females reflected a lack of interest or a lack of opportunity or a combination of both was beyond the scope of this investigation. Envision Statements The interviews concluded with questions about where the Scholars had envisioned being 10 years after assuming their first professional position. Many individuals laughed and said they had no “plan” and were just immensely grateful that they had a career and a lifestyle that they enjoyed very much. Many of those who went into the biotechnology industry confessed that they would never have considered that a career option when they were staring as postdoctoral fellows. Rather, they ended up taking advantage of unexpected opportunities, or taking a risk on a new entrepreneurial venture. Originally, I wanted to be 100 percent academic, so the fact that I am in the biotechnology industry is wholly unexpected. (Scholar) Scholars who had made 5- or 10-year plans claimed they advanced far more then they had anticipated. Overall, the Scholars expressed a very high level of personal satisfaction with both their careers and their lifestyles. I never imagined I would be where I am today; I was going to give academia a try. (Scholar) I never imagined being at a major research facility. Opportunities here are very different. And I am committed to teaching, which I did not anticipate when I started here. (Scholar) In general the top-ranked candidates were also very satisfied with their career progression, although several of them mentioned being frustrated with how much time they spent writing grant proposals. Several of them commented that they had no idea that a non-academic career could

OCR for page 30
Evaluation of the Markey Scholars Program be so fulfilling, and that they wished they had known it earlier. The committee observed that, generally speaking, Markey Scholars and top-ranked and competitive candidates were pretty happy with their careers Running an independent research laboratory was definitely the target. However, I always imagined myself in some university or medical school—not in the position I am right now [at a Research Institute]. I think teaching is very useful for focusing the directions of the experiments. (Top-Ranked Candidate) What I am doing now is not what I envisioned. It is more cerebral and less hands-on. My life style and general satisfaction are much higher doing what I am doing now. (Top-Ranked Candidate) I want to be in a place where basic science and translational science coexist. I see the future of research as being highly multidisciplinary. I have come to appreciate that the culture differences and culture clashes between the different disciplines have slowed us down. I would like to get back to the lab bench rather than consuming my time in administration. (Top-Ranked Candidate) The scary part was that you are stepping out of a plane without a parachute. I guess this is what I thought I would be doing, but I never realized that I would be spending the majority of my time “begging for money” (writing grants) and not be in the lab so much. I really miss being in the lab. (Competitive Candidate) I would never have imagined myself where I am now. My career has been full of surprises and turns. Nowadays, when I’m mentoring people, I always tell them to have a 5-year plan. It’s a nice idea which I was never able to follow. (Competitive Candidate) I always knew I would be close to an interesting and challenging area; if the funding and timing had been better, I could have held my own in academia. (Competitive Candidate) When I was a postdoctoral fellow I only thought of academia. Once I was introduced to the pharmaceutical industry, I came here without any expectations. I thought that I would be doing research at the bench much longer. I guess it was karma, fate, and serendipity that let me supervise all these people. And I think that what I do is not that far from the academic world. (Competitive Candidate) As a postdoctoral fellow at a major academic institution, you are on a high plateau of basic research; it is the be all and end all of life. I think my perspective has broadened since then. I see the research as important and something I want to do, but now I put more weight on the clinical and teaching responsibilities. I put more emphasis on seeing that the research is applied. The main difference is

OCR for page 30
Evaluation of the Markey Scholars Program seeing the greater importance of things outside of basic research. (Competitive Candidate) In some ways, yes, but I never get to do experiments anymore. I sit at my desk writing and reading mostly. This is somewhat disappointing and not what I was expecting; it is a very stressful existence. I never anticipated the stress, the worry, and the horrible feeling of not getting a grant. (Competitive Candidate) If I didn’t get the Markey Award, I was amenable to working in industry where I wouldn’t have to spend all my time writing. If I was going to write all the time, I would want to write cheap sleazy novels and make some significant dollars doing that! (Competitive Candidate) Practice of Medicine and Impact of Medical Training on Research At the time of the award, 95 percent of M.D. Scholars and 60 percent of M.D./Ph.D. Scholars had active medical licenses. The remaining persons with M.D.s had never completed a residency or had completed a residency but opted not to practice medicine. At the time of the interview, 10 to 12 years later, 50 percent of the M.D.s still had active medical licenses while only 44 percent of the M.D./Ph.D.s had kept their licenses current. There appear to be two groups of Scholars who held a M.D. degree but no license: those who never got a medical license (i.e., did not complete a residency) and those who gave up the practice of medicine after concluding they were progressing well down a path to a successful basic research career. Getting the Markey gave me the freedom not to pursue the residency. In retrospect, it may have been good for me because it meant the department couldn’t pressure me to take on clinical duties. (Scholar) I considered doing a residency for the next two or three years at the time that I applied for the Markey Award, but when I got it I decided not to pursue a residency. (Scholar) I had not practiced medicine since before I got the Markey…. I found it too distracting to bounce back and forth between the hematology/oncology clinic and laboratory. (Scholar) I am not licensed to practiced medicine. I found it too difficult to do both medicine and research well and made a choice. It was just too difficult to juggle with a family life. (Scholar) I think I would have been too unsatisfied knowing that I couldn’t do as good a job at either one as I could do if I was focusing. (Scholar)

OCR for page 30
Evaluation of the Markey Scholars Program As time passed, the clinical work became a distraction, so I focused totally on research and gave up my license. (Scholar) I came to the conclusion that there were just far too many doctors in this country, and I wanted to do the basic science. They’d never miss me in the clinics; but if I’m not at the lab bench things will never get done. I like the interpersonal relationships with my peers in research much better than I do in clinical research. (Scholar) A much higher percentage of M.D.s or M.D./Ph.D.s from the candidate groups maintained active medical licenses (88 percent) when compared to clinically-trained Scholars. Several of them intimated they had maintained their licenses as either a “security blanket” or to supplement their basic research faculty salaries. It may be that the Markey award helped the Scholars to commit definitively to research careers. I had my personal reasons not to go into the clinical field. I felt I could not achieve the maximum results in my work if I practiced both research and clinical work. I did not want to sacrifice any more family time to my work either. I also knew that I wanted to do ventures in the biotechnology industry. All in all, staying in research science was the best choice for me. (Top-Ranked Candidate) Sometimes the clinic can be uplifting change of pace when the research isn’t going so well, and it works both ways. (Top-Ranked Candidate) My basic science career is gone; my clinical science career is going strong. I am making lemonade. I have tailored my research to my surroundings. (Top-Ranked Candidate) In the beginning I did one month service a year—the work was 3-4 night calls a month and that work doubled my salary. I still do night calls and that makes it possible for me to do independent research as well as clinical work. (Top-Ranked Candidate) I know practically no one in my peer group who is active clinically and doing laboratory research. Back in the ‘80’s, when I was still a fellow, people had the idea of combining clinical work and bench research. But I think that has pretty much gone by the wayside now. (Competitive Candidate) None of the Scholars interviewed declared that getting a medical degree was a mistake. Rather, they viewed their clinical training as making them better scientists, in that they were more “in tune” with the impact of disease on patients, and the need for research in specific areas.

OCR for page 30
Evaluation of the Markey Scholars Program My clinical training has recently influenced my research directions, however. (Scholar) My research really impacts on my clinical work—the question is what other tools can I use to help the patients? Translational research is the driving force in my lab—the real motivation, but I have to admit that I have never got the “ah hah” feeling with the patients and then ran back to the lab to do an experiment. (Scholar) My ultimate goal is to apply my research to patient treatment. This is always on my mind and is driving my research interests. (Scholar) Physicians understand that medicine is an art, not a science, and that every patient is a new chance to learn. The ability to learn and adapt is the key. (Scholar) My clinical work helps me set my priorities for my research projects. (Scholar) As somebody who did clinical medicine part-time, I always viewed myself as somebody having a different role to play. I always saw my role as teaching fellows, residents and students to try and bridge the clinical world with the translational basic science world, and to try to provide the students with insights into the mechanisms of disease. (Scholar) Every once in a while a clinical patient will stimulate my research ideas, but not vice versa. (Competitive Candidate) I think my research activity makes me a better doctor. The biggest schism I’ve ever seen is developing between people who do science and people who do medicine. I think this is destructive. (Competitive Candidate) It is very important for me to be an oncologist because I see how little progress we have made, and I think it makes me do better science. (Competitive Candidate) I bring to discussions on clinical trials a real world perspective on cancer treatments. (Competitive Candidate) If you want to do research as an M.D. in a medical school, the idea is that you had better support yourself, because it is a luxury thing that you are doing for your own fun. (Competitive Candidate) Participation in Translational or Clinical Research When queried about whether they participated in translational research, which the committee defined as taking findings from the bench

OCR for page 30
Evaluation of the Markey Scholars Program TABLE 4-14 Percentage of Interviewees Engaged in Clinical or Translational Research, by Group Group Engaged in Clinical Researcha Engaged in Translational Research Planning Future Translational Research Scholars 22 34 55 Top-Ranked 16 31 45 Competitive 28 38 50 All 22 34 52 aRespondents could be in engaged in all three categories of research. to bedside and requiring Internal Review Board approvals, only 34 percent of all Scholars responded affirmatively (Table 4-14). Many of the M.D. and M.D./Ph.D. Markey Scholars, defined “doing” clinical research as advising on design and analysis, which was not categorized as an affirmative response by the committee. Few were actually heading up clinical trials, citing the daunting amount of paperwork. Even fewer Scholars, 22 percent, were actually engaged in clinical research. It is interesting to note, however, that over half of the Scholars indicated plans for future participation in translational research. Roughly the same percentage of individuals from the candidate groups were involved with patient research and clinical trials, especially those candidates who were working in pharmaceutical firms and heading up clinical trials was their primary function. Design of the Markey Program At the end of the interview, the Scholars were asked how they would have improved the program if it were to be offered again. Several recurring themes emerged from the final comments made by the Scholars: They greatly appreciated the Trust’s philosophy of “we have faith and trust in the scholar.” In hindsight, they were quite grateful for the lack of bureaucracy imposed by the Markey Trust and the flexibility produced by the Trust in the Scholar as opposed to an investment in a project. The supportive atmosphere, even for such well qualified scientists as the Scholars, was highly appreciated and several Scholars mentioned the “pat on the back” they received at the meetings meant more than the funds. The intellectual stimulation provided by the scientific meetings, even though they were frequently outside the Scholar’s area of expertise,

OCR for page 30
Evaluation of the Markey Scholars Program was invigorating and prepared them for a more broad-minded approach to science. Unrestricted funds are incredibly important for my scientific development. I didn’t fully realize it until my Markey, Searle, and Packard funds were gone. (Scholar) The program was inspired. The selection committee and Trustees clearly cared. Secondly, many career development awards don’t give people sufficient funds to be independent. The intelligent part of the Markey Scholars awards was that it started with a major infusion of funds, and then wound down over the years. The fact there were no strings attached was extremely important. Whoever designed the Scholars awards understood how NIH works and doesn’t work, and how people’s careers are best supported. (Scholar) I thought the Markey was really unique; the duration of support was really great and one of the things that made the Markey so special was the people who were involved in directing its operation were really outstanding. If you can get people like that together, things work and it felt great to be a part of it. The success of the Markey is the quality of the people who were running it. (Scholar) One of the more important aspects was not the dollars but a feeling of protection in some way and camaraderie, both between the participants and the people who were in some way involved with the Markey Foundation. What I would like to convey is not just the idea that it is great to give people lots of dollars but that it is equally important to address the welfare of the scholars; money alone is not what it is about. (Scholar) The Markey had a really positive impact. This kind of a pat on the back, keep-on-going-you’re-doing-OK, really gives you courage, confidence, extra motivation to go on; it was really a positive thing in psychological terms just as much as material. (Scholar) I really like the idea of bridge funding, and if they are looking for new funding models, I would like to see someone consider funding a person around the 4-5 year point of a junior faculty position. That is when they are on the brink of getting really interesting results, but if the person is working on a high risk project, it will be very difficult to get funding. That is the point when people are close, but not yet at the jackpot. (Scholar) It was difficult to get the Scholars to offer constructive criticism of the program, as many responded initially that the program was ideal as it was originally designed. However, when pressed, they made the following seven suggestions for improvements:

OCR for page 30
Evaluation of the Markey Scholars Program Have a more formal mentoring system. Many Scholars reported a “feeling” that someone on the Markey Selection Committee was looking out for them, but they didn’t know who, and some wished they could have gone to the Committee members directly for advice. The additional year for postdoctoral studies should be optional. Early classes were required to spend an additional year in the postdoctoral environment. They could change laboratories, but they could not start their own labs. Several Scholars in later classes protested this requirement, and it was eventually dropped. It should be noted that when queried about the additional one-year of postdoctoral study requirement, 51 percent of the Scholars felt this was a good idea, and another 41 percent said it had no effect on their future plans. An additional 6 percent felt the extra year was a burden and petitioned the Markey committee to remove this requirement (which was done after the third class). Those who felt the additional year was a good idea cited such factors as that the extra year gave them time to finish experiments, time to collect sufficient pilot data to be competitive for NIH awards, and time to conduct a job search. Provide counsel during job negotiations, especially with startup packages. Several Scholars cited the Burroughs Wellcome (Burroughs Wellcome Fund and Howard Hughes Medical Institute, 2004) job negotiation support as an ideal way to help the job applicant to assess the competitiveness of the job offer. Scholars recommended that such a system be installed in any new program. The salary component could have been shifted to the equipment/ supply budget if needed. Several Scholars noted that they felt their department was getting a “free ride” and that they were especially resentful, therefore, when the department chair tried to offer a reduced startup package. Clinical Scholars noted that as they were providing clinical service, at least some of their salaries should have come from the clinical department. Several queried whether it would have created a greater sense of commitment by the department if they had paid the salaries. Reduce the number of years of support to fund more Scholars. All the Scholars were extremely sad that such a wonderful program had only 7 classes. Several, who did not understand that the Trust had to be spent out within a fixed period of time, questioned whether it would it have made more sense to trim the program back to six years from seven to get more people enrolled. Do not exclude Scholars who move to government, the biotechnology industry, or HHMI from attending the annual meetings, although it is appropriate to stop additional payments. Many of the Scholars who migrated to HHMI and the biotechnology industry were extremely sad to be “kicked out of the club” of Markey Scholars when they left their aca-

OCR for page 30
Evaluation of the Markey Scholars Program demic appointments. All Scholars were invited to the last Markey Meeting in Puerto Rico in 2002. Encourage collaborations by providing seed-money funds. Additional funds could have been ear-marked specifically for starting collaborations between the Markey Scholars. As their research fields or even disciplines rarely overlapped, collaborations or interdisciplinary projects were difficult to start. Perhaps the addition of a competitive fund of about $50,000 per project would have lowered the barriers to these collaborations and supported innovative research lines. During the interview process, the interviewer noticed a subtle change in tone between respondents in the first three classes, and those in subsequent classes. The earlier class members had very fond memories of the Markey program and appeared extremely grateful for the opportunity to prove their scientific merit. Subsequent classes, while still grateful, frequently expressed the opinion that if they had not received the Markey award, they would have gotten “something else” of equal value or merit; that is, the Markey award was wonderful, but it was just one of several awards for which they could have applied and would have received. Individuals from the two candidate groups were asked to comment on what they thought would contribute to the design of a successful program with the goal of developing a person’s biomedical career. The candidates suggested a new funder should consider the following six possibilities when planning funding strategies: Supporting advanced postdoctoral fellows. Several of the candidates mentioned that since postdoctoral fellowships were taking longer (between 4-5 years is not atypical now), that a new funding program should consider targeting individuals in the latter years of their postdoctorate. Providing merit awards for senior scientists in the future. By developing a series of “merit awards” with significant funds attached, funders could develop an excellent mechanism to stimulate innovative thinking in experienced scientists. Supporting collection of pilot data for a second NIH grant with a new line of investigation. Providing funds to a mid-level scientist to start a new line of research could promote significant risk taking or innovation in this population of scientists. Underwriting interdisciplinary projects. As science in the 21st century becomes more interdisciplinary, funds need to be set aside for developing projects that bring a variety of disciplines together, and enhance collaborative activities, even over geographical boundaries. A specific suggestion was to require a co-PI system, where one PI was a M.D. and the other a Ph.D. in a complementary basic science field. Numerous M.D.s and

OCR for page 30
Evaluation of the Markey Scholars Program M.D./Ph.D.s left bench science in the early 1990s during the last downturn in the NIH funding cycle. This pool of individuals may be ideally suited for this sort of co-PI project. Funding more diversely so as to find the “stars.” Several candidates commented that it might be a wise idea to cast the net wider when looking for applicants—that is, to consider non-conventional applications that show a lot of thought or promise and to take the risk that this person could follow through on the ideas. Providing more mentoring throughout the career timeline. This is a repeating theme among both the Scholars and candidates. Many individuals felt that more explicit mentoring during the job search and early faculty years was essential, but that this mentoring should continue through to tenure if possible. I guess I would give people money to play with: relax on how well matured the ideas are … give them money between their postdoctoral fellowship and beginning of the career that they can make mistakes with or go down blind alleys. Take the pressure off … let them get going without worrying about doing their most exciting research right away. (Candidate) As a “survivor” of the crash in funding of the early 90’s I am especially sensitive to the needs of young faculty… . I was almost forced out of science due to a lack of funding in those early years, but now I consider myself truly successful. (Candidate) The ones who are really trying to move basic findings into clinical practice are the ones who need the most support. They should target additional funds to further down the career pipeline to early mid-career plus institutional reforms to provide a more stable and better environment for dual career people. (Candidate) I would perhaps like to see the pie sliced a little thinner, spread out a little bit more amongst people. One of the elements of science is the unknown—you never know who is going to do well. (Candidate) Candidate Comments on Markey Award Several of the candidates used the interview period as an opportunity to declare their frustration with the Markey Award process. The Scholar Selection Committee did not provide written critiques of the application packages for unsuccessful candidates. The rejected applicants found this lack of feedback on their Markey application particularly distressing. They would have greatly appreciated a constructive rejection letter that indicated how they had scored in areas such as: quality of the scientific proposal, skills necessary to perform the experiments outlined in the

OCR for page 30
Evaluation of the Markey Scholars Program scientific proposal, letters of recommendation, and productivity. The candidates believed that since they spent a significant amount of time and effort preparing a proposal, the least they could expect in return was some sort of guidance on how to improve subsequent proposals. There were also several complaints that there appeared to be a “coast-bias” in that the majority of the awardees came from either the West or East Coast of the United States, and that this in some way was a form of cherry-picking. The third theme that appeared was that there was an apparent lack of diversity in the awardees; nearly all Scholars were white males.