Enhancing Philanthropy's Support of Biomedical Scientists: Proceedings of a Workshop on Evaluation (2006)

Douglas M. Fambrough

The Searle Scholars Program makes grants to support the independent research of exceptional young scientists in chemistry and the biomedical sciences. The funds that support these awards come from trusts established under the wills of John G. and Frances C. Searle. Mr. Searle was president of G. D. Searle & Company, of Skokie, Illinois, a research-based pharmaceutical company. Mr. and Mrs. Searle expressed the wish that some of the proceeds of their estates be used for the support of research in medicine, chemistry, and the biological sciences.

In 1980, members of the Searle family, acting as consultants to the trustees of the trusts established under the wills of Mr. and Mrs. Searle, recommended the development of a program of support for young biomedical scientists. This idea evolved into the Searle Scholars Program, which is funded through grants from the family trusts to the Chicago Community Trust and administered by the Kinship Foundation in North-brook, Illinois.

Searle family members and founding director Cedric Chernick identified the need to fund exceptional young scientists just as their independent research careers were beginning. The Searle Scholars Program thus became a prototype for assisting outstanding young scientists at a critical point in their research careers. The initial awards were made in 1981. Through 2005 the program has made 407 awards totaling about $70 million. The current policy is to make 15 awards each year. Each awardee receives $240,000 over three years to support his or her research program.

This paper addresses (1) how Searle scholars are selected; (2) the

mechanisms that have been used to evaluate the program as a whole, as well as Searle scholars, in the postaward period; and (3) the conclusions that have been drawn from these evaluations.

The success of the program rests on the selection of young scientists who subsequently develop and sustain research programs that have a major impact on the progress of science and/or who subsequently make major contributions to science through their leadership. The selection of Searle scholars is based on recommendations made by the program’s Scientific Advisory Board. Given the breadth of fields supported by the program, each advisor must possess expertise in a broad range of research areas and have excellent scientific judgment, a strong sense of fairness, and a talent for working with others to arrive at selection of the most promising candidates. The board currently consists of 12 advisors distinguished for their research and leadership in fields of interest to the program (see Figure 1). The willingness of scientists of such stature to serve on the board is itself a testament to the strength of the program.

FIGURE 1 The 2005 Searle Scholars Program Scientific Advisory Board.

To evaluate the success of the program we should first consider what we know about the scholars at the time awards are made. The program uses several levels of selection to arrive at its final selection of scholars. The first level is restriction of the applicant pool to a set of invited institutions. For the 2005 competition, there were 125 invited institutions, each allowed to submit two candidates. These 125 include the universities and research institutes ranking highest in total federal support for research in chemistry and biomedical sciences plus an additional number of renowned research institutes. These invited institutions are listed on the program’s Web site (http://www.searlescholars.net).

Institutions, especially the larger ones, conduct an intramural competition to select their applicants (a second level of selection). For the 2005 competition there were 193 applicants from 122 institutions. For the 2006 competition there are 135 invited institutions; however, 55 of these may each submit only a single candidate.

Applicants are expected to be pursuing independent research careers in biochemistry, cell biology, genetics, immunology, neuroscience, pharmacology, and related areas in chemistry, medicine, and the biological sciences. For the 2006 competition, candidates should have begun their first appointment at the assistant professor level on or after July 1, 2004, and therefore be in their first or second year. This appointment must be a tenure-track position and must be in an academic department of an invited institution. Potential applicants whose institutions do not have tenure-track appointments are advised to consult with the scientific director of the program prior to preparing an application. In these instances, eligibility is determined on a case-by-case basis.

The program uses a two-step review process in which the applicant pool is first reduced to about 40 finalists, who are then evaluated further at a two-day meeting of the Scientific Advisory Board. Two rounds of discussion of the candidates lead to selection of 15 new Searle Scholars.

Information on which selection of the scholars is made consists of the application and supporting letters of recommendation. The application is a relatively short document compared to other proposal packages; its principal components are as follows:

• Abstract of a research proposal (250-word limit).

• Standard data on the candidate’s education.

• Brief summaries of doctoral dissertation and postdoctoral research (if the most common career path has been followed).

• List of publications, a measure of scientific productivity.

• Applicants may include up to five reprints of publications with the

application and are asked to provide a description of their contribution to each.

• Research program (one-page) overview.

• Applicants are asked to describe their vision of their overall research program 5 to 10 years from now, including why it might be novel and important.

• Research proposal (3.5 pages plus figures and bibliography). In this section, applicants should present their best ideas. The program is especially interested in supporting those who are creative and willing to propose possibly high-risk but also potentially high-impact research. The Searle Scholars Program does not support a particular project. Searle funds may be used to supplement ongoing projects or to initiate new endeavors.

• Career goals (about 200 words). The instructions for completing this section are vague, and it is left to applicants to decide how to describe their individual goals. Sometimes this section reveals insight into an applicant’s motivation, leadership ability, generosity, or other laudable qualities that cannot easily be assessed through other parts of the application.

• Chairperson’s letter. The chairperson is asked to describe in some detail the commitment the institution is making to the candidate: facilities, start-up funds, space, teaching load, etc.

• Letters of reference (three). These typically include the candidate’s graduate and postdoctoral mentors. Those writing such letters are asked to assess the candidate, comment on independence and originality, and compare the candidate to a list of 150 Searle scholars selected over the past 15 years.

A somewhat amusing but quite telling aspect of the letters of recommendation is the responses given to the statement: “In comparison with others I have known at the same stage in their careers, the applicant is in: the top 1 percent, top 5 percent, top 10 percent, or average.” Although it seems unlikely that the writer has actually known hundreds of people in this category, the 15 Searle scholars selected in 2005 were ranked in the top 1 percent on 31 letters and the top 5 percent on the remaining 7 letters for which this ranking was done. This must attest to the enthusiasm with which the writers endorsed the candidates.

The timeline for the selection process for 2006 applicants was as follows:

• Application deadline: September 30, 2005.

• All application abstracts sent to each advisory board member: ~October 10.

• Advisory board members return review preference sheets: ~October 30

• Assigned applications (approx. 30) sent to each advisor: ~November 10

• First-round ranking of applicants by each advisor due: ~January 7, 2006

• Selection of “Final 40” made by director and advisory board chair.

• Copies of applications of all finalists sent to each advisor: ~January 15

• Advisory board meeting to select 2006 Searle scholars on February 19–21.

• Awardees contacted, and, if there are any declines, alternates are contacted in order.

• Public announcement: once all awardees have signed acceptance forms that include agreement to terms of the award.

• Funding begins on July 1, 2006.

After reading the full set of abstracts, advisors complete the review preference sheets, which allow them to indicate which applications fall within their areas of expertise and which might pose a conflict of interest. Based on all of the advisors’ preferences, each advisor is assigned about 30 applications to rank, and each application is assigned to at least two advisors. To help with the first-round reviewing process, advisors are given score sheets that have been developed through discussions with the Scientific Advisory Board over many years. The categories for scoring are Originality of Research, Feasibility of Research, Potential Impact on Field, the applicant’s Training and References, and the applicant’s Publications. Some advisors use the aggregate scores from these categories to rank the applicants. However, advisors are free to submit rankings that do not correspond precisely with the category scores, and advisors are not required to use the score sheets at all.

The scientific director, in consultation with the chair of the Scientific Advisory Board, selects up to 40 finalists, based on the first-round rankings. Applicants ranked in the top 3 by any advisor are automatically included among the finalists, and this group generally is about 30. The remaining finalists are those who received the highest aggregate scores from the advisors. The advisors review the list of finalists and are invited to suggest additional applicants to include (there is typically one addition each year through this mechanism).

Once finalists have been selected, the Scientific Advisory Board meets to conduct its final review. In preparation, complete applications are sent to each of the 12 advisors, who are asked to evaluate all that they can. Primary and secondary reviewers are assigned to present each application. Advisors are not required to prepare written critiques, and “pink sheets” are not prepared by the program.

At the two-day Scientific Advisory Board meeting, the finalists are each discussed on the first day, and each finalist is given a score based on balloting by the advisors. The deliberation on each candidate is focused largely on two questions:

• Has the candidate significantly affected the direction of research in the labs where she/he was a graduate student and a postdoctoral fellow?

• With Searle support, would the candidate pursue novel research that, while perhaps of high risk, promises high reward?

The scores are revealed at the end of the first day; advisors are asked to consider these initial rankings and to be prepared to compare candidates. On the second day, the candidates are discussed again. Fifteen are chosen, plus several ranked alternates. Although those chosen for an award seldom turn it down, there have been three cases in which the candidate had each already accepted another award that precluded acceptance of the Searle award. (The Searle Scholars Program does not impose on the Searle scholars any restrictions regarding acceptance of other awards.)

The program has kept a database of all its scholars, asking for annual updates. In 1996 the database was the starting point for developing the Web site. At that time every former scholar was located and, with a single exception, current information was put on the site. The Web site is updated frequently and includes links to the institutional or commercial Web sites where the scholars hold positions. The Web site serves several functions, including:

• facilitating networking among Searle scholars,

• providing updated information on nearly all the 393 Searle scholars,

• serving as a source of information on eligibility and application issues, and

• keeping the Searle family informed.

The Web site has a high ranking for search engines such as Google, a nice happenstance that has encouraged scholars to keep their information current. The Web site is overseen by the scientific director, giving him an overview of the program and providing a starting point for evaluation of Searle scholar success.

The program has not conducted a comprehensive, quantitative evaluation. However, some measures of Searle scholar success are immediately evident. Most striking is that virtually all former scholars hold high positions in academia or in the biotech/pharmaceutical industries. For those scholars remaining in academia (the great majority), about half are tenured or tenure-track faculty members in medical schools, while the remainder are tenured or tenure-track in nonmedical school departments of universities. Except for a few who have turned entirely to clinical work, scholars maintain their research programs and publish in the most respected journals in their fields.

During the three-year funding period, scholars attend annual meetings held in Chicago, where they present their research work. These events give the director, attending advisors, and Searle family members opportunities to interact with the scholars and evaluate their research progress and personal styles. The scientific director also visits many of the scholars’ labs.

Searle scholars have received numerous other awards that validate their selection. The program does not maintain an exhaustive database of these awards, but it does track a few of the most prestigious ones. The data are impressive. Of the approximately 250 scholars receiving Searle awards between 1981 and 1996, 24 are now members of the National Academy of Sciences. Table 1 lists these scholars by the date of their Searle award (Searle “Class”).

Five Searle scholars have been among the rather few scientists to receive MacArthur Foundation “Genius” awards (see Table 2).

The program receives numerous testimonials from scholars. Most often these accompany the final scientific report that the scholars are required to submit at the end of their funding period. These testimonials point to multiple benefits of a Searle award:

• allows a scholar to undertake risky research not fundable by other agencies,

• facilitates obtaining other research funds,

• accelerates promotion to higher academic rank,

• attracts graduate students and postdoctoral fellows, and

• provides a community of scholars that is inspirational and useful for networking.

The majority of scholars offer unsolicited testimonials like these:

The program as a whole benefits from considerable oversight by the Searle family, which can alter the program at any time. Two Searle family members, serving as the Searle Scholars Team, act as liaisons between the

program and the Searle family. The team members attend the Scientific Advisory Board meeting and the annual scholars meeting. In addition, the program administrator and scientific director prepare a short report on the program each year. Less frequently, the scientific director prepares presentations for the Searle family that include program evaluations and comparison with other programs that have similar missions.

The Searle Scholars Program is a prototype for support of young chemists and biomedical scientists. Over the past 25 years it has made nearly 400 awards. The process through which candidates are selected by invited institutions and evaluated appears to have succeeded in the selection of a remarkable group of scientists. Almost without exception, these Searle scholars have made substantial contributions to their research fields. Rising to leadership positions, they have received numerous other awards that reflect both their promise and accomplishments. Qualitative evaluation of the program has been accomplished through tracking of all former scholars, through perusal of the database of their current positions, through noting their publications and the awards they have received, through testimonials from the scholars themselves, and through the enthusiastic commitment of the program’s distinguished Scientific Advisory Board.