Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 130
An Assessment of the SBIR Program at the National Institutes of Health 5 Program Management at NIH 5.1 INTRODUCTION The congressional charge to the National Academies was to assess the SBIR program at NIH, and to suggest possible areas for improvement. In this chapter, we focus primarily on the latter: areas where NIH might make improvements to its SBIR program. In doing so, we primarily utilize case studies, interviews with NIH staff and other stakeholders, and secondary materials, as well as data from the NRC surveys and other statistical sources. The focus of the chapter is to provide an objective review of the management of the NIH SBIR program, with a view to providing recommendations for improvement. The latter are described in a separate chapter. The structure on this chapter follows the logic of the awards cycle at NIH starting, with outreach activities to attract the best applicants, through topic development, selection, and funding, and concluding with commercialization support and a discussion of metrics and data. 5.2 BACKGROUND The NIH SBIR program started soon after the program was launched, in 1983. It has expanded steadily with the growth of extramural research at NIH, and effectively doubled over the past four years as NIH funding doubled. The program is now the second-largest, after DoD, and funded approximately $552 million in SBIR awards in FY2006. Most of these awards are made in the form of grants; about 5 percent are contracts focused on specific NIH needs. Almost all others are not designed to
OCR for page 131
An Assessment of the SBIR Program at the National Institutes of Health generate results that are purchased by NIH, unlike the procurement-oriented programs at DoD and NASA. The NIH program has a number of defining characteristics, some of which are addressed in more detail in the remainder of this chapter. Investigator-initiated research. NIH is the only agency where the topics areas in the program solicitation (request for applications) are guidelines, not mandatory limitations on research topics. Larger awards. NIH now consistently exceeds the SBA awards size guidelines for Phase I and Phase II, utilizing a blanket SBA waiver to do so. Peer-driven selection procedures. NIH appears to depend more than most other SBIR programs on external peer review for advice on award selection, although final decisions remain the responsibility of NIH staff. Regulatory concerns. NIH is the only agency whose research often requires approval from the FDA before it can reach the market. This creates an important barrier to commercialization. Multiple awarding components. Twenty-three Institutes and Centers (ICs) at NIH award fund their own SBIR awards, using a range of procedures and with different degrees of integration with other programs. Together, these characteristics give the NIH program a unique character, and have informed management of the program in a number of important ways. 5.3 OUTREACH Outreach activities at NIH are extensive, compared to some other agencies, and have received significant attention from the NIH SBIR/STTR Program Office in recent years. The activities appear in general to have had three primary objectives: To ensure that SBIR attracts the most qualified applicants; To reach geographical areas often perceived to be underserved; and To reach specific demographic groups that are perceived to be underserved (e.g., businesses owned by women and minorities). Mechanisms for achieving these objectives include: National SBIR conferences, which twice a year bring together representatives from all of the agencies with SBIR programs, usually at locations far from the biggest R&D hubs (e.g., the spring 2005 national conference was in Omaha, Nebraska). The National NIH SBIR conference held annually, in Bethesda, MD.
OCR for page 132
An Assessment of the SBIR Program at the National Institutes of Health The annual Program Administrators’ bus tour. An annual swing through several “under-represented” states, with stops at numerous cities along the way. Participants always include the NIH Program Coordinators. Web sites and listservs. NIH maintains an extensive Web site1 containing application information and other support information. A number of explanatory presentations are available online. NIH also allows users to sign up for a news list-serve. Agency publications and presentations. NIH does not appear to use print publications to any significant degree to publicize SBIR (except as NIH events are reported in other publications, for example at the state level). NIH does use electronic publications, such as the NIH Guide for Grants and Contracts, to publicize Funding Opportunity Announcements as well as the Commercialization Assistance Program and the Niche Assessment Program. Demographic-focused outreach. NIH regularly participates in several conferences designed to reach specific demographics. Overall, there are currently no metrics in place to determine whether the above three objectives have been met in the past or are now being met. Interviews at NIH suggest that the staff believes more outreach is required, and that raising the size of awards has been the most important recent NIH outreach initiative. Some staff members suggest that bigger awards attract better applicants. NIH has strongly supported the SWIFT bus tour, and the NIH SBIR/STTR Program Coordinator has gone on all recent tours personally.2 Staff members claim to have noticed a spike in applications from visited states and regions, but have no empirical evidence matching bus tours with increased applications. A review of IC Web sites also indicates that they provide a range of online information from very basic to “fancy bells and whistles.” The Institutes and Centers (ICs) vary greatly in their resources and talent to launch attractive and informative Web pages. It could therefore be helpful if the NIH SBIR/STTR Program Office could develop a standard information package that the Institutes could then adapt for their particular programs, e.g., to display their own particular list of initiatives. 5.3.1 Attracting the Best Applicants The NIH staff notes that average scores for SBIR awards have trended upward (NIH scores range from 100 (best) to 500 (worst), so an upward trend indicates relatively weaker applications.) Some staff members have stated that the 1 Accessed at: <http://www.nih.gov/grants/funding/sbir.htm>. 2 SWIFT is a multistate bus tour periodically undertaken by SBIR Program Administrators from different agencies to fuel technology growth and development across different regions by promoting awareness of the SBIR programs.
OCR for page 133
An Assessment of the SBIR Program at the National Institutes of Health rapid expansion of funding in the program, together with the trend in marginally funded scores, means that relatively weak applications are being funded. This observation raises two questions: Is this perception accurate—is the quality of funded SBIR applications low relative to those that receive other NIH funding? If so, does this mean that there are other better-qualified companies who are not applying for SBIR? Low relative scores. From discussions with staff, it appears that the paylines3 for SBIR awards at the different IC’s are substantially higher than for RO1 awards,4 and these gaps have grown recently. This implies that projects funded through SBIR are receiving worse peer-review scores than projects funded through other mechanisms. NIH management decided not to share scoring data with the research team, so it is difficult to determine whether or to what extent reality matches perceptions in this area. However, it seems likely that these different scores may well be the result of using a selection process that is primarily aimed at selecting academic applications for basic research and adapting it for use with SBIR, which has different objectives and indeed different selection characteristics. For example, commercialization plans are supposed to play an important role in selection for SBIR, but not for other NIH awards. It does not appear that program staff has undertaken research either to substantiate this perception or to investigate possible alternative explanations for differential scores between RO1 and SBIR applications. New companies are applying. More than 30 percent of winning applications are from companies not previously funded by the NIH SBIR program.5 New companies participate in the annual conferences, and hits on the Web site continue to increase. The new entrants in the program illustrate the attractiveness of SBIR awards but do not address the qualifications of the applicant companies. Burden on staff. There are “cultural” issues that may affect perceptions of project or company quality. In interviews and responses to the NRC Program Manager Survey, many NIH staff noted that SBIR applicants and awardees placed a disproportionately high burden on agency staff, compared to similar applicants and awardees in other programs. Michael-David Kerns of NIA may have expressed this issue most clearly, observing that “We spend a disproportionately large amount of time with program administrators interacting with both 3 The payline is defined as the score for the worst-scoring application that is still funded. 4 RO1 awards are grants made to individual researchers. They constitute the most common form of NIH award, and are also sometimes used as an informal comparison group for SBIR awards. However, as explained below, they are different, and comparisons between these groups are invalid. 5 See Section 220.127.116.11: New Winners.
OCR for page 134
An Assessment of the SBIR Program at the National Institutes of Health potential and actual SBIR-STTR applicants.6 These potential and actual SBIR-STTR applicants send emails and telephone much more than other categories of applicants (for basic research grant programs at NIH-NIA), making tremendous demands upon the time of program administrators and the grants management specialists…. Some of the reluctance and the comparatively low regard for the SBIR-STTR Programs, is the amount of time that would-be applicants attempt to and actually engage program administrators in marketing and selling their project and product idea. Even after having explained, usually more than once, that program administrators at NIH-NIA are not in the position of “buying” any project and/or product, the SBIR-STTR potential applicants persist in marketing and selling their projects and products. NIH-NIA program administrators are not accustomed to and do not welcome attempts by individuals to “sell” anything”7 5.3.2 Applications and Awards from Underserved States Chapter 3 on program awards illustrated the extent to which awards have been concentrated geographically. A single zip code in San Diego has received more than twice as many awards as any other zip code in the country. Massachusetts and California alone account for 36 percent of Phase I awards 1992-2005. Even though there has been some increase in awards to underserved states, data for FY2005 shows that six states received zero Phase I awards, and a further four states received one or two.8 A better approach to the issue of underrepresentation would be to look at applications per scientists and engineer. The distribution of the latter reflects the distribution of scientific and engineering talent, which should tend to predict applications and awards as well. As Table 5-1 shows, there are wide variation in the number of applications per 1,000 scientists and engineers, indicating that scientists and engineers in some states use the SBIR program much more—in fact up to twenty times more—than those in other states. This does raise some important practical questions for the NIH program. To begin with, it points to a somewhat different set of “underserved” states. While 6 The Small Business Technology Transfer Program (STTR) reserves 0.3 percent of federal extramural R&D funding (vs. 2.5 percent for the SBIR program) for competitive awards to facilitate cooperative R&D between small business concerns and U.S. universities and research institutions, with potential for commercialization. STTR was established as a companion program to the SBIR program, and is executed in essentially the same manner. There are, however, distinct differences. Most notably, each STTR proposal must be submitted by a team that includes a small business (as the prime contractor for contracting purposes) and at least one research institution. The project must be divided such that the small business performs at least 40 percent of the work and the research institution(s) performs at least 30 percent of the work. The remainder of the work may be performed by either party or a third party. 7 Response to NRC Program Manager Survey, April 2006. 8 See Section 3.2.4: Phase I—Distribution Among the States and Within Them.
OCR for page 135
An Assessment of the SBIR Program at the National Institutes of Health TABLE 5-1 NIH SBIR Phase I Applications per 1,000 Scientists and Engineers MA 50.5 NJ 13.7 MN 10.1 AL 7.4 NV 5.0 MD 33.4 HI 13.5 OH 9.9 TX 7.0 LA 4.8 UT 23.2 CO 13.5 IL 9.7 ND 6.8 NE 4.7 NH 23.1 CT 13.0 MT 9.7 FL 6.5 KS 4.7 CA 19.1 WA 12.7 DC 9.7 KY 6.4 AR 4.4 VT 18.7 NY 12.6 WY 9.3 IA 6.3 OK 3.9 RI 16.2 SD 12.3 NM 9.0 MO 6.1 ID 3.4 DE 16.0 PA 11.5 WI 8.0 GA 6.0 SC 3.1 VA 15.2 NC 11.0 TN 7.9 IN 5.7 MS 2.5 OR 14.5 ME 11.0 AZ 7.4 MI 5.7 WV 2.2 AK 1.4 SOURCE: U.S. Census; National Institutes of Health. states with low numbers of applications per scientists and engineer tend to have low numbers of applications overall and hence low numbers of awards, only five of the bottom ten states in Table 5-1 are also among the bottom ten states in overall awards. Some underserved states have made substantial efforts to win more awards in recent years. This approach has been partly supported by the FAST program.9 While a comprehensive analysis of the FAST program is not available, interviews with state agency staff and program participants suggest that, despite its limited funding, the program has been successful in helping to generate additional applications. Additional applications do not, however, always translate into increased awards. For example, the state of Louisiana has made significant outreach efforts that have resulted in an increase in the number of Phase I applications to NIH from six in 1998 to 20 in 2001. However, during that period the number of awards increased only modestly, from 0 to 2. More experience with the application process may generate a more positive outcome over time. 5.3.3 New Applicants Awards and applications data from NIH (described in detail in Chapter 3) suggest that about 40 percent of applicants for Phase I have not previously won an NIH SBIR award, and that about 30 percent of Phase I awards go to these companies. 9 The Federal and State Technology Partnership Program (FAST) Program is operated by the SBA, and provides states with a limited amount of matching funds to be used to strengthen the technological competitiveness of small business concerns in states. See <http://www.sba.gov/sbir/indexfast.html>.
OCR for page 136
An Assessment of the SBIR Program at the National Institutes of Health FIGURE 5-1 Percentage of winning companies new to the NIH SBIR program, 2000-2005. SOURCE: National Institutes of Health. Figure 5-1 shows that that the number of new winners has fallen slowly but steadily in recent years. However, this is likely explained by the fact that there are many more previous winners in the potential applicant pool each year. 5.3.4 Conclusions In general, the data above support the hypothesis that the NIH SBIR program is open to new companies, and continues to attract them, and that it is also open to companies from outside the major biomedical research hubs in states such as California, Massachusetts, and Maryland. However, it is also worth noting that some at NIH—including NCI in its institutional response to the NRC Program Manager Survey, suggested that funding for this outreach was severely constrained: We need to have annually committed funds to support a reasonable number of HSA and Grants Management staff to travel to the two national meetings as well as the annual NIH SBIR/STTR Conference which is now being held offsite. If the NIH Conference is held in Bethesda, then logistics funds are needed to support the Conference. Either funds should be made available from the SBIR/STTR set-aside for outreach, or Institutes should make a standing commitment to support these activities.10 10 NCI response to NRC Program Manager Survey, April 2006.
OCR for page 137
An Assessment of the SBIR Program at the National Institutes of Health 5.4 TOPICS Like other agencies, NIH publishes areas in which it is interested in funding research, known as “topics.” These topics are published in the annual NIH Omnibus Solicitation. But unlike the other SBIR agencies, where technical topic descriptions tightly limit awards, NIH topics are guidelines, not boundaries. The agency is proud of this “investigator-initiated” approach. Researchers are encouraged to submit applications on any topic that falls within the broad mandate of the IC funding agencies—which covers the entire universe of biomedical research. This description of the SBIR funding as “investigator-initiated” is broadly accurate. However, in recent years, an increasing percentage of awards have been made through alternative mechanisms. The Program Announcement (PA) mechanism operates through the regular selection procedure, but marks certain areas as being of special interest to NIH; the Request for Applications (RFA) mechanism goes further, and earmarks dollars within the SBIR set-aside specifically for selected topic areas. PAs now account for about 20 percent of Phase Is, and RFAs for a further 5 percent. These are discussed briefly below, and in more detail in Chapter 3. 5.4.1 Standard Procedure at NIH—The Omnibus Annual Solicitation The Annual Omnibus Solicitation lists all the topics from all of the ICs at NIH (and two other HHS SBIR participating agencies, CDC and FDA who use NIH to manage their SBIR program). The Solicitation describes areas in which research applications are encouraged, but applications outside these topic areas are welcomed. The topics listed in the annual solicitation are broad guides to the current research interests of the ICs. These topics are developed by individual ICs for inclusion in the annual Omnibus Solicitation. Typically, the NIH SBIR/STTR Program Office sends a request to the individual Program Administrators (PMs), the SBIR points of contact at each IC. These PMs in turn meet with division directors and determine the focus of SBIR topics within the IC. Division directors review the most recent Omnibus Solicitation (with their staff), and suggest changes and new topics based on recent developments in the areas of particular interest to the IC, or agency-wide initiatives with implications within the IC. The revised topics are then resubmitted for publication by the SBIR office at the Office of Extramural Research (OER), which provides a further review. 5.4.2 Procedures for Program Announcements (PAs) and Requests for Applications (RFAs) PAs and RFAs are NIH’s version of the mission-driven approach to topics used in particular by the procurement agencies—DoD and NASA. Essentially,
OCR for page 138
An Assessment of the SBIR Program at the National Institutes of Health they are tools through which the Institutes and Centers (ICs) can encourage firms to propose project that meet IC research priorities. RFAs are announcements of research funding areas that the IC expects to prioritize. The two types of announcement indicate different levels of IC interest. RFAs are high priority areas that have funding from SBIR set aside for them. In effect, they are operated much like the more rigid topics at other agencies. PAs are simply announcements of interest—applications received in response go through the same SBIR application process as other applications. However, as described in Chapter 3, ICs may announce that awards made under a PA can be for a longer time period (several additional years) and also for more money than the standard guidelines or even than the average award at NIH. While the PA applications go through the same selection process as other SBIR applications, IC’s may exercise discretion and decide to fund an application under a PA over other better-scoring applications. Discussions with agency staff suggest that this occurs only at the margin (i.e. a decision between two projects both close to the payline). RFAs indicate more interest from the IC in two respects. First, applications in response to a RFA compete for a separate pool of SBIR funding that the IC carves out of its general SBIR pool specifically to serve the RFA. Second, these applications are not selected using the normal Center for Scientific Research (CSR)11 process. Instead, RFA applications go through a separate review process, normally internal to the relevant IC. Both PA and RFA announcements are published by one or more ICs and reflect the top research priorities at the ICs. NIH tries to ensure that while PAs and RFAs define a particular problem, they are written broadly enough to encompass multiple technical solutions to the defined problem. PAs and RFAs appear to be the result of efforts to develop a middle ground between topic-driven and investigator-initiated research. Essentially, by layering PA/RFA announcements on top of the broad, standard solicitation, NIH seeks to focus some resources on problems that it believes to be of pressing concern, while retaining the flexible investigator-initiated approach that has served the agency well. In a recent interview, the NIH SBIR/STTR Program Coordinator indicated that NIH plans to increase the percentage of SBIR funds allocated to more targeted research through these mechanisms. 5.5 SELECTION The peer review process at NIH is by far the most elaborate of all the SBIR agencies. It is operated primarily through the Center for Scientific Research 11 The Center for Scientific Research manages the review process for all NIH awards, except the small number managed in-house by individual ICs (such as the SBIR RFAs).
OCR for page 139
An Assessment of the SBIR Program at the National Institutes of Health (CSR). CSR is a separate IC which serves only the other ICs—it has no direct funding responsibilities of its own. The system has been criticized on a number of fronts, most notably for being inhospitable to innovation,12 and because in tests of peer review processes elsewhere in biomedical research a significant degree of randomness in results has been identified.13 Nonetheless, peer review is deeply entrenched at NIH, and the selection of SBIR awards at NIH operates through the peer review that has been implemented agency wide. 5.5.1 Study Sections Applications for NIH SBIR awards are received at CSR and are assigned to a particular study section (as review panels are known at NIH) based on the technology and science involved in the proposed research. Panels can either be permanent panels legally chartered (established and defined) by Congress, or temporary panels designated for operation by NIH, called Special Emphasis panels (SEPs). Most SBIR applications are assigned to temporary panels, many of which specialize in SBIR applications only. Specialized panels at NIH are increasingly used because the requirements for assessing SBIR applications—notably the commercialization component—are quite different from the analysis required to assess the basic research conducted under other NIH grant programs. However, several respondents to the NRC Program Manager Survey at NIH noted that some study sections did consider all kinds of applications, and they did not believe this was the optimal way to review SBIR applications. A program manager at NCI observed that “More and more mixing of mechanisms is occurring in study sections once devoted to SBIRs, thus diluting the focus.”14 CSR is organized into four divisions, each of which is divided into Integrated Review groups (IRGs) by science/technology (e.g., infectious diseases, immunology). Each IRG manages a number of study sections.15 Neither CSR nor the study sections are organized by either disease or IC—they reflect scientific distinctions only. Special Emphasis Panels (SEPs) are reconstituted for each funding cycle. Almost all SBIR applications are reviewed by SEPs, which have a broader technology focus than the permanent chartered panels. Members can attend no more than 12 SEP study sections in 6 years. Section membership shifts with scientific trends. 12 D. F. Horrobin, “The philosophical basis of peer review and the suppression of innovation,” Journal of the American Medical Association, 263:1438-441, 1990. 13 T. Jefferson, et al., “Measuring the Quality of Editorial Peer Review,” Journal of the American Medical Association, 287:2786-2790, 2002. 14 Response to NRC Program Manager Survey, April 2006. 15 For example, the immunology IRG has seven permanent and two temporary study sections.
OCR for page 140
An Assessment of the SBIR Program at the National Institutes of Health The second kind of study section, known as chartered (permanent) study sections, usually has a narrow technical focus (e.g., host defenses, innate immunity). Most sections are chartered, and their members are semi-permanent; sitting for 4-8 years out of every 12. Most SEPs draw the majority of their applications from a subset of ICs. For example, the immunology IRG covers applications that refer to about 15 ICs, but 50 percent of its work comes from NAIAD, with a further 33 percent from NCI, reflecting the technical specialization of the SEP. NIH guidelines are that at least one panelist (member of the study section) should have small business background. However, some Scientific Review Administrators (SRAs) appear to be making a greater effort to get panelists with entrepreneurship experience. One recent panel, for example, had 13 small business representatives out of 25 panelists.16 That constituted a change for that panel: Previous panels in that technical area had been dominated by academics. NIH guidelines mandate 35 percent female and 25 percent minority panelists on each panel.17 There were numerous comments from agency staff and awardees about the difficulties of getting study sections with an appropriate mix of expertise. Some respondents to the NRC Program Manager Survey also focused on the need for more training for reviewers. Connie Dresser at NCI, for example, noted that “SBIR training needs to be mandatory for all SBIR reviewers in that they need to know what they should not be focusing on or why they should not be comparing SBIR content with R01 content. Also, we need people with marketing training and experience in review. The university types know text book information about marketing, not real-world marketing.”18 Other comments were more trenchant: “One basic flaw, in addition to the fundamental methodological deficiencies, is the reliance upon academic scientists to conduct reviews of SBIR-STTR applications. To put it simply: They are not qualified.”19 One additional point on this subject was made by an NIH staff member. She noted that the selection process would be improved by the addition of professional consumers of medical producers, e.g., users of MRI technology, as well as experts in its development.20 16 NIH staff interview. 17 See Center for Scientific Review, “Overview of Peer Review Process” for detailed discussion of the peer review process at NIH. <http://cms.csr.nih.gov/ResourcesforApplicants/PolicyProcedureReview+Guidelines/OverviewofPeerReviewProcess/>. 18 Response to NRC Program Manager Survey, April 2006. 19 Michael-David Kerns, NIA, Response to the NRC Program Manager Survey, April 2006. 20 Amy Swain, NCRR. Response to NRC Program Manager Survey, April 2006.
OCR for page 158
An Assessment of the SBIR Program at the National Institutes of Health FIGURE 5-5 Supplementary Phase I awards at NIH, 1993-2003. SOURCE: National Institutes of Health. in order to help a recipient pay for unexpected costs. While practices vary at individual ICs, it appears that up to 25 percent (or up to $50,000) of current annual funding for an individual grant can be awarded by the program manager without further IC or NIH review (budget permitting). More substantial supplements must be more extensively reviewed, but are not unknown. All supplemental requests require documentation. Full applications are required for competing supplements, and administrative supplements need at least a budget page and a letter justification. For Phase I, supplements remain relatively rare, averaging less than 20 annually in recent years. They are also not especially large, and in no cases have NIH Phase I supplements totaled more than $1 million for a given fiscal year, Still, the data indicate that the size of Phase I supplementary awards are growing at NIH (see Figure 5-5). Supplementary awards are also available for Phase II, where they are more significant. As shown in Figure 5-6, the number of Phase II supplement awards has hovered around 30. Thus about 10 percent of all Phase II awards receive supplementary funding. 5.7.3 Duration of Awards Just as the size of awards has grown, NIH has extended the period of support as well. In FY2002 and FY2003, more than 5 percent of all Phase I awards received a second year of support, with a median value of about $200,000. Year one and year two awards cannot be easily aggregated into a single
OCR for page 159
An Assessment of the SBIR Program at the National Institutes of Health FIGURE 5-6 Supplementary Phase II, Year One awards at NIH, 1992-2003. SOURCE: National Institutes of Health. “Phase I award” at NIH owing to characteristics of the NIH awards database. However, the rapidly growing number of year two awards—which in FY2003 were equal to 6.3 percent of all 2002 Phase I, year one awards—as well as the jump in median size in 2000, suggests that this mechanism is of growing importance at NIH. NIH staff and recipients alike agree that 6 months is too short to complete Phase I work in many biomedical disciplines. NIH usually approves requests for “no-cost” extensions to one year or even longer. No-cost extensions simply ex- FIGURE 5-7 Phase I, Year Two awards at NIH, 1992-2003. SOURCE: National Institutes of Health.
OCR for page 160
An Assessment of the SBIR Program at the National Institutes of Health FIGURE 5-8 Third year of support for Phase II awards, 1992-2003. SOURCE: National Institutes of Health. tend the term of the award without providing additional funding. No other agency offers such a liberal extension program. For Phase II, NIH also offers extended funding beyond the standard 24 months of support. Figure 5-8 contains estimates of Phase II, year three support calculated on the basis of NIH data (see Chapter 3 for detailed calculations). The steadily rising numbers of Phase II, year three grants in recent years suggest that third year support is becoming an important component of NIH SBIR activity. In FY2002 and FY2003, more than 10 percent of awards received a third year of support. In a few cases, NIH goes further. Ten grantees have received a fifth overall year of SBIR support, a few for even longer period. 5.7.4 Award Size: Conclusions The data shown in Chapter 3 indicate that the size of awards at NIH is rising, that additional administrative support is of increasing importance, and that the duration of awards (and support) is expanding as well. One important question might be why NIH is making these large awards. A second question might concern the growing number of extended awards. Both are discussed in Chapter 3, but conclusive answers are not available partly because neither question has been directly addressed by NIH, at least in materials that are publicly available. One final point should, be noted, drawn from conversations with agency staff and from responses to the NRC Program Manager Survey: NIH has repeatedly sought to convert Phase I STTR’s to Phase II SBIR’s and vice versa, as the
OCR for page 161
An Assessment of the SBIR Program at the National Institutes of Health circumstances related to the research change. SBA has denied these appeals, for reasons that are not clear to NIH staff. Unless SBA can find convincing justifications for this position, it would appear that a change of policy here could be warranted. 5.8 COMMERCIALIZATION SUPPORT 5.8.1 Background Since its inception in 1982, the SBIR program has aimed to increase “commercialization innovations derived from Federal research and development” (Public Law 97-219). After reauthorization in 1992, agencies were required to consider commercialization potential as part of its review process. The reauthorization also included a provision for technical assistance services to help grantees “develop and commercialize new commercial products and processes.” SBA then issued a rule stating that assistance efforts focused on bringing products to market could be supported by up to $4,000 per Phase I award and up to $4,000 per year for each Phase II award. Subsequent interpretations of the rule by SBA supported aggregation of these funds for an SBIR technical assistance program. 5.8.2 Overview NIH has recognized that many SBIR Phase II winners struggled to survive the period between the end of SBIR Phase II and market entry, and in June 2002, the Office of Extramural Programs at NIH (OEP) began to provide commercialization assistance to SBIR winners in June 2002. This assistance is now rendered through the Technical Assistance Program (TAP). Thus far, OEP has initiated three pilot assistance programs and two follow-on, full-scale assistance program under the TAP: The Pilot NCI Commercialization Assistance Program (PCAP) supported 47 SBIR Phase II winners (related to NCI only) and concluded in March 2003. The Pilot Niche Assessment Program (PNAP) was made available to a maximum of 100 SBIR Phase I winners on a first-come, first-serve basis. The pilot program had finished assisting 45 projects as of February 16, 2005, and ended in August 2005. Pilot Manufacturing Assistance Program. In FY2007, NIH plans to pilot an additional assistance program targeting the many manufacturing issues small companies face when trying to commercialize their SBIR-funded products. In partnership with the NIST Manufacturing Extension Partnership (MEP) program, the pilot is aimed at providing transitional support as Phase II awardees move to a manufacturing stage. The goal
OCR for page 162
An Assessment of the SBIR Program at the National Institutes of Health is to help companies make better decisions when developing their operational transition strategies (method of scale up, cost estimation, quality control, prototyping, design for manufacturability, facility design, process development/improvement, vendor identification and selection, plant layout, etc.) NIH has engaged Dawnbreaker of Rochester, NY, to operate this program. Twenty-five (25) NIH SBIR Phase II awardees are expected to participate. The Commercialization Assistance Program (CAP) was launched in July 2004 as the first full-scale, ongoing commercialization assistance program. Two cohorts of 114 firms each have completed the program as of January 2007. 5.8.3 The Commercialization Assistance Program (CAP) The perceived success of PCAP prompted OEP to launch the Commercialization Assistance Program (CAP) as its first full fledged, ongoing TAP “menu” item. It is open to companies funded by all NIH ICs. Larta Institute (Larta) of Los Angeles, CA,40 was selected by a competitive process to be the contractor for this program.41 The Larta contract began in July 2004, and will run for five years. During the first three years, three cohorts of SBIR Phase II winners will receive assistance. Years four and five will cover follow-up work, as each cohort is tracked for 18 months after completion of the assistance effort. CAP Program details. The assistance process for each group typically includes: Provision of consultant time for business planning and development. Business presentation training. Development of presentation materials. Participation in a public investment event organized by Larta. Eighteen months for follow-up and tracking. Participants. Based on interviews with NIH staff and Larta, the typical CAP participant is: A small technology-oriented business; Founded by an engineer or physician turned entrepreneur; In operation for 5 to 10 years; and 40 Larta Web site, accessed at: <http://www.larta.org>. 41 Larta was founded by Rohit Shukla who remains as its Chief Executive Officer. It assists technology oriented companies by bringing together management, technologies, and capital to accelerate the transition of technologies to the marketplace.
OCR for page 163
An Assessment of the SBIR Program at the National Institutes of Health FIGURE 5-9 CAP participants, by industry sector. SOURCE: National Institutes of Health. Substantially reliant on government grants because of limited outside funding. These companies have typically not yet generated meaningful sales, but appear to have significant commercial upside. As of January 1, 2004, NIH had 634 active SBIR Phase II projects from 455 companies across 23 Institutes and Centers. All of these companies were invited to participate in the CAP program42 and a total of 114 companies participated. Approximately 75 chose to participate in a series of investment workshops offered in Orange County, CA; San Francisco, CA; Washington, DC; Chicago, IL; and Boston, MA, which allowed participants to present their respective business opportunities to a group of investors, and to receive feedback on the effectiveness of their presentations. Participation by industry. The two largest industry sectors in CAP are Medical Devices (37 or 29 percent of total participants) and Biotech (29 or 23 42 NIH SBIR Technical Assistance Program, Office of Extramural Programs, Enrollment Criteria.
OCR for page 164
An Assessment of the SBIR Program at the National Institutes of Health percent of total participants). The Northeast region accounts for 35 percent of total participants and the West 32 percent of total participants.43 Areas of focused assistance. Three primary “Tracks,” areas of focused assistance, were added by NIH after the pilot based on participant feedback. The three tracks are: The Regulatory Track, for participants in need of a strategy for FDA approval. The Licensing Track, for participants in need of documentation for establishing relationships with potential licensees. The Strategic Alliance Track, for participants in need of documentation for establishing joint ventures, collaborative agreements, or other similar partnerships. Each Track is further adapted to the special needs of two industry sectors: Biomedical Devices (includes all medical devices and device-based products) and Biotechnology (includes all drugs and biologic-based products). The distribution of the current CAP participants by “Track” is represented in Figure 5-10.44 5.8.4 Niche Assessment Program (NAP) (for Phase I Winners) Sometimes scientific researchers do not have the entrepreneurial skills to assess other applications or niches for their SBIR-developed technology. As a result, they may underestimate its true market value. This program assesses the market opportunities and needs and concerns of the end-users and helps to discover new markets for possible entry. The NAP aims to assist SBIR Phase I winners in identifying and evaluating various market opportunities for commercialization (e.g., licensing, sales, partnering). This effort is operated by Foresight Science and Technology, Inc. (Foresight) of New Bedford, MA.45 It has three phases: Foresight gathers relevant information on the technology from the participant and begins to identify potential commercial applications. Foresight and the participant determine the technology application that warrants detailed analysis. This application is analyzed by Foresight to determine end-user needs, current and emerging competing technologies, market dynamics, socioeconomic trends, market drivers, market size, the 43 NIH CAP Participants by State, March 1, 2005. 44 Update, SBIR Technical Assistance Program, February 16, 2005. 45 Foresight is a scientific consulting firm offering market research, technology assessment, and valuation and licensing services to the medical, pharmaceutical and biotechnology industries. They focus on helping move technology from the laboratory to the marketplace and assess approximately 300 new technologies annually.
OCR for page 165
An Assessment of the SBIR Program at the National Institutes of Health FIGURE 5-10 CAP distribution by track. SOURCE: National Institutes of Health. potential technology’s possible market share, potential technology’s current competitive advantages, and strategies for improving the technology’s competitiveness. Foresight develops a market entry strategy including how to market the technology to end-users and attract Phase III partners. The strategy also projects revenues from the sale or licensing of the technology, possible “launch” customers, testing centers, suppliers, manufacturers, and other parties potentially interested in the technology (e.g., beta testers). Foresight may also make introductions to potential partners. Each step concludes with an electronic report plus follow-on discussions. 5.8.5 Outcomes and Metrics 18.104.22.168 Pilot NCI Commercialization Assistance Program Evaluations were completed at 6, 12, and 18 months following culmination of this pilot program, when 32 participants presented at an investor/partner Forum in March 2003.
OCR for page 166
An Assessment of the SBIR Program at the National Institutes of Health Participants were not obligated to provide feedback. However, 13 (40 percent) of the 32 companies reported that they had received additional private sector investment and/or sales related to the technology opportunity they presented at the Forum. Cumulative private sector funding and sales received within 18 months following completion of this program totaled almost $38 million.46 Unsurprisingly, these results were highly skewed: A majority of these funds were received by five of the companies—Computer Science Innovations, Focus Surgery, High Throughput Genomics, Phoenix Pharmacologics, and Vaccinex. Approximately $18 million—or about 47 percent of the total—was generated through the sale of one of these companies.47 Of course, this minimal assessment does not provide or even suggest grounds for a causal link between the program and these results. 22.214.171.124 Commercialization Assistance Program Two cohorts (2004/2005 and 2005/2006) have completed the CAP training program, and results have been very encouraging though not yet definitive. Evaluation data are collected from the companies at the conclusion of the program, and at 6, 12, and 18 months afterwards. These data indicate that firms going through the CAPM program are attracting funding, as Table 5-4 illustrates. NIH has also developed some intermediate metrics that indicate project impact. However, as these metrics are not compared with other groups of companies that have not gone through the CAP program, it is difficult to draw conclusions from them. Data collected six months after the CAP showed a strong increase in commercialization, and in particular in the conclusion of commercialization agreements, which increased for the 2004/2005 cohort by 87 percent (up from 23 at the baseline to 43 6 months later). These data are encouraging, and are bolstered by discussions with individual participants that indicate that participants find this program to be of considerable value. Development of a control group of some kind would add considerably to the power of this analysis. 5.9 EVALUATION AND ASSESSMENT Traditionally, NIH has not conducted outcomes assessment on its SBIR and STTR programs, or indeed on other programs. More recently, the NIH SBIR/STTR Program Office has initiated a number of activities aimed at infusing more data into the operation of the program, Most notably, in 2003 NIH followed on from its agreement to fund the NRC study with a separate NIH Survey of Phase II 46 NIH Office of Extramural Programs. 47 OER would not disclose the exact details of these outcomes citing confidentiality restrictions.
OCR for page 167
An Assessment of the SBIR Program at the National Institutes of Health TABLE 5-4 Funding for CAPM Firms Year 2004/2005 2005/2006 Number of companies in CAP 114 114 Number receiving investments 24 13 Percent of total 21.1 11.4 Total investment to date $22,414,078 $45,636,520 SOURCE: National Institutes of Health. recipients. Using somewhat different methodologies from the NRC Phase II Survey, with concomitantly different strengths and weaknesses, the NIH Survey broke important ground, and provided results that have been used throughout this analysis. Discussions with agency staff and responses to the NRC Program Manager Survey indicate widespread views that the program does not have the resources needed to develop an evaluation and assessment program sufficient to manage a program of this size and scope. Phil Daschner, from NCI, for example noted that “More resources should be available to program staff that track and evaluate objective benchmarks for past institutional and investigator productivity.” In its institutional response to the NRC survey, NCI observed that “we still do not have reliable tools to capture in an ongoing way success stories from our grantees. It is a considerable undertaking to get evaluation funds and go through the OMB process. Methods have been identified to capture outcomes, but funds are not FIGURE 5-11 Aggregate number of partnership- and deal-related activities by category. SOURCE: National Institutes of Health.
OCR for page 168
An Assessment of the SBIR Program at the National Institutes of Health available to support a sustainable effort to track SBIR/STTR outcomes. This is a critical and long-term need.”48 More specifically, as NIDA noted, “More time should be spent following up on grants near their end and after they no longer received NIH funding. We know little about Phase III and whether or not it actually occurs. Most time is spent funding the grant and administering it, but little or no time is spent on follow-up and evaluation.”49 Currently, the NIH SBIR/STTR Program Office must seek one-time funding for any significant assessment activity; this largely precludes longitudinal approaches needed for effective use of evaluation and assessment. 48 NRC Program Manager Survey, April 2006. 49 Ibid.