National Academies Press: OpenBook

SBIR/STTR at the National Institutes of Health (2015)

Chapter: 5 Quantitative Outcomes

« Previous: 4 SBIR and STTR Awards at NIH
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×

5

Quantitative Outcomes

This chapter analyzes outcomes related to the efforts by NIH to address the congressional mandate to increase commercialization of federally funded research and to stimulate technological innovation under its SBIR and STTR programs.1

This chapter reviews quantitative data provided through the 2014 Survey of Phase II SBIR/STTR recipients at NIH. It focuses primarily on the commercialization outcomes and knowledge effects from SBIR/STTR awards, as well as the longer term impact on the companies themselves. The chapter is followed by an Annex that includes more detailed descriptions and analysis.

There is evidence that NIH has achieved some success in tracking outcomes of SBIR/STTR awards, but more work needs to be done. NIH is currently building electronic links to the Small Business Administration (SBA) outcomes database now under development and working to develop its own outcomes tracking system (see Chapter 3); however, data from these sources are not yet available. Thus, the analysis of outcomes in this report is based primarily on the 2014 Survey by the Academies, which tried to survey all SBIR and STTR Phase II awardees for the period FY2001-2010 inclusive.2

A detailed description of the methodology underlying the 2014 survey is provided in Appendix A of this report (see Box 5-1). The full text of the survey is provided in Appendix C. Overall, the survey of NIH Phase II SBIR/STTR award recipients generated 726 responses. In cases where company information, as opposed to individual project information, was collected, multiple responses

_______________

1The participation of women and minorities is discussed in Chapter 6, and although some data are presented here on SBIR/STTR and agency mission, those are discussed further in Chapter 2.

2See Appendix A for a detailed description of the survey methodology used in this report.

Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×

BOX 5-1
Workshop on Improving University-SBIR/STTR Linkages

A workshop convened on February 5, 2014, by the committee considered a range of issues concerning universities and the SBIR/STTR programs.a Participants at this workshop addressed a range of topics including:

  • Improving linkages between SBIR/STTR programs at agencies and the universities,
  • Aligning with university accelerator initiatives,
  • Supporting improved links between state and local innovation and entrepreneurship programs and the universities, and
  • Supporting shifts in culture at universities to incentivize faculty to pursue SBIR/STTR funding.b

_______________

a See http://sites.nationalacademies.org/PGA/step/sbir/PGA_086819.htm.

b These issues and others related to the SBIR/STTR programs and universities will be addressed in detail in the upcoming NASA report on the STTR program.

from the same company were aggregated and then averaged to provide a better view of company-level activities.

A more detailed presentation of the data collected via the survey, including response rates, is included at the end of this chapter in Annex 5-A.

COMMERICIALIZATION

As with our other reports on the SBIR program, we have adopted a broad view of commercialization, taking it to include additional investments from outside the SBIR/STTR programs as well as sales and licensing revenues. In addition, given the long time to market required for many life sciences technologies, we have been careful to include a range of benchmarks and metrics, having determined that no single metric can appropriately capture such a broad concept.

That said, we focus first on different ways of measuring sales and other types of commercial revenue as well as further investment. In line with previous studies by the Academies and consistent practice at all agencies, investment beyond Phase II is recognized as acknowledgement by third parties that the project has developed technologies of marketable value. For many projects, further investment is required before commercial sales can begin. An extended discussion of approaches to measuring commercialization is contained in the Annex to this chapter.

NIH is also in some ways a special case for commercialization. Because NIH is not an agency where the SBIR/STTR programs are designed to generate

Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×

technologies for use by the agency itself, markets have to be found outside the agency. The path to such markets is particularly difficult for the large percentage of projects that require U.S. Food and Drug Administration (FDA) approval; such approval was required for 45 percent of the surveyed projects. Regulatory approval is expensive—sometimes extremely expensive—and time consuming. These are two formidable challenges for small companies. It is therefore especially important to capture milestones along the way to commercialization as well as commercial sales and related revenues.

Sales and Revenues

Perhaps the single most used metric for assessing SBIR-type programs is revenue or licensing fees. As recommended in a previous Academies’ report3 overreliance on this particular metric may lead to incorrect conclusions about the program, although they are important considerations.

Reaching the Market

The first question in this section concerns reaching the market: Did the project generate any sales, and if not, are sales expected (a necessary question given the long cycle time of some projects)? Responses are summarized in Figure 5-1.

Overall, just less than one-half of projects reported some sales or licensing revenues, and a further 25 percent expected sales in the future. These data are similar to those generated by the previous survey of NIH SBIR-only awardees by the National Research Council (NRC)4 in 2005.5

Amount of Sales and Licensing Revenues

Simply identifying the percentage of projects reaching the market is an important metric, but it is not sufficient; it is also necessary to understand the scale and distribution of sales. The 2014 survey asked those who reported some sales of the technology developed for the surveyed project to report the amount of sales, grouped into ranges. These data are summarized in Figure 5-2.

Most reported sales at the lower end of the scale: 62 percent were less than $500,000 and more than onehalf of those sales (39 percent of the total) were under $100,000. Four percent reported revenues of at least $20 million, while 8 percent reported sales of between $5 million and $20 million.

_______________

3National Research Council, An Assessment of the SBIR Program at the National Institutes of Health, Washington, DC: The National Academies Press, 2009.

4Effective July 1, 2015, the institution is called the National Academies of Sciences, Engineering, and Medicine. References in this report to the National Research Council or NRC are used in an historic context identifying programs prior to July 1.

5National Research Council, An Assessment of the SBIR Program at the National Institutes of Health.

Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×

images

FIGURE 5-1 NIH SBIR/STTR sales and licensing revenues (percentage of respondents).
NOTE: N=602. See Table 5-7 in the chapter annex for details.
SOURCE: 2014 Survey, Question 32.

Markets by Sector

The 2014 Survey also asked respondents about the market sectors in which sales were made. Overall, 58 percent identified the private sector, followed by export markets (17 percent). Sales to federal agencies comprised 6 percent of sales (see Figure 5-3).

Further Investment

The ability of SBIR/STTR projects and companies to attract further investment has traditionally been an important measure of SBIR/STTR commercialization outcomes.6 There has also been interest in the sources of additional funding for high-tech innovation. Although the United States has historically been a leader in venture capital and angel investment, these are not the only or even the primary sources of additional investment funding for NIH SBIR/STTR projects.

Overall, more than 80 percent of respondents indicated that their project received additional investment in the technology related to the surveyed project.7 As

_______________

6See National Research Council, An Assessment of the SBIR Program, Washington, DC: The National Academies Press, 2008.

72014 Survey, Question 30. N=572.

Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×

images

FIGURE 5-2 Distribution of total sales and licensing revenues by range (percentage of respondents reporting sales).
NOTE: N=263 (projects reporting sales). See Table 5-8 in the chapter annex for details.
SOURCE: 2014 Survey, Question 34.

with prior surveys, the data show small amounts of additional funding are most likely. Table 5-1 shows the amount of funding received. About 76 percent of all projects received less than $1 million in additional investment with the median amount of additional funding equaling $300,000. Four respondents (less than 1 percent) reported receiving $50 million or more in additional funding, while 9 percent reported receiving $5 million or more. These data highlight funding challenges for these companies because the cost of Phase III clinical trials has recently been estimated at $26,000 per patient.8 Phase III trials can require the enrollment of more than 1,000 patients.9

Of the 470 respondents that reported additional funding, 44 percent was from private-sector sources, 9 percent was from venture capital sources, and 14 percent was from angel and other private equity investors. Twenty-one percent reported strategic investments from partners, which is especially important in the context

_______________

8Jon Hess, “Clinical Operations: Accelerating Trials, Allocating Resources and Measuring Performance,” Cutting Edge Information, October 12, 2014.

9Avik S. Roy, “Stifling New Cures: The True Cost of Lengthy Clinical Drug Trials,” FDP Project Report 5, Manhattan Institute, April 2012.

Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×

images

FIGURE 5-3 Markets for NIH SBIR/STTR products and services: Percentage of total sales (mean of all responses/category).
NOTE: N=265 (projects reporting sales). See Table 5-9 in the chapter annex. For this question, each respondent reports a percentage distribution. Values above are calculated by deriving the mean value for all the responses received for each category.
SOURCE: 2014 Survey, Question 36.

TABLE 5-1 Additional Funding Received by Funding Mechanism and Amount

Percentage of Responses
NIH Total SBIR Awardees STTR Awardees Phase IIB Awardees
None ($0) 17.8 19.0 11.2   3.4
Under $100,000 24.3 23.2 30.3   6.9
$100,000$499,999 21.7 21.5 22.5 13.8
$500,000$999,999 11.7 11.4 13.5 20.7
$1,000,000$4,999,999 15.7 15.5 16.9 31.0
$5,000,000$9,999,999   3.5   3.7   2.2 10.3
$10,000,000$19,999,999   2.3   2.5   1.1 10.3
$20,000,000$49,999,999   2.3   2.3   2.2   3.4
$50,000,000 or more   0.7   0.8
BASE: TOTAL RESPONDENTS
ANSWERING QUESTION
572 483 89 29
 
Mean 2,560 2,698 1,813 4,666
Median    300    300    300 3,000

SOURCE: 2014 Survey, Question 30.

Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×

TABLE 5-2 Distribution of Responses Related to Additional Investment Funding by Source of Funds

Percentage of Responses
NIH Total SBIR Awardees STTR Awardees Phase IIB Awardees
Non-SBIR/STTR federal funds 25.2 21.9 41.4 28.6
Private Investment: U.S. Sources 44.2 45.7 36.8 67.9
Venture capital (VC)   9.5   9.5   9.2 21.4
U.S. angel funding or other private 13.6 14.5   9.2 14.3
equity investment (not VC)
Friends and family 11.4 11.4 11.5 10.7
Strategic investors/partners 20.5 21.0 18.4 17.9
Other sources   9.1   9.3   8.0 17.9
Foreign Investment   5.5   5.5   5.7
Financial investors   2.4   2.4   2.3
Strategic investors/partners   3.7   3.3   5.7
Foundations   3.6   2.9   6.9

SOURCE: 2014 Survey, Question 31.

of life sciences where large pharmaceutical and medical device companies are a critical part of the path to market (see Table 5-2).

Twenty-five percent of respondents reported funding from non-SBIR/STTR federal sources, which would include such potential funders as BARDA, the Centers for Medicare & Medicaid Services, and the Veterans Administration, or the Department of Defense.

As shown in Table 5-13 in the chapter annex, 24 percent of respondents reported funding from other external sources, including 16 percent from state and local governments and 9 percent from research institutions. Overall, the most utilized funding source was the company itself (58 percent) and in many cases personal funds (26 percent).

SBIR/STTR and Clinical Trials

Many NIH-funded projects face the challenge of proceeding through clinical trials before they can seek success in the marketplace. Survey responses indicate that this was true for 46 percent of projects.10 For these companies, the road to successful conclusion of clinical trials is very challenging (Table 5-3). More than one-third of projects had abandoned the process, and about another one-third

_______________

102014 Survey, Question 40. N=584.

Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×

TABLE 5-3 Outcomes for Projects Requiring FDA Approval

Percentage of Responses
NIH Total SBIR Awardees STTR Awardees Phase IIB Awardees
FDA process abandoned 35.5 35.9 33.3   5.3
Preparation under way for clinical trials 34.4 35.5 28.2 47.4
IND granted   4.7   4.1   7.7 10.5
In Phase 1 clinical trials   4.7   5.1   2.6
In Phase 2 clinical trials   9.4   7.4 20.5 15.8
In Phase 3 clinical trials   2.3   1.8   5.1
Completed clinical trials   9.0 10.1   2.6 21.1
BASE: NIH PROJECTS REQUIRING FDA APPROVAL 256 217 39 19

SOURCE: 2014 Survey, Question 41.
NOTE: IND refers to Investigational New Drug

were preparing for entry into clinical trials. Nine percent of projects had completed clinical trials (21 percent of Phase IIB projects).

These figures are difficult to interpret because the survey covered a 10-year period. Newer projects are more likely to be preparing for clinical trials or in earlier stages than are older projects. Still, it is fair to conclude that a majority of those requiring approval will not in the end receive it given that the probability of a project being abandoned increases with its age.

These data can be compared with the general set of outcomes for all efforts to receive FDA approval. Table 5-4 shows that 8 percent of those projects entering the clinical trials process eventually received approval, while 30 percent of projects were approved for Phase 1 clinical trials. Survey responses match these estimates quite closely, which is noteworthy given that Table 5-4 includes data from larger and much better funded companies.

SBIR/STTR companies utilize a wide range of funding sources to meet the demands of clinical trials. Figure 5-4 shows that the largest single source was internal company funding (51 percent) followed by SBIR/STTR itself (32 percent) and Phase IIB (22 percent). Angel funding, venture funding, and strategic funding from other companies were each mentioned by 15 percent of respondents.

For those receiving SBIR/STTR support of some kind for clinical trials, 48 percent thought that the funding was extremely or very useful, while 33 percent said it was not at all useful.11 Among those who received Phase IIB funding, positive responses were higher—more than three quarters of respondents said that

_______________

112014 Survey, Question 45. N=79.

Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×

TABLE 5-4 Outcomes for FDA Applications at Different Phases

Stage Overall Probability of Success Conditional Probability of Success Approximate Time
Preclinical
Toxicology 1 to 6 years
Clinical 6 to 11 years
Investigational New Drug Application 40%
Phase I 30% 75%
Phase II 14% 48%
Phase II 9% 64%
Approval 0.6 to 2 years
New Drug Application 8% 90%
Market 11 to 14 years
Phase IV/Post-Market surveillence

NOTES: “Overall probability of success” is the unconditional probability of researching a given stage. For example, 30 percent of drugs make it to Phase I testing. “Conditional probability of success” shows the probability of advancing to the next stage of the process conditional on reaching a given stage. For example, the probability of advancing to Phase III testing conditional on starting Phase II testing is 48 percent.

SOURCE: Adapted from Joseph A. DiMasi, Ronald W. Hansen, and Henry G. Grabowski, “The Price of Innovation: New Estimates of Drug Development Costs,” Journal of Health Economics 22:151-85, 2003.

it made a “tremendous difference,” while 24 percent said that it made little or no difference (N=21, so responses should be treated with caution).12

However, while the additional funding was regarded as helpful, it was not seen as sufficient. Figure 5-5 shows that of the 21 Phase IIB responses, one-third thought the funding was not sufficient even to complete preparation for clinical trials, and a further 29 percent thought it was sufficient only for completion of those preparations. Five percent (one respondent) thought it was sufficient for Phase 3 trials.

KNOWLEDGE EFFECTS

One of the four congressionally mandated objectives for the SBIR/STTR programs is to “stimulate technological innovation.” Although patents and peer-reviewed papers are not the only useful way to assess the development and transmission of knowledge by small high-tech companies, they offer a useful starting point.

_______________

122014 Survey, Question 46. N=21.

Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×

images

FIGURE 5-4 Sources of funding reported for FDA clinical trials (numbers of responses).
NOTE: N=226 (projects requiring FDA approval). Numbers do not sum to 226 because respondents could select more than one answer. BARDA is the Biomedical Advanced Research and Development Authority. It “provides an integrated, systematic approach to the development and purchase of the necessary vaccines, drugs, therapies, and diagnostic tools for public health medical emergencies.” http://www.phe.gov/about/BARDA/Pages/default.aspx, accessed August 8, 2015.
SOURCE: 2014 Survey, Question 4.

Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×

images

FIGURE 5-5 Sufficiency of Phase IIB funding for different phases of clinical trials (percentage of respondents).
NOTE: N=21.
SOURCE: 2014 Survey, Question 47.

Figure 5-6 shows the number of patents related to all SBIR/STTR awards reported by companies participating in the 2014 Survey. About two-thirds of respondent companies received at least one such patent, and about 13 percent received 10 or more.

The survey also asked about patents related to the specific project being surveyed. About 53 percent of respondents reported at least one patent related to the project, and 4 percent reported at least 10 (see Table 5-5).13

In addition to patents, the survey asked about articles in peer-reviewed journals. Meetings with company executives indicated that, for many companies, even though technical knowledge and trade secrets are very important, the company strongly supported peer-reviewed publication. In part, companies saw this as marketing among peers, both for eventual products and a means of attracting talent. Eighty percent of survey respondents reported publishing at least one peer-reviewed article related to the surveyed project. Forty-two percent reported publishing at least three articles, as shown in Figure 5-7. As shown in Table 5-19 in the annex section of this chapter, two-thirds of Phase IIB responses reported publishing at least 3 articles and 21 percent reported publishing at least 10.

_______________

132014 Survey, Question 39.1.2. N=186.

Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×

images

FIGURE 5-6 Number of patents reported related to all company SBIR/STTR awards (percentage of company-weighted responses).
NOTE: N=409.
SOURCE: 2014 Survey, Question 12.

TABLE 5-5 Number of Patents Received Related to Surveyed Project

Percentage of Responses
NIH Total SBIR Awardees STTR Awardees Phase IIB Awardees
0 47.0 47.2 46.3 25.9
1 23.7 23.0 27.5 22.2
2 12.0 12.1 11.3   7.4
3 or 4   8.2   7.6 11.3 22.2
5 to 9   5.2   5.9   1.3 14.8
10 or more   4.0   4.3   2.5   7.4
1 or more 53 52.8 53.8 74.1
 
Mean   1.63   1.71   1.23   3.11
Median   1   1   1   2
BASE: TOTAL RESPONDENTS
ANSWERING QUESTION
502 422 80 27

SOURCE: 2014 Survey, Question 38.1.

Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×

images

FIGURE 5-7 Number of peer-reviewed articles relating to surveyed project (percentage of respondents).
NOTE: N=508.
SOURCE: 2014 Survey, Question 38.

Another mechanism for knowledge transfer is the development of links between SBIR/STTR companies and their projects with research institutions (RIs). Sixty-five percent of SBIR respondents reported some connection to an RI. All STTR respondents are required to partner with an RI. Many reported that RI faculty worked on the surveyed project (39 percent used faculty as consultants), while a smaller number reported that the technology was originally developed at and/or was licensed from the RI (see Table 5-6).

COUNTERFACTUALS

Because there is no available matched set of companies that did not receive SBIR/STTR Phase II funding at precisely the point in time that surveyed companies did receive funding, it is not possible to develop an appropriate control group against which to measure impacts (see discussion of the Academies efforts to do so in Appendix A). However, it is at least possible to ask—as previous surveys by the Academies and the Government Accountability Office (GAO) have done—what the company itself believed might have happened had SBIR/STTR funding not been available. While this is of course subjective, the company is best suited to provide these answers.

Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×

TABLE 5-6 Connections to Research Institutions (RIs) and SBIR/STTR Awards

Percentage of Responses
NIH Total SBIR Awardees STTR Awardees Phase IIB Awardees
The PI for this project was at the time of the project an RI faculty member 17.4   6.1 77.8   3.6
The PI for this project was at the time of the project an RI adjunct faculty member 10.4 12.1   1.1 17.9
Faculty member(s) or adjunct faculty member(s) worked on this project in a role other than PI 38.7 38.2 41.1 35.7
Graduate students worked on this project 22.3 21.3 27.8 21.4
The technology for this project was licensed from an RI 16.0 14.0 26.7 17.9
The technology for this project was originally developed at an RI by one of the participants in this project 20.2 17.1 36.7 17.9
An RI was a subcontractor on this project 37.4 35.5 47.8 53.6
None of the above 29.7 35.1   1.1 32.1
BASE: TOTAL RESPONDENTS
ANSWERING QUESTION
569 479 90 28

SOURCE: 2014 Survey, Question 71.

Because alternative funding especially for long-cycle projects and those requiring FDA approval is difficult to acquire, it is not surprising that 7 percent of respondents believed that the project would definitely or possibly have proceeded without funding. Conversely, almost three-quarters of respondents said that it would likely or definitely not have proceeded.14

These data have interesting wider implications for debates about early-stage funding; notably, they suggest poor support for the “crowding out” hypothesis (that public funding displaces private investment). Awardees in our survey—presumably those with the closest knowledge of funding prospects for the project—overwhelmingly believed it to be unlikely that alternative private funding would be found. These results also underscore the importance of SBIR/STTR funding for these small companies.

_______________

142014 Survey, Question 24. See Table 5-31 in the chapter annex.

Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×

COMPANY IMPACTS

Although the effect of SBIR/STTR funding on the company is not directly included in the congressional objectives for the program, helping small companies to become self-sufficient (and in some cases to grow rapidly) does have implications for program impacts and are therefore included in our analysis.

Small high-tech companies are often fluid in structure, and the 2014 Survey found that many participating companies changed structurally in recent years. Thirty-five percent established strategic partnerships with major players, while 21 percent spun off at least one company and 16 percent were acquired by or merged with another firm.15

Ideally, companies that receive SBIR/STTR funding become more stable and develop contracts that are not SBIR-related over time. This appears to be the case for NIH SBIR/STTR companies as dependence on SBIR/STTR funding is limited. Overall, 42 percent of respondents indicated that the SBIR/STTR programs were currently funding 10 percent or less of the company’s total research and development (R&D) effort, while about 34 percent indicated that they were funding more than one-half.16 This picture is reinforced by data on sources of company revenues. Thirty-four percent of responding companies reported zero SBIR/STTR revenues, while about 27 percent reported receiving more than one-half of the company’s revenues from SBIR/STTR.17

The survey also asked about the overall impact of SBIR/STTR on the company. As Figure 5-8 shows, 62 percent saw a highly positive or transformative effect, and another 35 percent reported a positive impact. Two respondents reported a negative or highly negative impact.

_______________

152014 Survey, Question 11. N=436 (companies). See Table 5-14.

162014 Survey, Question 10. N=421 (companies). See Table 5-25.

172014 Survey, Question 9. N=409 (companies). See Table 5-26.

Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×

images

FIGURE 5-8 Long-term impact of SBIR/STTR on companies (percentage of respondents).
NOTE: N=580.
SOURCE: 2014 Survey, Question 57.

Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×

ANNEX 5-A: QUANTITATIVE OUTCOMES

This annex elaborates the results of the 2014 survey of quantitative outcomes of the NIH SBIR/STTR programs, summarized in Chapter 5. As noted earlier, this quantitative assessment of outcomes from the NIH SBIR/STTR programs focuses primarily on two of the four congressionally mandated objectives for the program: commercialization of federally funded research and stimulation of technological innovation. Data on the participation of women and minorities is included in Chapter 6. Data on program management is included in Chapter 2.

FOCUS ON COMMERICALIZATION OUTCOMES

Although there are four statutory goals for the SBIR/STTR programs, subsequent legislation passed by Congress, as well as administrative policies pursued by NIH and the other major SBIR/STTR agencies focus primarily on the commercialization of SBIR/STTR technologies.18 Moreover, given that commercialization is among the more measurable outcomes of the SBIR/STTR programs, it has become a primary benchmark for program performance. The focus on commercialization, however, should not eclipse the requirement that the program meet all four congressionally mandated objectives. This Annex provides additional details of the commercial outcomes of the NIH SBIR/STTR programs, as well as quantitative outcome measures related to stimulating technological innovation.

SOURCES OF DATA

Although NIH was an early adopter of survey-driven outcomes research, the agency has not thereafter led the way on tracking outcomes. Other agencies have moved more rapidly to meet the need for data:

  • The Department of Defense (DoD) maintains the Company Commercialization Record, which requires all companies applying for DoD SBIR/STTR funding to update outcomes for all prior awards.
  • The National Science Foundation (NSF) utilizes a consultant to undertake phone interviews with recipients at set times several years after the end of the award.
  • The Department of Energy (DoE) internally tracks award outcomes for several years using its own metrics and methodologies.
  • The National Aeronautics and Space Administration (NASA) developed a tracking module as part of its Electronic Handbook, which has been used in recent years collect outcomes data.

_______________

18SBA Section 1.(c), SBIR/STTR Policy Directive, October 18, 2012, p. 3.

Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×

NIH managers are aware of this situation and have recently moved to correct this deficiency. According to the NIH SBIR/STTR Program Office, NIH has undertaken a twin-track approach. It has built an electronic bridge between the new SBA outcomes database (which is not yet online as of this writing). In addition, it is investing in a new module for its own internal Performance Outcomes Data System (PODS) database, which will address outcomes. These issues are discussed in more detail in Chapter 2 (Program Management).

Given the lack of available data from NIH, the quantitative data presented in this chapter are derived from the 2014 Survey of award recipients. We stress, however, that these data are descriptive only and should be regarded as providing insights into outcomes rather than definitive conclusions.19

The 2014 Survey is based primarily on the 2005 survey, with some additions and modifications. This 2014 survey was sent to two distinct populations: (1) all principal investigators (PIs) who received an NIH SBIR or STTR Phase II award between FY2001 and FY2010 inclusive and (2) in cases where PIs could not be reached, alternate company contacts at the targeted companies. Results from this survey provide quantitative insights that permit the analysis provided in this chapter. They are, where appropriate, compared to the results from the 2005 Survey.20

Appendix A provides a detailed discussion of the survey methodology, including response rates and potential survey bias. Below, a series of tables summarize Phase II responses for SBIR, STTR, and Phase IIB recipients.21 The 2014 Survey is reproduced in Appendix C. References to Phase II refers to SBIR and STTR awards, not including Phase IIB.

OUTCOMES

Commercialization

At NIH, the priority for SBIR/STTR is to support the development and commercialization of technologies that will improve the nation’s health. In contrast to DoD and NASA, it is not expected that SBIR/STTR technologies will be used by the agency itself. Sales are primarily made into the domestic private sector, although sales to other health-related government agencies are also quite substantial.

NIH SBIR/STTR companies also face a particular challenge, aside from the difficulties faced by all small companies as they seek to gain market traction for

_______________

19The committee previously sought to develop statistical comparisons with similar companies in similar sectors at similar stages of development, but these efforts were eventually abandoned as unworkable. See Appendix A for a discussion of this effort. A full description of the methodology employed for this survey and the resulting analysis is also provided in Appendix A.

20All comparisons to the 2005 survey are based on data from tables and analysis in National Research Council, An Assessment of the Small Business Innovation Research Program at the National Institutes of Health, Washington, DC: The National Academies Press, 2009, Chapter 4 and Appendix B.

21Phase IIB is an award made to some firms at the end of Phase II. It supports those working to complete clinical trials. A discussion of Phase IIB is included in Chapter 3 (Program Initiatives).

Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×

their products. Survey responses indicate that nearly one-half of respondent companies anticipated that they would need to successfully complete FDA clinical trials before their product could be marketed. Academic papers indicate that about 8 percent of products that file for clinical trials make it to the end of Phase 3, so these challenges are real and pervasive for these companies.

That said, SBIR/STTR program participants at NIH are—as at other agencies—small for-profit companies, and they must proceed in ways that do in the end provide a sustainable path forward for the business.

Defining “Commercialization”

Several important conceptual challenges emerge when seeking to define “commercialization” for the purposes of the SBIR/STTR programs. Like many apparently simple concepts, commercialization becomes progressively more difficult and complex as it is subjected to further scrutiny. For example:

  • Should commercialization include just sales or other kinds of revenue, such as licensing fees and funding for further development?
  • Should commercialization include only certain kinds of sales—excluding for example sales to government agencies?
  • What is the appropriate benchmark for sales? Is it any sales whatsoever, sufficient sales to cover the costs of awards, sales that lead to breaking even on a project, or sales that reflect a commercial level of success and viability? The last at least would likely be different for each project in each company.
  • Should commercialization include sales by licensees, which may be many multiples of the revenues provided to, but are largely reported by, the licensors?

For the purposes of this study, we deployed a broad net to capture a range of potentially useful data. Once acquired, these data can be analyzed in a variety of ways to provide multiple insights into this complex topic.22

Sales and Revenues

Perhaps the single most used metric for assessing SBIR-type programs is revenue or licensing fees. Although we have already cautioned against overuse of this metric—warnings that are applicable to the wide range of metrics adopted for use in the current assessment—sales and revenues are still important considerations.23

_______________

22For an overview of the commercialization metrics and survey used in this study, see Appendix A.

23Similar warnings can be found in the 2009 report on the NIH SBIR program—National Research Council, An Assessment of the SBIR Program at the National Institutes of Health, 2009, 81.

Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×

Reaching the market. The first survey question in this area concerns reaching the market: Did the project generate any sales, and if not, are sales expected (a necessary question given the long cycle time of some projects)? Responses are summarized in Table 5-7. About one-half of respondents reported some sales or licensing revenues, and a further one-quarter expected sales in the future. The percentage reporting sales to date was lower than for the 2005 Survey (57 percent). Those expecting sales in the future increased from 19 percent for the 2005 Survey to 25 percent (SBIR-only) for the 2014 Survey.

The 2005 Survey of SBIR companies found that 24 percent of respondent companies had no sales and expected none, 19 percent had no sales but expected sales in the future, and 57 percent had already generated sales from the surveyed project.24

Amount of sales and licensing revenues. Simply identifying the percentage of projects reaching the market is an important metric, but it is not sufficient. In addition, it is important to understand the distribution of sales. The survey also asked those who reported some sales of the technology developed for the surveyed project to report the amount of sales, segregated by tiers. These data are summarized in Table 5-8. Overall, 26 percent of SBIR respondents reported sales of at least $1 million.

Markets by sector. Because NIH is not itself a significant market, it is not surprising that most sales are made to the domestic private sector. Furthermore, because health care products are increasingly a global business, it is also not surprising that export markets accounted for 17 percent of sales (see Table 5-9). These figures are very similar to those for the 2005 Survey.25

Employment

As with prior surveys, the 2014 Survey asked respondents both about the size of the company at the time of the award and the current size, in terms of number of employees. At the time of the award, 60 percent of responding companies had fewer than 10 employees. The median was 7 employees (see Table 5-10).

The survey also asked about current employment (as of 2014). Respondents reported that the median size of companies was still 7 employees, but that the mean size had grown significantly, from 19 to 88. Among the 7 percent of

_______________

24National Research Council, An Assessment of the SBIR Program at the National Institutes of Health, 249.

25National Research Council, An Assessment of the SBIR Program at the National Institutes of Health, 251.

Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×

TABLE 5-7 NIH SBIR/STTR Sales Outcomes

Percentage of Responses
NIH Total SBIR Awardees STTR Awardees Phase IIB Awardees
No sales to date 50.8 50.9 50.5 48.3
No sales to date nor are sales expected 25.9 25.9 25.8 10.3
No sales to date, but sales are expected 24.9 25.0 24.7 37.9
Any sales to date 49.2 49.1 49.5 51.7
Sales of product(s) 38.7 38.3 40.9 48.3
Sales of process(es)   2.8   2.8   3.2
Sales of services(s) 17.4 17.7 16.1 17.2
Other sales (e.g. rights to technology, licensing, etc. )   9.1   8.4 12.9   6.9
BASE: TOTAL RESPONDENTS
ANSWERING QUESTION
602 509 93 29

NOTE: Respondents could report multiple types of sales for a single project, so the types of sales do not sum to “Any sales to date.”
SOURCE: 2014 Survey, Question 32.

TABLE 5-8 Distribution of Total Sales Dollars, by Range

STTR SBIR Phase IIB
Under $100,000 34.1 40.6 23.1
$100,000$499,999 22.0 22.6 23.1
$500,000$999,999 17.1 10.4
$1,000,000$4,999,999 14.6 14.2 23.1
$5,000,000$9,999,999   6.1   7.7
$10,000,000$19,999,999   2.4   2.3
$20,000,000$49,999,999   2.4   1.9   7.7
$50,000,000 or more   1.9 15.4
BASE: ANY SALES RESULTING FROM THE PROJECT 38 212 13

NOTE: See Pie Chart in Figure 5-2.
SOURCE: 2014 Survey, Question 34.

Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×

TABLE 5-9 Markets by Sector

Percentage of Responses
NIH Total SBIR Awardees STTR Awardees Phase IIB Awardees
Domestic private sector 57.5 57.1 60.1 49.2
Export Markets 16.9 16.3 20.5 34.6
DoD/NASA/Primes   2.0   2.0   1.4   0.0
NIH   0.9   0.7   1.8   0.4
Other federal agencies   3.4   3.4   3.1 12.0
State or local governments   4.2   4.8   0.7   2.3
Other (Specify below, if applicable) 15.2 15.6 12.4   1.5
BASE: ANY SALES RESULTING FROM THE PROJECT 265 228 37 13

SOURCE: 2014 Survey, Question 36.

TABLE 5-10 Number of Employees at Time of Award

Percentage of Companies Responding
NIH Total SBIR Awardees STTR Awardees Phase IIB Awardees
0   0.5   0.3   1.6
1   3.7   3.1   6.8
2   7.1   6.1 12.5   5.7
3 or 4 19.7 19.5 20.8
5 to 9 29 27.7 35.9 35.4
10 to 19 19.2 20.1 14.1 24.9
20 to 49 12.5 14   4.4 12.4
50 to 99   3.6   3.7   3.1   8.6
100 or more   4.8   5.5   0.8 12.9
Mean 19 21 10 33
Median   7   8   5 12
BASE: TOTAL RESPONDENTS
ANSWERING QUESTION
418 354 64 17

NOTE: Answers from individual respondents were aggregated and averaged for each company, and company responses are reported above.
SOURCE: 2014 Survey, Question 14.1.

Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×

TABLE 5-11 Employment at Time of Survey

Percentage of Companies Responding
NIH Total SBIR Awardees STTR Awardees Phase IIB Awardees
0 13.3 12.5 17.3 14.4
1   5.6   5.8   4.8
2   7.0   6.2 11.7   5.7
3 or 4 14.2 14.3 13.7   2.9
5 to 9 19.7 19.0 23.6 20.1
10 to 19 18.0 18.1 17.2 18.2
20 to 49 11.9 12.7   7.7 17.2
50 to 99   2.8   3.3   5.7
100 or more   7.4   8.0   4.0 15.8
Mean 88 101 16 47
Median   7   7   5 16
BASE: TOTAL RESPONDENTS
ANSWERING QUESTION
410 347 63 17

NOTE: Answers from individual respondents were aggregated and averaged for each company, and company responses are reported above.
SOURCE: 2014 Survey, Question 14.2.

firms now reporting at least 100 employees, some had grown substantially (see Table 5-11).26

Further Investment

The ability of SBIR/STTR projects and companies to attract further investment has traditionally been a defining metric for SBIR/STTR outcomes.27 There has also been interest in the sources of additional funding for high-tech innovation. The United States has historically been a leader in venture capital and angel investment.

_______________

26Of related interest, see Link and Scott, Employment Growth from Public Support of Innovation in Small Firms (W.E. Upjohn Institute for Employment Research, 2012). The authors use the 2005 data and compare actual employment growth with a prediction of what would have been expected to happen without the SBIR award. Interestingly, comparing the counterfactual gains from the SBIR program across agencies, they find that the gap between the actual employment and the counterfactual predicted employment (if the SBIR award had not been received) is greatest for the NIH awards.

27See National Research Council, An Assessment of the SBIR Program, Washington, DC: The National Academies Press, 2008.

Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×

TABLE 5-12 Additional Funding by Phase and Amount

Percentage of Responses
NIH Total SBIR Awardees STTR Awardees Phase IIB Awardees
None ($0) 17.8 19 11.2   3.4
Under $100,000 24.3 23.2 30.3   6.9
$100,000$499,999 21.7 21.5 22.5 13.8
$500,000$999,999 11.7 11.4 13.5 20.7
$1,000,000$4,999,999 15.7 15.5 16.9 31
$5,000,000$9,999,999   3.5   3.7   2.2 10.3
$10,000,000$19,999,999   2.3   2.5   1.1 10.3
$20,000,000$49,999,999   2.3   2.3   2.2   3.4
$50,000,000 or more   0.7   0.8   0   0
Mean 2,560 2,698 1,813 4,666
Median   300   300   300 3,000
BASE: TOTAL RESPONDENTS
ANSWERING QUESTION
  572   483     89     29

NOTE: Table includes 17.8 percent of respondents who answered in the positive to Q29 (any sales?), but then reported zero sales.
SOURCE: 2014 Survey, Question 30.

Overall, about 80 percent of respondents indicated that their company had received additional investment in the technology related to the surveyed project.28 As with prior surveys, most respondents received small amounts of additional funding. Table 5-12 shows responses related to additional funding. Three-quarters of all NIH respondents reported receipt of less than $1 million. One percent reported receiving $50 million or more in additional funding; 9 percent reported receiving $5 million or more. Given that the cost of Phase 3 clinical trials has recently been estimated at $26,000 per patient29 and that can require the enrollment of more than 1,000 patients,30 the funding challenge for SBIR/STTR companies is immediately apparent.

Of those projects that received additional funding, 44 percent reported funding from U.S. private-sector sources, 25 percent from non-SBIR/STTR federal sources, and 24 percent from other external sources. Seventy percent reported additional funding from their own company, including 26 percent who reported

_______________

282014 Survey, Question 30. N=572.

29Hess, “Clinical Operations.”

30Roy, “Stifling New Cures.”

Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×

personal investments. Overall, 10 percent and 14 percent of those who received additional funding identified U.S. venture capital and U.S. angel and other private equity investors as the sources, respectively. Twenty-one percent and 4 percent reported strategic investments from U.S. partners and foreign partners, respectively (see Table 5-13).

TABLE 5-13 Distribution of All Reported Additional Investment Funding by Source of Funds

Percentage of Responses
    Source of Funding NIH Total SBIR Awardees STTR Awardees Phase IIB Awardees
Non-SBIR/STTR Federal Funds 25.2 21.9 41.4 28.6
Private Investment: U.S. Sources 44.2 45.7 36.8 67.9
Venture capital (VC)   9.5   9.5   9.2 21.4
U.S. angel funding or other private equity investment (not VC) 13.6 14.5 9.2 14.3
Friends and family 11.4 11.4 11.5 10.7
Strategic investors/partners 20.5 21 18.4 17.9
Other sources   9.1   9.3   8.0 17.9
Foreign Investment   5.5   5.5   5.7
Financial investors   2.4   2.4   2.3
Strategic investors/partners   3.7   3.3   5.7
Other External Sources 23.9 19.8 43.7 21.4
State or local governments 15.8 14.8 20.7 17.9
Research institutions (such as colleges, universities or medical centers) 9.3 5.5 27.6 3.6
Foundations   3.6   2.9   6.9
Internal Sources 70.0 74.0 50.6 71.4
Your own company (including money you have borrowed) 58.2 62.4 37.9 57.1
Personal funds 26.0 27.4 19.5 21.4
BASE: TOTAL RESPONDENTS
ANSWERING QUESTION
507 420 87 28

SOURCE: 2014 Survey, Question 31.

Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×

TABLE 5-14 Company-Level Changes

Percentage of Responding Companies
NIH Total SBIR Awardees STTR Awardees Phase IIB Awardees
Entered into strategic partnership with major industry player 34.9 35.8 30.7 35.4
Established one or more spin-off companies 20.8 22.2 13.3 18.2
Been acquired by/merged with another firm 16.0 14.4 24.1   8.6
Planning to make an initial public offering in the next 3 years   3.6   4.0   1.4   5.7
Made an initial public offering   2.6   2.5   3.5   2.9
None of the above 43.3 42.8 45.9 38.8
BASE: TOTAL COMPANIES
ANSWERING QUESTION
436 365 71 17

NOTE: Responses do not sum to 100 percent because respondents could select more than one answer.
SOURCE: 2014 Survey, Question 11.

Company-Level Commercialization Through Mergers and Acquisitions

SBIR/STTR firms often commercialize their technology through mergers or other company-level activities. Greater than 43 percent of responding companies indicated that they had not been acquired, had not implemented or planned an initial public offering (IPO), and had not established a spin-off (see Table 5-14). Conversely, greater than one-third had entered into a strategic partnership with a major industry player, 21 percent had established one or more spin-off companies, and 16 percent had been acquired by or merged with another firm.

Commercialization Training and Marketing

NIH has provided commercialization training for SBIR/STTR awardees for 10 years, primarily through an arrangement with a third-party provider (see Chapter 3, Program Initiatives). Thirty-nine percent of SBIR respondents and 24 percent of STTR respondents engaged in commercialization training at NIH.31 More than 38 percent considered the training to be valuable or extremely valuable (see Table 5-15). Conversely, about one-quarter of participants thought it was not very valuable or not at all valuable. Phase IIB respondents tended to see the training as less valuable, perhaps because they were already farther down the

_______________

312014 Survey, Question 49. N=570.

Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×

TABLE 5-15 Value of Commercialization Training

Percentage of Responses
NIH Total SBIR Awardees STTR Awardees Phase IIB Awardees
Extremely or very valuable 38.2 37.4 45.0 26.7
Extremely valuable 11.1 10.2 20.0   6.7
Very valuable 27.1 27.3 25.0 20.0
Somewhat valuable 36.2 36.9 30.0 26.7
Not very valuable 20.3 20.9 15.0 40.0
Not at all valuable   5.3   4.8 10.0   6.7
BASE: ACCEPTED
COMMERCIALIZATION
ASSISTANCE IN CONNECTION
WITH AWARD
207 187 20 15

SOURCE: 2014 Survey, Question 51.

commercialization path. STTR recipients saw the training as slightly more valuable than did SBIR recipients, perhaps reflecting that they had less information about commercial strategy to begin with given their strong academic roots.

However, about 27 percent of respondents indicated that they were likely to use the existing agency commercialization program in the future. Forty-nine percent expressed a preference to use the funding for their own marketing efforts, as permitted under the reauthorization legislation for SBIR/STTR (see Table 5-16). This is money that the grantees must use for paying for the training.

One question added to the 2014 Survey asked whether the company has at least one full-time staff person for marketing. This question provides another metric of the extent to which the company has focused on marketing. Forty-three percent of respondant companies reported that their company had at least one full-time marketing staff.32

Conclusions: Commercialization at the Company Level

Evidence from the 2014 Survey provides useful insight into the commercialization record of SBIR/STTR companies at NIH, on a number of dimensions. The data confirm that a substantial percentage of projects do indeed commercialize through sales of products or services and/or through the receipt of additional development funding.

_______________

322014 Survey, Question 13.

Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×

TABLE 5-16 Use of Different Commercialization Support

Percentage of Respondents
NIH Total SBIR Awardees STTR Awardees Phase IIB Awardees
Continue to use the agency’s program   26.8   26.6 28.1 11.1
Use the funding for your own marketing consultant   49.0   47.7 56.2 40.7
Neither   24.2   25.7 15.7 48.1
BASE: TOTAL RESPONDENTS
ANSWERING QUESTION
559 470 89 27

SOURCE: 2014 Survey, Question 52.

Forty-nine percent of respondents indicated that their company had already recorded sales of products or services derived from the awarded project. A further one-quarter of respondents were expecting sales in the future. Given the relatively short time between the award date for some of these awards and the survey date, and the long time-to-market cycle for products with regulatory requirements to meet, these expectations are not unreasonable. NIH does provide independent data against which the validity of the survey responses can be cross-checked.

Overall, the scale of commercialization is limited. About three-quarters of respondents with project-related revenues indicated that these revenues totaled $1 million or less.33 About 10 projects reported sales greater than $50 million and another 13 or 14 reported sales between $10 million and $50 million.

Additional investment is another important metric for commercialization. Many Phase II projects are not yet ready for the marketplace at the end of the award period, especially given the need for regulatory compliance.34 Three-quarters of all respondents reported receiving less than $1 million in additional funding. One percent reported receiving $50 million or more in additional funding; 9 percent reported receiving $5 million or more. The source of additional funding varied. About one-quarter of respondents mentioned non-SBIR/STTR federal funding, while 44 percent mentioned U.S. private-sector funding (including 10 percent for venture capital and 14 percent for angel investments). U.S. strategic investors were also important (21 percent).

In conclusion, these data support our view that SBIR/STTR funding is associated with outcomes that meet congressional mandates for commercialization. In

_______________

33See Figure 3-9.

34It is important to bear in mind that sales could continue to accumulate for many years to come, i.e., the problem may be in the assessment time frame rather than the actual return on investment.

Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×

the future, better data, especially collected by the agency, would allow for a more definitive conclusion and also more detailed understanding of the links between agency programs and initiatives and outcomes.

Knowledge Effects

One of the four congressionally mandated objectives for the SBIR/STTR programs is to “stimulate technological innovation,” which is often equated with patenting activity. However, in the context of small business, this standard metric of innovation does not capture the entire story: patenting is important, but it is also expensive, and SBIR/STTR funds cannot legally be used for this purpose. During case study discussions (for this and previous reports by the Academies on SBIR), company executives explained that patents have their limitations and are expensive. As a result, they prefer to keep their technology secret or to rely on first-mover advantages and other market-based leverage to defend their technologies.

However, standard metrics provide at least a starting point for quantitative analysis. Consequently, the survey addressed several intellectual property (IP)-related metrics: patents, trademarks, copyrights, and peer-reviewed papers.35

Patents

Patents are to some degree the life blood of high-tech firms. Overall, about two-thirds of companies (and more than 80 percent of Phase IIB recipients) reported the award at least one patent related to any SBIR/STTR-funded technology; 13 percent reported at least 10 related patents (see Table 5-17).

The survey also asked questions about IP related to the surveyed award (Table 5-18). Greater than one-half of respondents reported receiving at least one patent related to the surveyed technology. Nine percent reported receiving 5 or more related patents, and 4 percent reported 10 or more. STTR respondents reported similar numbers. The share of respondents reporting at least one patent is slightly down from the 2005 survey (57 percent).36

_______________

35The values of these knowledge repositories vary. Any unique item, painting, photo, or music score can be copy-written for a modest fee. Trademarks include more processing, as registered trademarks need to be unique in their field so as not to impinge on another prior trademark’s domain. A patent can be valuable IP, and patents have been correlated with prosperity. Refereed journal articles as a metric are not as highly valued outside of academia as inside, although company executives state in meetings that publications help to attract and keep high-quality staff and also provide additional validation for—and publicity about—their technology.

36National Research Council, An Assessment of the SBIR Program at the National Institutes of Health, 265.

Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×

TABLE 5-17 Number of Patents Related to All Company SBIR/STTR Awards

Percentage of Companies Responding
NIH Total SBIR Awardees STTR Awardees Phase IIB Awardees
0 33.6 30.8 48.8 18.3
1 14.0 16.0   3.2 12.2
2 10.2   9.1 16.2
3   8.3   8.6   6.9 25.4
4   5.8   5.0 10.3   9.1
5 to 9 15.1 16.2   9.0 10.7
10 or more 12.9 14.2   5.6 24.4
1 or more 66.4 69.2 51.2 81.7
 
Mean   4.74   5.22   2.11   5.40
Median   2.00   2.00   1.00   3.00
BASE: TOTAL COMPANIES
ANSWERING QUESTION
409 347 63 16

SOURCE: 2014 Survey, Question 11.

TABLE 5-18 Patents Awarded Related to Surveyed Project

Percentage of Responses
NIH Total SBIR Awardees STTR Awardees Phase IIB Awardees
0 47.0 47.2 46.3 25.9
1 23.7 23.0 27.5 22.2
2 12.0 12.1 11.3   7.4
3 or 4   8.2   7.6 11.3 22.2
5 to 9   5.2   5.9   1.3 14.8
10 or more   4.0   4.3   2.5   7.4
1 or more 53 52.8 53.8 74.1
 
Mean   1.63   1.71   1.23   3.11
Median   1   1   1   2
BASE: TOTAL RESPONDENTS
ANSWERING QUESTION
502 422 80 27

SOURCE: 2014 Survey, Question 38.1.

Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×

Copyrights and Trademarks

Regarding copyrights and trademarks, about one-quarter of respondents reported receiving a trademark related to the surveyed project, while less than 10 percent reported receiving a copyright.37

Peer-Reviewed Publications

Publications in peer-reviewed journals and conference proceedings are a standard method for disseminating scientific knowledge. As with the first-round assessment by the Academies, several case study respondents noted that publication in peer-reviewed journals was an essential part of the firm’s work.38

For the purposes of this assessment, peer-reviewed publications are important for two reasons:

  • They validate the quality of the research being conducted with program funds.
  • They are themselves the primary mechanism through which knowledge is transmitted within the scientific community.

The survey therefore asked about peer-reviewed publications (Table 5-19). Eighty percent of SBIR respondents and 85 percent of STTR respondents indicated that an author at the surveyed company had published at least one related scientific paper. Overall 42 percent reported publishing three or more related papers. The median number of publications for Phase IIB respondents was five.

Links to Universities

The survey asked a number of questions about the use of university staff and facilities on the surveyed project. Overall, about two-thirds of SBIR respondents and essentially all STTR respondents reported a university connection of some kind.

There were substantial differences between SBIR and STTR with regard to the kind of university linkage (Box 5-2). Seventy-eight percent of STTR respondents and 6 percent of SBIR respondents reported that the PI was a university faculty member. STTR respondents were also more likely to report that technology was licensed from the research institution (27 percent vs. 14 percent) and that the technology was originally developed at the research institution by a project team member (37 percent vs. 17 percent) (see Table 5-20).

_______________

372014 Survey, Question 38.2 and Question 38.3.

38National Research Council, An Assessment of the SBIR Program at the National Institutes of Health, Appendix D.

Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×

TABLE 5-19 Peer-reviewed Scientific Publications Related to the Surveyed Project

Percentage of Responses
NIH Total SBIR Awardees STTR Awardees Phase IIB Awardees
0 20.1 21.1 14.8 16.7
1 19.7 20.8 13.6   4.2
2 17.9 17.6 19.8 12.5
3 or 4 17.1 16.4 21.0   8.3
5 to 9 14.2 14.1 14.8 37.5
10 or more 11.0 10.1 16.0 20.8
1 or more 79.9 78.9 85.2 83.3
 
Mean   5.7   5.6   6.2 17.8
Median   2   2   3   5
BASE: TOTAL RESPONDENTS
ANSWERING QUESTION
508 427 81 24

SOURCE: 2014 Survey, Question 38.4.1.

Respondents were also asked to identify the universities with which they worked in various capacities on this project. Although the type of help varied widely, some universities were mentioned by a number of respondents. Overall, 255 different research institutions were identified from 488 projects. Those mentioned by four or more respondents are listed below in Table 5-21 (see Appendix D for the complete list of university mentions). Many of the names on this list are large state universities, a number of which have in recent years focused on technology transition as well as basic research, although two of the top three universities mentioned are private universities. Although far from a perfect metric, we believe these data provide a preliminary indication of the connections between specific universities, university systems, and the NIH SBIR/STTR programs.

Finally, for 85 percent of companies in the sample, at least one founder had an academic background (see Table 5-22) and for 60 percent of companies at least one founder was most recently employed at a research institution (see Table 5-23).

Conclusions: Knowledge Effects

What emerges from these data is a picture of companies that are dynamic centers of technological innovation, a considerable amount of which is protected

Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×

BOX 5-2
Survey Response Rate and Non-Respondent Bias

As noted in the introduction to this report, and described in detail in Appendix A, the committee recognizes the limitations of the survey effort underlying the data presented in this chapter. The 2014 Survey was sent to every principal investigator (PI) who received a Phase II award from NIH during fiscal years 2001-2010. PIs were asked to complete a maximum of two questionnaires. The preliminary population prior to contact was 3,375. Of these, 1,723 PIs could not be contacted at the SBIR or STTR company listed in the NIH awards database. The remaining 1,652 awards constitute the effective population of the survey. Of those 1,652 potential awards, there were 726 responses. This corresponds to a response rate of 21.5 percent of the entire set of awards and a 43.9 percent response rate from the effective population.

The committee acknowledges that because no information was gathered from non-respondents, the data are likely to be biased toward surviving firms. At the same time, the committee notes that successful PIs who left the original firm to start a new venture and successful firms that merged or were bought out by other firms are also excluded from the results. The committee suggests that, where feasible, future assessments of the SBIR program include comparisons of non-awardees, such as in matched samples (Azouley et al., 2014) or regression discontinuity analysis (Howell, 2015).a In addition, future assessments should document the root cause of non-responsiveness. For example, determining whether the company is still in business even if the PI is no longer with the firm could provide useful evidence about the effectiveness of the SBIR award.

_______________

aAzoulay, Pierre, Toby Stuart, and Yanbo Wang, “Matthew: Effect or Fable?” Management Science, 60(1), pp. 92-109, 2014. Howell, Sabrina, “DOE SBIR Evaluation: Impact of Small Grants on Subsequent Venture Capital Investment, Patenting, and Achieving Revenue.” Paper presented at the National Academy of Sciences, Engineering, and Medicine Workshop on the Economics of Entrepreneurship, June 29, 2015.

through the patent system. About two-thirds of companies reported receipt of at least one patent based on their work under SBIR/STTR contracts, while 53 percent reported receipt of at least one patent related to the surveyed project only specifically.

SBIR/STTR companies participate at a high level in the standard form of technical knowledge dissemination: publishing in peer-reviewed journals. Eighty percent of respondents reported that their company published at least one article based on the SBIR-funded work, and more than 40 percent reported publication of more than three such papers.

Finally, some SBIR/STTR companies are closely connected to the universities. About 70 percent of respondents reported a university connection

Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×

TABLE 5-20 Links to Universities

Percentage of Responses
NIH Total SBIR Awardees STTR Awardees Phase IIB Awardees
The PI for this project was at the time of the project an RI faculty member 17.4   6.1 77.8   3.6
The PI for this project was at the time of the project an RI adjunct faculty member 10.4 12.1   1.1 17.9
Faculty member(s) or adjunct faculty member(s) worked on this project in a role other than PI 38.7 38.2 41.1 35.7
Graduate students worked on this project 22.3 21.3 27.8 21.4
The technology for this project was licensed from an RI 16.0 14.0 26.7 17.9
The technology for this project was originally developed at an RI by one of the participants in this project 20.2 17.1 36.7 17.9
An RI was a subcontractor on this project 37.4 35.5 47.8 53.6
None of the above 29.7 35.1   1.1 32.1
 
BASE: TOTAL RESPONDENTS
ANSWERING QUESTION
569 479 90 28

NOTE: Responses do not sum to 100 percent because respondents could select more than one answer.
SOURCE: 2014 Survey, Question 71.

on the surveyed project, across a number of different kinds of linkages, and 22 universities were specifically mentioned as playing a role in at least five reported projects. This suggests that SBIR and STTR may in some cases play a potentially important role in supporting the practical implementation of university research.

SBIR/STTR AND COMPANIES

SBIR/STTR programs have a range of effects on companies, which affect their ability to operate and grow. Data about companies can help to define the technological space in which the SBIR/STTR programs operate. In addition, a review of the SBIR/STTR share of overall company activities can provide insights into the degree of dependence on SBIR/STTR for individual companies.

Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×

TABLE 5-21 University Participants Mentioned by Four or More Respondents

Research Institution Number of Mentions
University of Michigan 14
Duke University 10
Johns Hopkins University   8
Massachusetts Institute of Technology   7
University of North Carolina Chapel Hill   7
University of Pittsburgh   7
University of Virginia   7
Indiana University   6
Pennsylvania State University   6
University of Florida   6
University of Illinois Chicago   6
University of Massachusetts Medical School   6
University of Utah   6
Vanderbilt University   6
Children’s Hospital Boston   5
MD Anderson Cancer Center   5
Medical University of South Carolina   5
Oregon Health & Science University   5
Texas A&M University   5
UC San Francisco   5
University of Arizona   5
University of Kentucky   5
Case Western Reserve University   4
Cornell University   4
Dartmouth College   4
Harvard University   4
Mayo Clinic   4
University of California, Berkeley   4
University of Connecticut   4
University of Louisville   4
University of Minnesota   4
University of New Mexico   4
University of Pennsylvania   4
Washington University of St Louis   4

SOURCE: 2014 Survey, Question 60.

Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×

TABLE 5-22 Number of Academic Founders

Percentage of Company Responses
NIH Total SBIR Awardees STTR Awardees Phase IIB Awardees
0 14.7 14.3 16.8 18.2
1 42.3 44.7 30.5 31.1
2 26.7 26.3 28.9 31.6
3 10.2   9.1 15.7   7.7
4   3.7   3.4   5.4 11.5
5 or more   2.3   2.2   2.7
Mean   1.54   1.5   1.76   1.63
Median   1   1   2   2
BASE: TOTAL RESPONDENTS
ANSWERING QUESTION
445 370 74 17

SOURCE: 2014 Survey, Question 5.4.

TABLE 5-23 Previous Employment of Founders

Percentage of Company Responses
NIH Total SBIR Awardees STTR Awardees Phase IIB Awardees
Research institution 59.4 56.1 75.4 78.9
Other private company 52.7 54.6 43.1 41.1
Government   4.7   4.8   4.6   5.7
FFRDCs or National Labs
Other 10.3 11.5   4.0   5.7
BASE: TOTAL COMPANIES
ANSWERING QUESTION
453 375 77 17

NOTE: Responses do not sum to 100 percent because respondents could select more than one answer.
SOURCE: 2014 Survey, Question 6.

Impact on Company Formation

SBIR/STTR can have a profoundly catalytic impact on company formation. Seventeen percent of respondent companies were founded because of the SBIR/STTR programs, and a further 27 percent were formed in part because of the program (see Table 5-24).

Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×

TABLE 5-24 SBIR/STTR Impact on Company Formation

Percentage of Responding Companies
NIH Total SBIR Awardees STTR Awardees Phase IIB Awardees
Yes 17.3 17.9 14.2 13.4
In part 26.8 27.1 25.4 37.3
No 55.9 54.9 60.3 49.3
BASE: TOTAL COMPANIES
ANSWERING QUESTION
464 382 82 17

SOURCE: 2014 Survey, Question 8.

SBIR/STTR Share of R&D Effort

The survey asked respondents to estimate how much of their company’s total R&D effort—defined as man-hours of work for scientists and engineers—was devoted to SBIR/STTR-funded projects. Overall, 42 percent of SBIR/STTR respondents indicated that the programs funded 10 percent or less of total effort, while 34 percent indicated that they funded greater than one-half (see Table 5-25).

These data correspond fairly closely to responses from Question 9, which asked what percentage of company revenues during its current year were from SBIR/STTR awards. Thirty-four percent of companies reported zero SBIR/STTR revenues, while 27 percent reported receiving greater than one-half of revenues from SBIR/STTR. Three percent were entirely dependent on SBIR/STTR (see Table 5-26).

Prior Use of the SBIR/STTR Programs

Although a more linear interpretation of the innovation process would imply that ideas are tested in Phase I, prototyped in Phase II, and commercialized in Phase III, actual practice involves multiple iterations, or projects that must restart with an earlier phase, or multiple efforts needed to meet specific problems.

The survey asked respondents to indicate how many of the prior SBIR/STTR Phase I awards received from the NIH and other agencies were related to the project and technology being surveyed. Table 5-27 summarizes the responses. Less than 20 percent of projects received no other related SBIR/STTR awards. Greater than 20 percent received at least three additional related awards. These data strongly support the view that innovative products emerge from clusters of activity, rather than from simple straight line development from Phase I to Phase II to commercialization.

Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×

TABLE 5-25 Percentage of R&D Effort Funded by SBIR/STTR

Percentage of Company Responses
NIH Total SBIR Awardees STTR Awardees Phase IIB Awardees
0% 31.5 32.9 24.4 35.4
1-10% 10.5 10.5 10.4 25.8
11-25% 12.8 13.1 11.2   1.0
26-50% 11.5 11.0 14.2   1.4
51-75% 14.7 14.1 18.1 19.1
76-100% 18.9 18.4 21.7 17.2
Mean 33 32 38 29
Median 18 18 38   5
BASE: TOTAL COMPANIES
ANSWERING QUESTION
421 355 66 17

SOURCE: 2014 Survey, Question 10.

TABLE 5-26 Percentage of Company Revenues from SBIR/STTR

Percentage of Responding Companies
NIH Total SBIR Awardees STTR Awardees Phase IIB Awardees
0% 34.0 35.8 24.4 46.9
1-10% 13.9 14.0 13.2 28.2
11-25% 12.2 11.3 16.5   2.9
26-50% 12.8 13.1 10.9
51-75%   9.9   8.7 16.0   7.7
76-99% 14.0 14.4 12.2 11.5
100%   3.3   2.6   6.9   2.9
Mean 29.0 28.0 35.0 20.0
Median 18 18 18   5
BASE: TOTAL COMPANIES
ANSWERING QUESTION
409 344 66 17

SOURCE: 2014 Survey, Question 9.

Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×

TABLE 5-27 Prior SBIR/STTR or STTR Phase I Awards Related to the Surveyed Project

Percentage of Responses
NIH Total SBIR Awardees STTR Awardees Phase IIB Awardees
0 18.1 18.4 16.3 11.1
1 43.9 44.0 43.0 44.4
2 16.5 15.4 22.1 22.2
3 or 4 13.9 14.3 11.6 22.2
5 to 9   5.5   5.6   4.7
10 or more   2.2   2.2   2.3
1 or more 81.9 81.6 83.7 88.9
 
Mean   1.92   1.93   1.83   1.59
Median   1   1   1   1
BASE: TOTAL RESPONDENTS
ANSWERING QUESTION
547 461 86 27

SOURCE: 2014 Survey, Question 39.1.

Turning to prior Phase II awards, about three-quarters of respondents reported at least one related Phase II award, while 11 percent reported at least three (see Table 5-28).

Long-Term Effects on the Recipient Company

Although SBIR/STTR awards have direct effects on specific projects, they can also have a longer term effect on the trajectory of company development. The survey asked respondents about the impacts directly. The results are summarized in Table 5-29.

These results indicate an overwhelmingly positive impact. Overall, 97 percent of SBIR/STTR respondents reported a positive effect, and 62 percent reported a transformative effect. Two respondents out of 570 reported negative effects.

Respondents were also asked to describe these effects in their own words. Key aspects of their comments are reported below, focused on the ways in which SBIR/STTR and STTR made a profound difference to the company in the long term.

Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×

TABLE 5-28 Prior SBIR/STTR or STTR Phase II Awards Related to the Surveyed Project

Percent of Responses
NIH Total SBIR Awardees STTR Awardees Phase IIB Awardees
0 23.9 25.0 17.9 11.5
1 46.0 43.9 57.1 38.5
2 18.9 19.8 14.3 34.6
3 or 4   8.1   8.3   7.1 15.4
5 to 9   1.9   1.8   2.4
10 or more   1.1   1.1   1.2
1 or more 76.1 75.0 82.1 88.5
 
Mean   1.34   1.35   1.32   1.54
Median   1   1   1   1.5
BASE: TOTAL RESPONDENTS
ANSWERING QUESTION
528 444 84 26

SOURCE: 2014 Survey, Question 39.2.

TABLE 5-29 Long-Term Effects on Recipient Companies

Percentage of Responses
NIH Total SBIR Awardees STTR Awardees Phase IIB Awardees
Positive, highly positive, or transformative effect 96.5 96.5 96.7 92.9
Highly positive or transformative effect 61.8 62.7 56.7 71.4
Positive effect 34.7 33.8 40.0 21.4
No effect   3.2   3.1   3.3   7.1
Negative, highly negative, or disastrous effect   0.4   0.4
Negative effect   0.2   0.2
Highly negative or disastrous effect   0.2   0.2
BASE: TOTAL RESPONDENTS
ANSWERING QUESTION
570 480 90 28

SOURCE: 2014 Survey, Question 57.

Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×

Key Aspects of SBIR-Driven Transformation

It is not easy to summarize the numerous ways in which SBIR/STTR awards from NIH helped to transform recipient companies. What follows is therefore a limited list of impacts.

  • Supported company formation
  • Provided first dollars
  • Funded areas where venture capital and other funders were not interested
  • Supported development of critical company infrastructure
  • Opened doors to potential partners
  • Helped address niche markets too small for major players/funders
  • Funded technology development
  • Enabled projects with high levels of technical risk and high potential return
  • Supported adaptation of technologies to new uses, markets, and industry sectors
  • Funded development of core technology
  • Diversified expertise, allowed hiring of specialists
  • Gave companies immediate credibility
  • Funded researchers to enter business full time
  • Transformed company culture to become more market driven
  • Created new companies and kept companies in business (that would not exist without SBIR/STTR funding)
  • Helped increase the company’s knowledge base applied to later projects
  • Expanded the scope and scale of R&D capabilities
  • Supported technology development that led to spinoff companies

From these responses it is clear that small innovative companies are highly sensitive to the impact of outside factors. The sudden withdrawal of a sponsor can crush a company; a single contract can provide funding for 2 or 3 years of growth. Above all, these small companies are highly path dependent: what happens to them at a given moment can dramatically affect long-term outcomes.

In the end, SBIR/STTR can be viewed in many cases as a positive outside factor: one that provides funding, validation, and often market access not otherwise available. Even though it seems tenuous to link one award to the eventual success of a large corporation, that is, in fact, how some very small companies grow into large ones. The evidence from survey respondents suggests that this positive jolt is not an uncommon effect of these awards.

Other Company-Level Information

The survey asked about other potentially significant aspects of the company. Previous analyses of SBIR/STTR did not address a potentially important

Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×

intervening variable: industry sector. It is quite possible that commercialization outcomes may be affected by the average cycle time of product development in different sectors. For example, product cycle time is much shorter in software than in materials or medical devices. Overall, 92.3 percent of NIH SBIR/STTR survey recipients worked in the medical technology sector. Table 5-30 shows the distribution of responses by phase and sector.

This question was designed to provide an approximate map of activities by sector. There is considerable overlap between some categories, and respondents would have substantial leeway to define sectors differently, so these finding should be viewed as highly preliminary. A few key points emerge:

  • Almost all awards were to companies primarily working in medical technology.
  • About onethird of respondent companies were working in medical devices and biotechnology.
  • About onequarter were working on research tools.
  • A further 17 percent were working in pharmaceuticals, with the remainder spread across medical information technology and education.
  • Responses show that there are no substantial differences between SBIR and STTR respondents.

TABLE 5-30 Distribution of Responses by Sector Phase

Percent of Responses
NIH Total SBIR Awardees STTR Awardees Phase IIB Awardees
Aerospace and Defense 2.6 2.9 1.1 6.9
Aerospace 0.2 0.2
Defense-specific products and 2.6 2.9 1.1 6.9
services
Energy and the Environment 2.5 2.9
Renewable energy production (solar, wind, geothermal, bio-energy, wave) 0.2 0.2
Energy storage and distribution 0.7 0.8
Energy efficiency 0.3 0.4
Other energy or environmental products and services 1.6 1.9
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Percent of Responses
NIH Total SBIR Awardees STTR Awardees Phase IIB Awardees
Engineering   9.5 10.1   6.5   6.9
Engineering services   1.5   1.4   2.2
Scientific instruments and measuring equipment   5.9   6.8   1.1   3.4
Robotics   1   0.8   2.2
Sensors   4.1   4.9   3.4
Other engineering   0.3   0.2   1.1
Information Technology   8.6   7.8 12.9   3.4
Computers and peripheral equipment   2.1   2.3   1.1
Telecommunications equipment and services   0.2   0.2
Business and productivity software   1   1   1.1
Data processing and database software and services   3.1   2.7   5.4   3.4
Media products (including web-, print-, and wireless-delivered content)   3   2.9   3.2
Other IT   1.3   1.2   2.2
Materials (including nanotech for materials)   2.8   3.1   1.1
Medical Technologies 92.3 91.5 96.8 100
Pharmaceuticals 17.3 17.3 17.2 24.1
Medical devices 32.9 34 26.9 58.6
Biotechnology (including therapeutic, diagnostic, combination) 32.9 32.4 35.5 34.5
Health IT (including mobile, big data, training modules)   7.4   7.2   8.6   3.4
Research tools 25.8 25.2 29 10.3
Education materials   8.9   8.5 10.8
Other medical products and services   3.9   4.3   2.2
Other (please specify)   6.9   8   1.1   3.4
BASE: TOTAL RESPONDENTS
ANSWERING QUESTION
608 515 93 29

NOTE: Responses do not sum to 100 percent because respondents could select more than one answer.
SOURCE: 2014 Survey, Question 20.

Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×

COUNTERFACTUALS

It is always difficult to tightly determine the impact of a given SBIR or STTR award. Many factors affect the success and failure of companies and projects, and it is difficult to determine whether a specific factor was a necessary condition for success. Worse still, the large number of factors and the multiple paths to success and failure mean that it is unusual to be able to state with confidence that a particular intervention—in this case an SBIR or STTR award—constitutes a sufficient condition for a project’s success.

Still, it is worth considering what would have occurred absent SBIR or STTR funding from the perspective of those most likely to have detailed knowledge and understanding of their particular projects: the principal investigators. Accordingly, the 2014 Survey asked a series of questions focused on the likely effect of the absence of SBIR or STTR funding. Of course, asking recipients about the impact of funding raises possible conflicts of interest, so results should be interpreted with some caution. However, these awards are some years in the past now, and many recipients no longer apply for SBIR/STTR funding for a variety of reasons.

Project Go-Ahead Absent SBIR/STTR Funding

One approach has been to ask recipients for their own views on the impact of the program on their project or company. In particular, the survey asked Phase II recipients whether the project would have been undertaken absent SBIR/STTR funding and whether the scope and timing would have been affected. Responses are summarized in Table 5-31. About 7 percent of Phase II respondents indicated that the project probably or definitely would have proceeded without program funding. In contrast, almost 75 percent thought the project probably or definitely would not have proceeded absent program funding: 34 percent were definite and 41 percent thought it unlikely.

These data have interesting wider implications for debates about early-stage funding: they suggest a weakness in the “crowding out” hypothesis, because more than 70 percent of respondents (presumably those with the closest knowledge of funding prospects for the project) believed it unlikely that alternative funding would be found.

The small number of respondents (12) who believed the project might have proceeded without SBIR/STTR funding were asked additional questions about the impact on project scope, duration, and timelines. They responded as follows:

  • Project scope would have been narrower (67 percent)
  • Project would have been substantially delayed (75 percent)
  • Project would have taken longer (75 percent)
  • Project would not have hit necessary milestones (75 percent)
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×

TABLE 5-31 Project Undertaken in the Absence of this SBIR/STTR Award

Project Go-Ahead Absent Award Percentage of Responses
Yes   6.7
Definitely yes   1.7
Probably yes   5.0
Uncertain 19.0
Probably not 40.8
Definitely not 33.5
100.0
N= 179

SOURCE: 2014 Survey, Question 24.

Overall, these views indicate that SBIR/STTR funding was important not only for the go/no-go decision but also for the eventual shape and indeed likely impact of the project. Delay in bringing projects to conclusion—and hence to the point of potential market entry—can have a disastrous effect, because the window for market entry can be a narrow one.

Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 140
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 141
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 142
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 143
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 144
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 145
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 146
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 147
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 148
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 149
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 150
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 151
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 152
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 153
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 154
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 155
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 156
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 157
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 158
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 159
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 160
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 161
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 162
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 163
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 164
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 165
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 166
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 167
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 168
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 169
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 170
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 171
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 172
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 173
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 174
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 175
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 176
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 177
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 178
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 179
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 180
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 181
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 182
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 183
Suggested Citation:"5 Quantitative Outcomes." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR/STTR at the National Institutes of Health. Washington, DC: The National Academies Press. doi: 10.17226/21811.
×
Page 184
Next: 6 Participation of Women and Minorities »
SBIR/STTR at the National Institutes of Health Get This Book
×
Buy Paperback | $85.00 Buy Ebook | $69.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The Small Business Innovation Research (SBIR) and Small Business Technology Transfer (STTR) programs provide federal research and development funding to small businesses. In 2008, the National Research Council completed a comprehensive assessment of the SBIR and STTR programs. The first-round study found that the programs were "sound in concept and effective in practice." Building on the outcomes from the Phase I study, this second phase examines both topics of general policy interest that emerged during the first phase and topics of specific interest to individual agencies, and provides a second snapshot to measure the program's progress against its legislative goals.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!