National Academies Press: OpenBook

Venture Funding and the NIH SBIR Program (2009)

Chapter: 2 Study Methodology

« Previous: 1 Introduction
Suggested Citation:"2 Study Methodology." National Research Council. 2009. Venture Funding and the NIH SBIR Program. Washington, DC: The National Academies Press. doi: 10.17226/12543.
×

2
Study Methodology

Reflecting the two core questions (How many firms have actually been excluded by the ruling from participation in the NIH SBIR program? And what is the likely effect of this exclusion on these firms and on the NIH SBIR program) the methodology for this study is divided into two areas.

2.1
IDENTIFYING VENTURE-FUNDED FIRMS AND ESTIMATING THE “EXCLUSION EFFECT”

Identifying Venture-funded Firms

Identifying firms that have received venture funding is a challenge. SBIR-funded firms, which are in most cases privately held, are not required to reveal whether they have received third-party investment. As a result this information is not collected and stored by SBIR-funding agencies or SBA.

In order to establish the distribution of venture funding prior to the SBA ruling,1 this research initially focused on firms winning Phase II awards from 1992-2002 inclusive.

Phase II awards were selected for study because they account for the largest amount of NIH SBIR funding. As Phase II awards continue to grow in size, and are extended in length, the proportion of funding allocated for post-Phase I continues to grow. What is more, commercial success almost always comes after

1

The years prior to the ruling are years when a “natural” level of participation for venture-funded companies might be established.

Suggested Citation:"2 Study Methodology." National Research Council. 2009. Venture Funding and the NIH SBIR Program. Washington, DC: The National Academies Press. doi: 10.17226/12543.
×

Phase II rather than just Phase I—so a focus on the latter includes almost all commercial successes.

The focus on 1992-2002 coincides with the rapid development and maturation of the biotechnology industry. It also reflects the reauthorization of the program in 1992, which led to prioritization of the legislative goals of SBIR with an emphasis on commercialization. Finally, and not least, the focus on data from 1992 to 2002 is driven by the availability of data.

The firms winning Phase II awards also constitute the universe of firms addressed by the NRC’s Phase II Survey,2 and the NIH Phase II Survey,3 which provide the best available data on outcomes from the SBIR program. These data will be critical to the second phase of the analysis, identifying impacts.

The standard database on venture funding—also used by GAO in its 2006 study4—is the Thomson VentureSource database, and this is the primary source of venture funding data utilized in this National Research Council study.5 The NRC compiled a list of firms that received at least one Phase II award at NIH between 1992 and 2002 inclusive and Thomson VentureSource ran that list against its own database of firms that had received venture funding as of the end of 2006.

Because VentureSource requires that firms be identified based on company name, a process was developed to broaden the net, identifying all firms that could conceivably be matched to firms in the NIH awards database (using wildcards in the database search6). These possible matches were then tested manually against known company addresses to eliminate false positives from the results dataset.7

The final list from VentureSource provides considerable detail on venture capital investments: It indicates the name of the company, the date of the round of funding, the type of funding, and the amount.8 This appears to be the most

2

For details on the NRC Phase II Survey, which included at least one questionnaire to every Phase II winner 1992-2002 inclusive, see National Research Council, An Assessment of the SBIR Program at the National Institutes of Health, Charles W. Wessner, ed., Washington, DC: The National Academies Press, 2009.

3

For details on the methodology and scope of the NIH Phase II Survey, see National Institutes of Health, National Survey to Evaluate the NIH SBIR Program: Final Report, July 2003, available at <http://grants.nih.gov/grants/funding/sbir.htm>.

4

U.S. Government Accountability Office, Small Business Innovation Research: Information on Awards Made by NIH and DoD in Fiscal Years 2002 through 2004, GAO-06-565, Washington, DC: U.S. Government Accountability Office, April 2006.

5

A secondary database owned by <Inknowvation.com> was also used by GAO but was not made available to the NRC.

6

We supplied a list of firm names; VentureSource replaced part of the name with wildcards and then searched the database.

7

The VentureSource search results included information on the city and state of the firm, and these data were cross-checked against city and state data from the SBIR awards database.

8

“Type of funding” refers to terminology within the venture capital community. Thomson and others distinguish between seed funding, A round funding, B round funding and C round funding.

Suggested Citation:"2 Study Methodology." National Research Council. 2009. Venture Funding and the NIH SBIR Program. Washington, DC: The National Academies Press. doi: 10.17226/12543.
×

definitive list of venture-funded firms available and it is used in the first phase of this study.9

Estimating the “Exclusion Effect”

It is important to keep in mind that simply receiving some venture funding is in itself not disqualifying. To be disqualified from participating in the SBIR program, firms must be owned or controlled by firms that themselves fail one of the two tests—breaching the size requirement and/or the individual ownership requirement—outlined above in Box 1-1.10 Thus, the list of venture-funded firms was then analyzed to determine whether it was likely that these firms would in fact be excluded by the SBA ruling.

Unfortunately, privately owned firms are often very reluctant to provide information about their ownership structure. It is therefore not possible to determine directly which venture-funded firms are owned or controlled by their venture investors (and hence excluded) and which are not. Nor it is practical to examine the ownership structure of every venture capital firm that provides funding in order to determine whether their ownership structure or the collective character of their other investments breach the eligibility requirements.

It is also worth noting that firm ownership and control are by no means synonymous.11 Ownership of a majority of outstanding voting shares is sufficient to provide formal control. However, key personnel may still exert significant—sometimes predominant—control over key decisions. Conversely, 51 percent ownership is not necessarily required in order to exert effective control. However, both because the SBA ruling focuses on 51 percent ownership and because any statistical analysis must find ways to draw bright lines through murky questions, this study assumes that the critical delineator for the purpose of access to SBIR is 51 percent ownership.

After considerable discussion, and drawing on their extensive experiences, the Committee agreed on two proxies for venture control of a firm12:

9

It is important to note that there is considerable heterogeneity among venture capital firms themselves. For a review of some of the differences, see William A. Sahlman, “The Structure and Governance of Venture-capital Organizations,” Journal of Financial Economics, 27(2):473-521, October 1990.

10

There are also small biotechnology companies that are 51 percent owned by individuals but are excluded based on affiliation of a venture capital company’s other portfolio companies.

11

Eugene Fama and Michael Jensen distinguished between ownership and control in their classic 1983 paper. See Eugene Fama and Michael Jensen, “Separation of Ownership and Control,” Journal of Law and Economics, XXVI, 1983. The role of venture capital control of small innovative businesses is discussed in Paul Gompers and Josh Lerner, “The Venture Capital Revolution,” The Journal of Economic Perspectives, 15(2):145-168, Spring 2001.

12

Committee members with extensive knowledge of venture capital include Linda Powers (Toucan Capital Corporation), Michael Borrus (X/Seed Capital), Clark McFadden (Dewey & LeBoeuf, LLP), and Pete Linsert (Columbia Biosciences Corporation).

Suggested Citation:"2 Study Methodology." National Research Council. 2009. Venture Funding and the NIH SBIR Program. Washington, DC: The National Academies Press. doi: 10.17226/12543.
×
  • Venture capital investments of $5 million or more; or

  • At least two separate rounds of venture funding.

In both cases, it is reasonable to assume that 51 percent of company shares will have passed into the hands of the new investors.13

Using these proxies, the Committee screened the original list of venture-funded firms to develop a list of firms that it identified as having been excluded from the SBIR program on the basis of the SBA ruling. This screened list is referred to as “venture-funded firms” throughout this report.14

Next, this list was adjusted to take into account other grounds for exclusion. Most notably, this list includes firms that would be subject to exclusion from the SBIR program on other grounds, regardless of their venture capital ownership structure. This includes circumstances where the firm—

  1. Had more than 500 employees.

  2. Was purchased by a foreign firm, or by a U.S. firm not 51 percent individual-owned.

After eliminating firms that were venture-funded and met criteria 1 or 2 above for venture control, but also met one or more of the criteria above, we were left with a final list of venture-funded companies that were likely to be excluded based on the SBA ruling.

Grouping the Firms

In order to provide the most relevant analysis, the Committee focused on two sets of variables:

  • Whether a firm was or was not venture-funded, as defined above.

  • Whether a firm was among the top 200 most prolific winners of NIH Phase II funding during the 1992-2002 period.15

This approach generated the matrix in Table 2-1.

13

This assumption was later tested against companies known to be excluded on venture capital grounds. We found that our criteria in these cases matched known outcomes with a high degree of accuracy.

14

It is important to note that these categories of majority venture funded firms as opposed to venture-funded firms are frequently not static. As often happens in the biotechnology industry, a company may have some venture funding and be eligible for SBIR Phase I and Phase II awards, but then receive venture capital additional funding and become ineligible for further SBIR funding.

15

The numerical basis for whether a firm was among the top 200 most prolific winners was driven by the need for more detailed data and the constraints imposed by limited study resources. “Prolific” is defined as being among the winners of the most NIH Phase II SBIR awards in 1992-2002.

Suggested Citation:"2 Study Methodology." National Research Council. 2009. Venture Funding and the NIH SBIR Program. Washington, DC: The National Academies Press. doi: 10.17226/12543.
×

TABLE 2-1 Firms Funded by the NIH SBIR Program (1992-2002)

 

Top 200 Winners

Other Winners

Totals

Percent

VC-Funded

35

150

185

12

Not VC-Funded

165

1,186

1,351

88

Total

200

1,336

1,536

100

SOURCE: U.S. Small Business Administration Tech-Net Database, VentureSource.

NOTE: Table 2-1 refers only to majority venture-funded firms that meet the two core venture capital investment criteria for control—namely more than $5 million in investment or more than one round of venture control funding. The Top 200 winners accounted for 42.3 percent of Phase II awards during 1992-2002.

Majority venture-funded firms were clustered among the top 200 winners. They accounted for a higher percentage of the top 200 winners (17.5 percent) than they did of all NIH winners. Table 2-1 shows that overall, about 12 percent of firms that won SBIR Phase II awards from 1992-2002 inclusive were also venture capital-funded.

In addition, firms that were more prolific in winning SBIR awards were also more successful in attracting significant venture capital funding: 35 (17.5 percent) of the top 200 award winners were majority venture-funded, as against 11.2 percent of the remaining 1,336 SBIR award winners.16

2.2
METHODOLOGY FOR MEASURING THE IMPACT OF THE SBA RULING

The primary objective of this part of the assessment is to compare outcomes for SBIR projects implemented by venture-funded and non-venture-funded firms respectively.

Two kinds of outcomes are utilized:

  • Project-level outcomes identified by the NRC and NIH surveys. These include sales and the attraction of additional funding as the two key metrics.17

  • Firm-level outcomes, generated by cross-referencing firms with Phase II awards against revenue and employment data extracted from the Hoover’s database of small firms.

16

It is important to note, as well, that while SBIR enhances the success of venture-backed firms, the obverse is also true. It is also possible that the top 200 SBIR award winners had topics that were more advanced in their development.

17

Additional funding as used in the NRC and NIH surveys means funding from all sources, including VCs, angel funding, and additional non-SBIR federal funding (a major component).

Suggested Citation:"2 Study Methodology." National Research Council. 2009. Venture Funding and the NIH SBIR Program. Washington, DC: The National Academies Press. doi: 10.17226/12543.
×

Project-level Outcomes

Until recently, there were no available data on outcomes from SBIR programs through which to compare the performances of venture-funded and non-venture-funded firms. Recent surveys by the NRC and the NIH, however, provide at least initial indications of relative outcomes.18

The NIH survey generated 768 responses (1 per firm) and the NRC 496 responses from 368 firms.19 Together, the surveys generated responses covering at least one project from 861 firms that received Phase II awards during this period.20

Firm-level Outcomes

While the NRC/NIH data provide important insights into outcomes from specific surveyed projects, it is also useful to generate a different perspective based on the development of the firm, rather than an individual project.

Utilizing the Hoover’s database of small firms, we developed a dataset of current revenue and employee data, which provides a useful proxy for the overall commercial success of the firm.21 Unfortunately, Hoover’s does not maintain time-series data on individual firms, so our metrics were based on the size of firm revenues and number of employees as of the most recent data available from Hoover’s (in most cases, for 2006).

As each data point in this dataset had to be collected manually, it was not cost effective to collect individual data on more than 1,200 firms. Accordingly, while we collected data for all 183 venture-funded firms,22 we limited data collection for non-venture-funded firms to an equivalent random sample: 35 firms among the top 200 most prolific Phase II winners, and 148 firms from among the pool of firms that were not among the most prolific winners and were not venture-funded.

18

See National Institutes of Health, National Survey to Evaluate the NIH SBIR Program: Final Report, op. cit. The NRC survey results for NIH are presented in National Research Council, An Assessment of the SBIR Program at the National Institutes of Health, C. Wessner, op. cit.

19

There is some overlap between the projects surveyed by the NIH and NRC. However, as project ID data is not directly comparable, it is not possible to determine exactly the dimensions of that overlap.

20

Not every firm responded to the survey questionnaires. The 861 firms are a subset of the 1,536 firms that won Phase II awards 1992-2002.

21

The Hoover’s database of small firms is an extension of the well-known Dunn and Bradstreet database. It is the most comprehensive source of information on small businesses in the United States. Even so, it offers only a current snapshot (no historical data), and is not entirely comprehensive as it, in turn, is based in part on survey data. Hoover’s Small Business Database can be accessed on line at <http://www.hoovers.com>.

22

Table 3-4 derives the total number of firms that were excluded or possibly excluded as a direct impact of the SBA directive.

Suggested Citation:"2 Study Methodology." National Research Council. 2009. Venture Funding and the NIH SBIR Program. Washington, DC: The National Academies Press. doi: 10.17226/12543.
×

BOX 2-1

Downward Bias in the Estimation of the Impact of the SBA Directive

Assessing the impact of the SBA exclusion of majority venture-funded firms is difficult because obtaining data on the ownership structure of companies, especially private-held small firms, is hard to do. This assessment therefore relies on a sample of firms that received Phase II SBIR awards during the period 1992-2002 (before the directive was issued). The use of such data is likely to yield a downward bias in the estimate of the effects of the SBA exclusion for two reasons:

  • Recent shift of venture capital to the biotechnology sector. The life sciences and biotechnology were relatively embryonic industries during the sample period (before 2002). More importantly, venture capital started shifting towards biotechnology and life sciences after the Internet bubble burst in 2001.a Thus, the number and proportion of small life science-based firms receiving venture funding may be much higher than can be captured from data focused on the 1992-2002 period.

  • Venture funding barred as biotechnology firms matured. Leading papers in the economics of innovation literature point out that technological and product development opportunities (and thus, commercialization of research) become more prevalent as an industry matures.b Venture funding is focused mainly on product development. This implies that it is likely that the SBA ruling may be excluding a higher percentage of firms in this sector as the industry matures.

  

aDouglas P. Lee and Mark D. Dibner, “The Rise of Venture Capital and Biotechnology in the US and Europe,” Nature Biotechnology, 23:672-676, 2005.

  

bOn product innovation and commercialization in a wide variety of industries, see Edwin Mansfield, “Academic Research and Industrial Innovation: An Update of Empirical Findings,” Research Policy, 26(7-8):773-776, April 1998.

2.3
CASE STUDY AND OTHER DATA

One final area of analysis concerns the impact of the ruling on the biotech industry itself. As noted earlier, BIO and other groups have claimed that the exclusion of venture funded companies from SBIR will result in a critical “gap” in the funding flow, one that may prevent important discoveries from being commercialized.

To substantiate these claims, BIO conducted telephone and Internet surveys of its “emerging company” membership—defined as firms with fewer than 350 employees and no marketable products.23 The surveys suggest that a large

23

BIO appears to have contacted all 650 firms that make up its emerging company membership, with a response rate of about 41 percent. BIO provided no additional background information about the surveys. For a description of the surveys, see Appendix E of this report.

Suggested Citation:"2 Study Methodology." National Research Council. 2009. Venture Funding and the NIH SBIR Program. Washington, DC: The National Academies Press. doi: 10.17226/12543.
×

majority of responding biotech companies would apply for NIH funding absent the ruling, and that most would not use the funds for their lead product. Two thirds of the proposed research was reported to be focused on preclinical or discovery stages.

BIO also conducted six case studies designed to show that promising lines of early-stage research have been abandoned or delayed as a result of the ruling. These cases, however, do not provide counterfactual evidence based on products that were funded prior to the SBA ruling, which would have been excluded by the ruling.

These cases will be discussed in the context of the NRC’s own cases, which included some venture-funded firms. An analysis of the evidence submitted by BIO is included in Appendix E of this report.

Suggested Citation:"2 Study Methodology." National Research Council. 2009. Venture Funding and the NIH SBIR Program. Washington, DC: The National Academies Press. doi: 10.17226/12543.
×
Page 22
Suggested Citation:"2 Study Methodology." National Research Council. 2009. Venture Funding and the NIH SBIR Program. Washington, DC: The National Academies Press. doi: 10.17226/12543.
×
Page 23
Suggested Citation:"2 Study Methodology." National Research Council. 2009. Venture Funding and the NIH SBIR Program. Washington, DC: The National Academies Press. doi: 10.17226/12543.
×
Page 24
Suggested Citation:"2 Study Methodology." National Research Council. 2009. Venture Funding and the NIH SBIR Program. Washington, DC: The National Academies Press. doi: 10.17226/12543.
×
Page 25
Suggested Citation:"2 Study Methodology." National Research Council. 2009. Venture Funding and the NIH SBIR Program. Washington, DC: The National Academies Press. doi: 10.17226/12543.
×
Page 26
Suggested Citation:"2 Study Methodology." National Research Council. 2009. Venture Funding and the NIH SBIR Program. Washington, DC: The National Academies Press. doi: 10.17226/12543.
×
Page 27
Suggested Citation:"2 Study Methodology." National Research Council. 2009. Venture Funding and the NIH SBIR Program. Washington, DC: The National Academies Press. doi: 10.17226/12543.
×
Page 28
Suggested Citation:"2 Study Methodology." National Research Council. 2009. Venture Funding and the NIH SBIR Program. Washington, DC: The National Academies Press. doi: 10.17226/12543.
×
Page 29
Next: 3 Venture Funding for NIH Phase II Winners, 1992-2002 »
Venture Funding and the NIH SBIR Program Get This Book
×
Buy Paperback | $34.50 Buy Ebook | $27.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The Small Business Administration issued a policy directive in 2002, the effect of which has been to exclude innovative small firms in which venture capital firms have a controlling interest from the SBIR program. This book seeks to illuminate the consequences of the SBA ruling excluding majority-owned venture capital firms from participation in SBIR projects.

This book is part of the National Research Council's study to evaluate the SBIR program's quality of research and value to the missions of five government agencies. The other books in the series include:

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!