How many SBIR Phase II contracts lead to Phase III
Accuracy of the data The biggest challenge here will be the transient nature of the firms and the information.
PI’s come and go; firms shrink and grow; firms are acquired; firms may close down, move, or change names;
Answers often depend on whom you ask;
Firms that are very successful may have new management in place as a result of venture capital activity or other financial arrangements, or due to firm acquisition by another organization;
There is often a long gestation between award of the SBIR Phase II and achievement of significant revenues. Often other SBIR grants and other R&D may have occurred in the interval. There may not be anyone still at the firm knowledgeable of the link between the product and the SBIR;
The most serious analytical issue may be the dependency on self-reporting, as the agencies generally know little about commercialization except that which is self-reported by the firm.
Depth of the data – Does the data reach firm level variables, award data, projects, and outcomes? Conversely, what primary gaps in the data should be filled by primary research? The Committee will also need to assess data collected by agencies beyond that required by SBA, to see if there are opportunities and/or gaps.
The expanded role of DoD data Recent DoD collections include information on projects in the earlier studies, as well as in the next Fast Track: about one-third of the DoD collection is on projects awarded by other agencies. Note that information from the various data collection has not been cross-referenced and analyzed. It will take extensive effort to properly identify each project in each collection (as the collections for example lack common unique identifiers)
The form of the data – is the agency data in paper form or is it computerized?
Four substantial surveys have addressed commercial and other outcomes from SBIR: GAO (1992), DoD (1997), SBA (1999), and DoD Fast Track.54 In many areas, these surveys ask similar or identical questions, creating extensive databases of results relevant to many of the metrics being considered for use in this study.
The Fast Track surveys each addressed a single SBIR Phase II award, and collected some information on the firm. 80 to 90 percent of the questions were about the specific award. Some firms have only one award. Some have over 100. GAO (1992), SBA (1999) and DoD (1997) each surveyed 100 percent of the SBIR Phase II awards made from 1983 through an end date that was four years prior to the date of the survey: i.e., GAO (1992) surveyed, in 1991, all SBIR Phase II project awards from 1983 through 1987.55 These studies provide coverage for the early years of the program.
The existing survey results showed the distribution of commercialization to be quite skewed. For example, 868 of the 1310 reporting projects in the SBA survey had no sales. Fifty five had over $5 million in sales, one of which was over $240M, two were slightly over $100M, and five were between $46 M and $60M. Those 55 projects represent 1.5 percent of the number surveyed, 4.2 percent of the responses, but 76 percent of the total sales. This means that in collecting commercialization data, firm selection becomes critically important. Surveying a high percentage of the
See U.S. General Accounting Office, “Federal Research: Small Business Innovation Research shows success but can be strengthened.” Washington, D.C.: U.S. General Accounting Office, 1992. The DoD study on the commercialization of DoD SBIR was based on a survey of Phase II awards from 1984–1992. It involved an 80 in-person and 69 telephone interviews with SBIR firms, interviews with DoD program managers and laboratory officials. This study, completed in October 1997, is unpublished. The SBA study on the commercialization of SBIR was based on a 100 percent survey of Phase II awards from 1983 to 1993 of non-DoD agencies, and 43 in-person interviews with SBIR firms. This study, completed in July 1999 is unpublished. The DoD Fast Track study was conducted by the National Research Council. See National Research Council, The Small Business Innovation Research Program: An Assessment of the Department of Defense Fast Track Initiative, 2000, op. cit.
See GAO (1992) op. cit. An unpublished study by the SBA was completed in 1999, and an unpublished study by DoD was completed in 1997. See footnote 27 for description.