2
Findings and Recommendations
I.
NATIONAL RESEARCH COUNCIL (NRC) STUDY FINDINGS
-
The Small Business Innovation Research (SBIR) Program Is Making Significant Progress in Achieving the Congressional Goals for the Program. The SBIR program is sound in concept and effective in practice. With the programmatic changes recommended here, the SBIR program should be even more effective in achieving its legislative goals.
-
Overall, the Program Has Made Significant Progress in Achieving its Congressional Objectives by:
-
Stimulating Technical Innovation. By a variety of metrics, the program is contributing to the nation’s stock of new scientific and technical knowledge. (See Finding F.)
-
Using Small Businesses to Meet Federal Research and Development Needs. SBIR program objectives are aligned with, and contribute significantly to fulfilling the mission of each studied agency.1 In some cases, closer alignment and greater integration should be possible. (See Finding C).
-
Fostering and Encouraging Participation by Minority and Disadvantaged Persons in Technological Innovation. The SBIR program supports the growth of a diverse array of small businesses, including minority- and woman-owned business, by providing market access, funding, and recognition. To better assess this support, enhanced efforts to collect better data and to monitor outcomes more closely are required at some agencies. Also, more analysis is needed to improve understanding of the obstacles faced by minority- and woman-owned
-
-
-
-
-
businesses and the nature of the measures needed to address these obstacles. (See Findings D and E.)
-
Increasing Private Sector Commercialization of Innovation Derived from Federal Research and Development. The program enables small businesses to contribute to the commercialization of the nation’s R&D investments, both through private commercial sales, as well as through government acquisition, thereby enhancing American health, welfare, and security through the introduction of new products and processes. (See Finding B.)
-
-
SBIR Is Meeting Federal R&D Needs. SBIR plays an important role in introducing innovative, science-based solutions that address the diverse mission needs of the federal agencies.
-
The program has the potential to make systematic investments in high-potential, high-risk technologies that can meet specific mission needs and/or offer the potential of significant commercial development. Balancing the potential gains from high risk, but potentially high payoff, against technologies with more promising immediate commercial benefits, represents a challenge for management and appropriate evaluation.
-
-
Improving SBIR. As structured, the management of SBIR exhibits considerable flexibility, allowing program experimentation needed to address varied agency missions and technologies. This flexibility is commendable and has enabled the program to contribute to multiple agency missions. At the same time, more regular assessment is required, both internal and external, to inform agency management of program outcomes and to improve performance through analysis of program outcomes and regular adoption of agency “best practices.” (See Section II on Recommendations.)
-
-
Commercialization.
-
Commercialization Despite High Risk. Small technology companies use SBIR awards to advance projects, develop firm-specific capabilities, and ultimately create and market new commercial products and services.
-
Given the very early stage of SBIR investments, and the high degree of technical risk often involved (reflected in risk assessment scores developed during agency selection procedures), the fact that a high proportion of projects reach the market place in some form is significant, even impressive.
-
Although the data vary by agency, responses to the NRC Phase II Survey indicate that just under half of the projects do reach the marketplace.2
-
-
2 |
See Figure 4-1. In addition, the response to the survey may reflect a degree of self-selection bias, |
-
-
Multiple Indicators of Commercial Activity. The full extent of commercialization for both government and private use is insufficiently captured by simply recording sales of a product or service. The Committee’s analysis uses five measures of commercial activity, although sales in the marketplace are obviously of particular importance. These indicators are:
-
Sales (using a range of different benchmarks to indicate different degrees of commercial activity)
-
Additional non-SBIR research funding and contracts
-
Licensing revenues
-
Third-party investment (including both venture funding and other sources of investment)
-
Additional SBIR awards for related work
-
-
A Skewed Distribution of Sales. SBIR awards result in sales numbers that are highly skewed, with a small number of awards accounting for a very large share of the overall sales generated by the program.3 This is to be expected in funding early-stage technological innovation and is broadly consistent with the general experience of other sources of early technology financing by angel investors.4 Most projects, however, do not achieve significant commercial success; a few companies do.5
-
e.g., the successful companies (those still in existence) are more likely to respond. Survey bias can manifest itself in several ways, however, and in some cases understate returns. For a fuller discussion of this important caveat, see Box 4-1 in Chapter 4: SBIR Program Outputs. |
3 |
See Figure 4-2. |
4 |
Even venture capital—a proximate referent group not appropriate for direct comparison because venture investments do not provide the same kind of project financing—shows a high returns skew. Nonetheless, returns on venture funding tend to show a similar high skew as characterize commercial returns on the SBIR awards. See John H. Cochrane, “The Risk and Return of Venture Capital,” Journal of Financial Economics, 75(1):3-52, 2005. Drawing on the VentureOne database, Cochrane plots a histogram of net venture capital returns on investments that “shows an extraordinary skewness of returns. Most returns are modest, but there is a long right tail of extraordinary good returns. 15 percent of the firms that go public or are acquired give a return greater than 1,000 percent! It is also interesting how many modest returns there are. About 15 percent of returns are less than 0, and 35 percent are less than 100 percent. An IPO or acquisition is not a guarantee of a huge return. In fact, the modal or ‘most probable’ outcome is about a 25 percent return.” See also Paul A. Gompers and Josh Lerner, “Risk and Reward in Private Equity Investments: The Challenge of Performance Assessment,” Journal of Private Equity, 1(Winter 1977):5-12. Steven D. Carden and Olive Darragh, “A Halo for Angel Investors,” The McKinsey Quarterly, 1, 2004, also show a similar skew in the distribution of returns for venture capital portfolios. |
5 |
This is expected with research that pushes the state-of-the-art where technical failures are expected and when companies face the vagaries of the procurement process. A small proportion (3-4 percent) of projects generate more than $5 million in cumulative revenues. At NIH for example, only 6 of 450 projects in the recent NIH survey were identified as generating more than $5 million in revenues. |
-
-
-
As an example, just eight of the NSF-supported projects—each of which had $2.3 million or more in sales—accounted for over half of the total sales dollars reported by respondents.
-
Similarly, one firm selected for the NRC Phase II Survey accounted for over half of all licensing income that the 162 respondents reported they earned from the NSF SBIR-supported projects.6
-
-
SBIR Is an Input, Not a Panacea. SBIR can be a key input to encourage small business commercialization, but most major commercialization successes require substantial post-SBIR research and funding from a variety of sources. SBIR awards will have been in many cases a major, even critical input—but only one of many inputs.
-
SBIR Projects Attract Significant Additional Funding. SBIR funded research projects enable small businesses to develop the technical know-how needed to attract third-party interest from a variety of public and private sources, including other federal R&D funds, angel investors, and venture funds. The NRC survey revealed that 56 percent of surveyed projects were successful in attracting additional funding from a variety of sources.7
-
Federal Funding. Responses to the NRC Phase II Survey indicate that about 10 percent of Phase II projects were eventually supported by other federal research funding; over half received at least one ad-
-
-
6 |
See National Research Council, An Assessment of the SBIR Program at the National Science Foundation, Charles W. Wessner, ed., Washington, DC: The National Academies Press, 2008, p. 214. |
7 |
See NRC Phase II Survey, Questions 22 and 23, in Appendix A. SBIR awards are related to a range of other funding events for recipients, and interviews strongly suggest that SBIR does provide a positive validation effect for third party funders. This perspective is confirmed by some angel investors who see SBIR awards as a valuable step in providing funds to develop proof of concept and encourage subsequent angel investment. Steve Weiss, Personal Communication, December 12, 2006. Weiss, an angel investor, cites SBIR awards as an example of how the public and private sectors can collaborate in bringing new technology to markets. Further, see National Research Council, An Assessment of the SBIR Program at the National Institutes of Health, Charles W. Wessner, ed., Washington, DC: The National Academies Press, 2009. |
-
-
-
ditional related Phase I SBIR award, and more than half received at least one additional related Phase II award;8
-
Licensing. Sixteen percent of projects report licensing agreements in place for their technologies, with an additional 16 percent engaged in licensing negotiations at the time of the NRC Phase II Survey.9 Some companies are making substantial use of the technology.10
-
Venture Funding. Venture capital (VC) funding has, in some cases, also played a significant role for SBIR award winners. At NIH, for example, some 25 percent of the 200 companies that received the highest number of SBIR awards had also obtained funding from venture capital, totaling some $1.59 billion.11
-
Acquisition. In some cases, the SBIR funded technology developed sufficient commercial potential that investors bought the grantee company outright. For example, in 2000, Philips bought out SBIR recipient Optiva, reportedly for the sum of $1 billion.12
-
-
Better Documentation of Commercialization Is Needed. While the recorded successes of the program are significant, commercialization outcomes are not yet adequately documented in most of the agencies reviewed. Indeed, some agencies have made very limited efforts to track the evolution of firms having received SBIR awards. Understanding outcomes and the impact of program modifications on those outcomes is essential for effective program management.
-
8 |
See response to NRC Phase II Survey, Question 23. |
9 |
See response to NRC Phase II Survey, Question 12. |
10 |
While 5.6 percent of respondents to the NRC Phase II Survey reported licensee sales greater than $0, three licenses reported more than $70 million in sales each and accounted for more than half of all reported sales, with one project alone reporting more than $200 million in licensee sales. See Section 4.2.3.4. |
11 |
See Figure 4-5. The impact of the 2004 SBA ruling that excluded firms with majority ownership by venture firms is to be empirically assessed in a follow-on study by the National Research Council. On December 3, 2004, the Small Business Administration issued a final rule saying that to be eligible for an SBIR award “an entity must be a for-profit business at least 51 percent owned and controlled by one or more U.S. individuals.” U.S. Congress, House, Committee on Science, Subcommittee on Environment, Technology, and Standards, Hearing on “Small Business Innovation Research: What is the Optimal Role of Venture Capital,” Hearing Charter, June 28, 2005. |
12 |
With SBIR funding, David Giuliani, together with University of Washington professors David Engel and Roy Martin, formed a company in 1987, eventually known as Optiva Corporation Inc., to promote a new, innovative dental hygiene device based on sonic technology. In October 2000, Royal Philips Electronics, acquired Optiva Corporation, Inc., now known as Philips Oral Healthcare. The final terms were not disclosed. Royal Philips Electronics, accessed at <http://www.homeandbody.philips.com/sonicare/gb_en/03d-story.asp>. |
-
Effective and Flexible Alignment of SBIR with Agency Mission.
-
Flexible Program Management. The effective alignment of the program with widely varying mission objectives, needs, and modes of operation is a central challenge for an award program that involves a large number of departments and agencies. The SBIR program has been adapted effectively by the management of the individual departments, services, and agencies, albeit with significant operational differences, reflecting distinct missions and operational cultures. This flexibility in program management is one of the great strengths of the program.
-
Mission Support. Each of the agency SBIR programs is effectively supporting the mission of the respective funding agency:13
-
Defense.14 At DoD, considerable progress has been made in aligning SBIR-funded research with the strategic objectives of agency research and acquisition. In some parts of DoD, e.g., some Program Executive Officers within the Navy, significant success has been achieved in improving the insertion of SBIR-funded technologies into the acquisition process. The commitment of upper management to the effective operation of the program and the provisioning of additional funding appear to be a key element of success. Teaming among the agency, the SBIR awardees, and the prime contractors is important in the transition of technologies to products to integration in systems. The growing importance of the SBIR program within the defense acquisition system is reflected in the increasing interest of primes, who are seeking opportunities to be involved with SBIR projects—a key step toward acquisition.15
-
NIH.16 NIH and other public health agencies increasingly see SBIR as an important element in the agency’s translational strategy—designed to move technologies from the lab into the marketplace.17 The NIH
-
-
13 |
This assessment is based both on procedures for selecting topics and awardees aligned with agency mission, as well as outcomes analysis. Each of the five separate volumes prepared by the Committee on the SBIR program at the five individual agencies under review explores this issue in considerable depth. The Outcomes chapter of this volume summarizes these findings in more detail. |
14 |
National Research Council, An Assessment of the SBIR Program at the Department of Defense, op. cit. |
15 |
The growing interest of Defense prime contractors is recorded in National Research Council, SBIR and the Phase III Challenge of Commercialization, Charles W. Wessner, ed., Washington, DC: The National Academies Press, 2007. |
16 |
National Research Council, An Assessment of the SBIR Program at the National Institutes of Health, op. cit. |
17 |
The importance of this strategy to NIH is reflected in the NIH RoadMap, announced in September 2003. Recent interest at NIH, led by the National Cancer Institute in conjunction with the SBIR program offices, has underscored the growth in interest in the role and potential of the program. See <http://nihroadmap.nih.gov/>. |
-
-
-
awards cases examined focused on public health and biomedical science and technology. The NRC’s review of the SBIR program at NIH documents that SBIR program funds projects have a significant impact on public health: NIH considers a project’s impact on public health during the selection process, and recipients and NIH staff note that examining potential impacts is an important component in every application review.18 The FDA’s Critical Path Initiative is focused on similar concerns.19
-
NASA.20 At NASA, the primary metric for project success is the deployment of SBIR-funded technologies on space missions, where the agency can point to a number of significant impacts.21
-
Energy.22 At DoE, as at NASA, SBIR topics and project selection are heavily influenced by the agency research staff’s wider R&D responsibilities. At DoE, all technical topics in the SBIR solicitation are constructed to support the overall mission of each of the agency’s technical program areas.
-
NSF.23 At NSF, SBIR-funded research meets the agency’s primary mission of expanding scientific and technical knowledge.24 However, the SBIR program does so by tapping the scientific and engineering capabilities of an important sector—small businesses—that is almost entirely unserved by the remainder of NSF’s programs.25
-
-
18 |
The NIH Mission is “science in pursuit of fundamental knowledge about the nature and behavior of living systems and the application of that knowledge to extend healthy life and reduce the burdens of illness and disability.” |
19 |
The FDA describes its Critical Path Initiative as “an effort to stimulate and facilitate a national effort to modernize the scientific process through which a potential human drug, biological product, or medical device is transformed from a discovery or “proof of concept” into a medical product.” Accessed at <http://www.fda.gov/oc/initiatives/criticalpath/>. |
20 |
National Research Council, An Assessment of the SBIR Program at the National Aeronautics and Space Administration, Charles W. Wessner, ed., Washington, DC: The National Academies Press, 2009. |
21 |
NASA SBIR managers report that several SBIR projects relate to the Mars Rover. While these are considered to be valuable contributions to that mission, by definition they do not represent opportunities for high-volume sales. Interview with NASA Management, December 21, 2006. |
22 |
National Research Council, An Assessment of the SBIR Program at the Department of Energy, Charles W. Wessner, ed., Washington, DC: The National Academies Press, 2008. |
23 |
National Research Council, An Assessment of the SBIR Program at the National Science Foundation, op. cit. |
24 |
NSF’s mission is “To promote the progress of science; to advance the national health, prosperity, and welfare; to secure the national defense.” Accessed at <http://www.nsf.gov/nsf/nsfpubs/straplan/mission.htm>. |
25 |
Nearly 40 percent of U.S. scientific and technical workers are employed by small businesses. U.S. Small Business Administration, Office of Advocacy (2005) data drawn from U.S. Bureau of the Census; Advocacy-funded research by Joel Popkin and Company (Research Summary #211); Federal Procurement Data System; Advocacy-funded research by CHI Research, Inc. (Research Summary |
-
The SBIR Program Provides Significant Support for Small Business.
-
Substantial Benefits. The NRC Phase II Survey and NRC Firm Survey show that the SBIR program has provided substantial benefits for participating small businesses at all agencies, in a number of different ways. These benefits include:
-
Encouraging Company Foundation. According to the NRC Firm Survey, just over 20 percent of companies indicated that they were founded entirely or partly because of an SBIR award;26
-
Supporting the Project Initiation Decision. Over two-thirds of SBIR projects reportedly would not have taken place without SBIR funding.27 This finding reflects the known difficulties in funding commercial applications for early-stage technologies. The NRC Phase II Survey shows that SBIR seems to provide funding necessary to initiate early stage projects.
-
Providing an Alternative Development Path. Companies often use SBIR to fund alternative development strategies, exploring technological options in parallel with other activities.
-
Partnering and Networking. SBIR funding helps small businesses access outside resources, especially academic consultants and partners, helping to create networks and facilitating the transfer of university knowledge to the private sector.
-
Commercializing Academic Research. In addition to fostering partnerships between academic institutions and private firms, SBIR plays an important role in encouraging academics to found new firms that can commercialize their research.28
-
-
SBIR Support Is Widely Distributed. SBIR provides support for innovation activity that is widely distributed across the small business research community.
-
Large Number of Firms. During the fourteen years between 1992 and
-
-
#225); Bureau of Labor Statistics, Current Population Survey; U.S. Department of Commerce, International Trade Administration. |
26 |
For the breakout, see Table 4-7. Chapter 4 also includes a case study of Sociometrics, Inc., which illustrate the role of SBIR in firm formation. |
27 |
See Figure 4-19 for a breakdown of the survey findings. Thirty-eight percent reported that they definitely would not have proceeded with the project without SBIR funding and an additional 33 percent reported that they probably would not have proceeded with the project without SBIR funding. |
28 |
A significant number of firms involved in the program report that the founder was previously an academic. See Finding F-3. |
-
-
-
2005, inclusive, more than 14,800 different firms received at least one Phase II award, according to the SBA Tech-Net database.29
-
Many New Participants. The agencies tracking new winners indicate that at least a third of awards at all agencies go to companies that had not previously won awards at that agency.30 This steady infusion of new firms is a major strength of the program and suggests that SBIR is encouraging innovation across a broad spectrum of firms, creating additional competition among suppliers for the agencies with procurement responsibilities, and providing agencies with new mission-oriented research and solutions.
-
-
-
Benefits for Minority- and Woman-owned Small Businesses.
-
A Key Program Objective. One of the four congressional objectives for the SBIR program is to enhance opportunities for woman- and minority-owned businesses. SBIR’s competitive awards provide a source of capital for small innovative firms with pre-prototype technologies—the phase where funding is most difficult to obtain. The certification effect that SBIR awards generate can be especially valuable as a signal to the early-stage capital markets of the potential of the project and hence of the firm.
-
A Mixed Record. Woman- and minority-owned firms face substantial challenges in obtaining early-stage finance.31 Recognizing these challenges, the legislation calls for fostering and encouraging the participation of women and minorities in SBIR. Given this objective, some current trends are troubling. Agencies do not have a uniformly positive record in funding research by woman- and minority-owned firms. Efforts to document and monitor their participation have been inadequate at, for example, NIH. It is also true, however, that developing appropriate measures in this area is complex given, inter alia, the variations in the demographics of the applicant pool.32
-
Trends in Support for Woman-owned Businesses Vary Across Agencies.
-
-
29 |
See U.S. Small Business Administration, Tech-Net Database, accessed at <http://www.sba.gov/sbir/indextechnet.html>. |
30 |
See Section 4.5.3 for a discussion of the incidence of new entrants. See also Figures 4-21, 4-22, 4-23, and 4-24 for data on new entrants at NSF, NIH, and DoD. |
31 |
Academics represent an important future pool of applicants, firm founders, principal investigators, and consultants. Recent research shows that owing to the low number of women in senior research positions in many leading academic science departments, few women have the chance to lead a spinout. “Under-representation of female academic staff in science research is the dominant (but not the only) factor to explain low entrepreneurial rates amongst female scientists.” See Peter Rosa and Alison Dawson, “Gender and the commercialization of university science: academic founders of spinout companies,” Entrepreneurship & Regional Development, 18(4):341-366, July 2006. |
32 |
-
-
-
-
At DoD, which accounts for over half of the SBIR program funding, the share of Phase II awards going to woman-owned businesses increased from 8 percent at the time of the 1992 reauthorization (1992-1994) to 9.5 percent in the most recent years covered by the NRC Phase II Survey (1999-2001).33 This percentage increase occurred in a program that expanded significantly over the same time period, both in terms of the R&D base and the SBIR allocation.
-
At NSF, in recent years, woman-owned businesses have submitted an average of 213 Phase I proposals annually, and they have received an average of 26.6 Phase I awards annually. With the exception of the bump-up in 2002 and 2003, there is no upward trend. In other words, while woman-owned businesses do participate in the NSF SBIR program, that participation does not seem to be increasing over time.
-
At NIH, awards to woman-owned businesses have increased, and their share of all awards is trending upward.34 However, the percentage of female life scientists has been growing much faster.35
-
-
-
-
Support for Minority-owned Firms Has Not Increased Proportionately.
-
DoD. The share of Phase I awards to minority-owned firms at DoD has declined quite substantially since the mid 1990s and fell below 10 percent for the first time in 2004 and 2005.36 Data on Phase II awards suggest that the decline in Phase I award shares for minority-owned firms is reflected in Phase II.
-
NSF minority firm participation is higher than that of woman-owned firms.37 However, application rates have not been rising in line with changes in the demographics of the scientific and technical workforce, and success rates for woman- and minority-owned companies continue to lag those of other small businesses.38
-
NIH. Data only recently provided by NIH raises significant con-
-
-
-
33 |
See Figure 2-4 and Figure 3-12 in National Research Council, An Assessment of the SBIR Program at the Department of Defense, op. cit. |
34 |
See Figure 2-1 in National Research Council, An Assessment of the SBIR Program at the National Institutes of Health, op. cit. |
35 |
See Section 3.2.6.4 in National Research Council, An Assessment of the SBIR Program at the National Institutes of Health, op. cit. |
36 |
This statistic is drawn from the DoD Awards Database. See Figure 4-13. |
37 |
See Figure 4.2-15 and Figure 4.2-16 in National Research Council, An Assessment of the SBIR Program at the National Science Foundation, op. cit. |
38 |
See Figure 4.2-14 in National Research Council, An Assessment of the SBIR Program at the National Science Foundation, op. cit. |
-
-
-
-
cerns about the shares of awards being made to minority-owned firms. The numbers of awards and applications, as well as success rates, have all declined for minorities, for both Phase I and Phase II. The lack of accurate and complete data suggests insufficient management attention to this component of the program’s objectives.39
-
-
-
-
Stimulating Scientific and Technological Knowledge Advances. SBIR companies have generated many patents and publications, the traditional measures of activity in this area.
-
Multiple Knowledge Outputs. SBIR projects yield a variety of knowledge outputs. These contributions to knowledge are embodied in data, scientific and engineering publications, patents and licenses of patents, presentations, analytical models, algorithms, new research equipment, reference samples, prototypes products and processes, spin-off companies, and new “human capital” (enhanced know-how, expertise, and sharing of knowledge).
-
The NRC Phase II Survey found that 34 percent of NIH projects surveyed generated at least one patent, and just over half of NIH respondents published at least one peer-reviewed article.40
-
Linking Universities to the Public and Private Markets. The SBIR program supports the transfer of research into the marketplace, as well as the general expansion of scientific and technical knowledge, through a wide variety of mechanisms. With regard to SBIR’s role in linking universities to the market, about a third of all NRC Phase II and Firm Survey respondents indicated that there had been involvement by university faculty, graduate students, and/or a university itself in developed technologies. This involvement took a number of forms.41 Among the responding companies—
-
39 |
See Figures 2-1, 2-2, 3-10, and 3-11 in National Research Council, An Assessment of the SBIR Program at the National Institutes of Health, op. cit. |
40 |
See Table 4-23 in National Research Council, An Assessment of the SBIR Program at the National Institutes of Health, op. cit. The patenting activity of SBIR firms has also been highlighted in recent Congressional testimony. Representatives of the Small Business Technology Council claimed that SBIR-related companies produced some 45,000 patents, compared to just over 2,900 patents produced by universities in 2006. The same testimony, citing the National Science Foundation, reported that some 32 percent of U.S. scientists and engineers are employed by small businesses. (Testimony of Michael Squillante before the Senate Committee on Small Business and Entrepreneurship on July 12, 2006) The Small Business Administration reports that small businesses employ 41 percent of high-tech workers, including scientists, engineers, and computer workers. U.S. Small Business Administration, Frequently Asked Questions, June 2006, accessed at <http://www.sba.gov/advo/stats/sbfaq.pdf>. |
41 |
See Section 4.6.3.1 on university faculty and company formation. Also see Table 4-13 on university involvement in SBIR projects. |
-
-
-
More than two-thirds had at least one academic founder, and more than a quarter had more than one;
-
About one-third of founders were most recently employed in an academic environment before founding the new company;
-
In some 27 percent of projects, university faculty were involved as principal investigators or consultants on the project; d. 17 percent of Phase II projects involved universities as subcontractors; and
-
15 percent of Phase II projects employed graduate students.
-
-
These data underscore the significant level of involvement by universities in the program and highlight the program’s contribution to the transition of university research to the marketplace.
-
High-risk Focus. Projects funded by SBIR often involve high technical risk, implying novel and difficult research rather than incremental change. At NSF, for example, just more than half of the Phase I projects did not get a follow-on Phase II. One third of the Phase I awardees did not apply for Phase II awards, and of those that did, many did not get a follow-on Phase II for technical reasons. The chief reason for discontinuing Phase II projects was technical failure or difficulties.42 This effort to push the technological frontier is a strength of the program and necessarily involves project, but not program, failure.
-
Indirect Path Effects. There is strong anecdotal evidence concerning “indirect path” effects—that investigators and research staff gain knowledge from projects that may become relevant in a different technical context later on, and often in another project or even another company. These effects are not directly measurable, but interviews and case studies affirm their existence and importance.43
-
-
The Measurement Challenge. The diversity of missions among agencies involved with SBIR means that metrics and indicators that capture common goals and others that reflect the diversity in mission and mechanisms (e.g., contracts or grants) must be employed to measure outcomes and impacts.
-
Differing Goals. A fundamental difference exists between those agencies that procure SBIR-funded products for their missions and those that
-
-
-
normally do not. At DoD, the program is driven by the need to put the results of research into products and systems to support war-fighters; at NIH, the focus of the SBIR program is to conduct research that contributes to improvements in public health, almost always through eventual take-up by entities outside NIH.44
-
Differing Mechanisms. These distinctly different goals are reflected in the mechanisms used by the agency to implement the SBIR program.45 For example, the DoD employs contracts whereas the NIH generally employs grants for its SBIR awards.
-
Different Metrics. Developing metrics that can capture impacts in both of these areas is a challenge. Measuring the extent to which the program supports the agency mission in any nonprocurement agency like NSF is substantially different from measures applicable to a procurement agency like DoD or NASA. In some cases, the products of NIH-funded research need to be taken up by the private sector in order to be further developed before their contribution can be realized. In other cases, such as diagnostics, software, and educational materials to encourage better health, the SBIR awards can be sufficient in themselves.
-
-
SBIR Program Flexibility Is a Positive Feature and Should Be Preserved. As structured, the management of SBIR exhibits commendable flexibility. SBA and agency management have allowed the program managers the room to adapt the program to the needs of specific technologies and unique mission needs.
-
Multiple Missions. The varied missions of the agencies that fund SBIR have encouraged SBA to take a flexible approach to implementing program guidelines. SBA has provided waivers on funding size and other programmatic matters, and agencies have used considerable ingenuity in exploring the boundaries of the guidelines, particularly in terms of support for commercialization.46
-
Commendable Flexibility. This flexibility is highly commendable and should be expanded where possible. It is precisely the relatively wide
-
-
-
range of options available to the agencies and their SBIR program managers that helps to account for the effectiveness of the program. The need for flexibility is grounded in the very wide differences in objectives, structure, and culture across the agency SBIR programs as well as in the uncertainties of early stage project finance.47
-
Program Diversity.
-
Mission Agencies. As noted above, agencies with substantial hardware and systems procurement responsibilities rely primarily on contracts as the vehicle for commissioning research—including SBIR. Contracts are oriented toward the production of results for which the contracting agency has a specific need, often via a contract with strict terms and conditions, and clear deliverables. In a contracts environment, Phase I is seen as a means of testing the feasibility of solutions for a problem, and a single Phase II may be selected to implement the most promising of these options.
-
Research Agencies. Agencies without significant hardware or systems procurement generally use grants. Grants awarded to support promising academic proposals are often initiated by researchers with interests in a specific area. At NIH, for example, topics in the SBIR solicitation serve only as guidance to areas of interest, underscoring the NIH commitment to investigator-driven research. For research agencies, almost all sales are to the private sector, and the sequential funding approach of DoD is replaced by a wider competition among all Phase I and Phase II proposals in a given technical area.
-
Consequences of Different Approaches. Contracts and grants carry with them important differences in approach and agency mindset, and consequently in outcomes, metrics, and evaluation benchmarks. Such differences permeate all aspects of the program, and underscore the need for a very flexible approach to program guidelines. The grants-oriented approach at NIH has included substantial flexibility in the application of deadlines for completion, award size, and supplemental funding among other program attributes. The contracts-driven approach at DoD has resulted in tight deadlines, more limited funding flexibility within the program, and program goals based on clear deliverables.
-
-
Commendable Innovation in Program Operation. Since the late 1990s, the SBIR program has seen considerable innovation in its operations, a trend that has accelerated during this study. As the list below shows, some SBIR program managers have increasingly utilized the flexibility inherent
-
-
-
in the structure of such a widely dispersed program to introduce a range of initiatives covering almost all aspects of SBIR program management.
-
-
These include innovations and initiatives related to:
-
Award size and duration.
-
Topic development.
-
Gap funding.
-
Phase III commercialization support.
-
Matching funds for commercialization.
-
Commercialization training for SBIR awardees.
-
Outreach into the small business community, especially to underrepresented states.
-
Increased evaluation and assessment activity.
-
-
While not all of these changes have been made at each agency, these changes taken as a whole do indicate that the program is increasingly benefiting from an innovative management culture and is becoming better adapted to serve its goals. These changes also suggest that there are likely to be further innovations and adaptations of the program to the changing needs of agencies’ missions.
-
-
Procedural Issues. A review of topic development and selection procedures, which vary among agencies, indicated that in general, agencies work hard to ensure that applicants are treated fairly, and that topics are developed in line with agency needs. Specific recommendations for improving program management are provided in subsequent sections of this report.
-
The Need for Regular Internal and External Evaluation. While the initiatives taken by several agencies to adapt their program to serve their mission needs more effectively are welcome and their flexibility to do so is commendable, these efforts require periodic assessment to determine if these ‘operational experiments’ are successful and to identify the appropriate lessons learned. Moreover, some important innovations do not appear to be based on any detailed evaluation of problems with the existing mechanisms.
-
SBA and SBIR. The SBA has oversight responsibility for the eleven SBIR programs underway across the federal government. The agency is to be commended for its flexibility in exercising its oversight responsibilities. As noted above, the agencies adaptation of the program to fit their needs and methods of operation has proven fundamental to the program’s success.
-
Modest Resources. In general, SBA has been allocated modest resources to oversee SBIR. For example, efforts to build a unified data-
-
-
-
-
base continue to encounter difficulties and resources are apparently a constraint.
-
More Flexibility. Despite the generally flexible approach, there is evidence from the agencies that in some cases, SBA has believed itself obliged to limit flexibility in the application of its guidelines to the program. For example, projects initially funded as a Phase I STTR cannot be shifted to a Phase II SBIR, even if circumstances make this the most effective way to proceed.48
-
-
Venture Funding and SBIR.
-
Synergies. There can often be useful synergies between angel and venture capital investments and SBIR funding; each of these funding sources tends to select highly promising companies and technologies. Moreover, an SBIR award provides information to potential investors by validating the technology concept and commercial potential of an innovation.
-
Angel Investment. Angel investors often find SBIR awards to be an effective mechanism to bring a company forward in its development to the point where risk is sufficiently diminished to justify investment.49
-
Venture Investment. Reflecting this synergy, initial NRC review indicates about 25 percent of the top 200 NIH Phase II award winners (1992-2005) have acquired some venture funding in addition to the SBIR awards.50
-
-
-
-
Program Change. During the first two decades of the program, some venture-backed companies participated in the program, receiving SBIR awards in conjunction with outside equity investments. During this lengthy period, no negative impact on the program’s operation or effectiveness was apparent.
In a 2002 directive, the Small Business Administration said that to be eligible for SBIR the small business concern should be “at least 51 percent owned and controlled by one or more individuals who are citizens of, or permanent resident aliens in, the United States, except in the case of a joint venture, where each entity to the venture must be 51 percent owned and controlled by one or more individuals who are citizens of, or permanent resident aliens in, the United States.”51 The effect of this directive has been to exclude companies in which VC firms have a controlling interest.52,53
-
It is important to keep in mind that the innovation process often does not follow a crisp, linear path. Venture capital funds normally (but not always) seek to invest when a firm is sufficiently developed in terms of products to offer an attractive risk-reward ratio.54 Yet even firms benefiting from venture funding may well seek SBIR awards as a
-
-
51 |
Access the SBA’s 2002 SBIR Policy Directive, Section 3(y)(3) at <http://www.zyn.com/sbir/sbres/sba-pd/pd02-S3.htm>. |
52 |
This new interpretation of “individuals” resulted in the denial by the SBA Office of Hearings and Appeals of an SBIR grant in 2003 to Cognetix, a Utah biotech company, because the company was backed by private investment firms in excess of 50 percent in the aggregate. Access this decision at <http://www.sba.gov/aboutsba/sbaprograms/oha/allcases/sizecases/siz4560.txt>. The ruling by the Administrative Law Judge stated that VC firms were not “individuals,” i.e., “natural persons,” and therefore SBIR agencies could not give SBIR grants to companies in which VC firms had a controlling interest. The biotechnology and VC industries have been dismayed by this ruling, seeing it as a new interpretation of the VC-small business relationship by SBA. See for example, testimony by Thomas Bigger of Paratek Pharmaceuticals before the U.S. Senate Committee on Small Business and Entrepreneurship, July 12, 2006. |
53 |
This paragraph is a correction of the text in the prepublication version released on July 27, 2007. |
54 |
The 1994-2004 period saw a decline in venture investments in seed and early stage and a concomitant shift away from higher-risk early-stage funding. The last few years have seen some recovery in early-stage finance. See the discussion of this point in the Introduction chapter. See also National Science Board, Science and Technology Indicators 2006, Arlington, VA: National Science Foundation, 2006. This decline is reportedly particularly acute in early-stage technology phases of biotechnology where the investment community has moved toward later-stage projects, with the consequence that early-stage projects have greater difficulty raising funds. See the testimony by Jonathan Cohen, founder and CEO of 20/20 GeneSystems, at the House Science Committee Hearing on “Small Business Innovation Research: What is the Optimal Role of Venture Capital,” July 28, 2005. |
-
-
-
means of exploring a new concept, or simply as a means of capitalizing on existing research expertise and facilities to address a health-related need or, as one participant firm explained, to explore product-oriented processes not “amenable to review” by academics who review the NIH RO1 grants.55
-
Some of the most successful NIH SBIR award winning firms—such as Martek—have, according to senior management, been successful only because they were able to attract substantial amounts of venture funding as well as SBIR awards.56
-
Other participants in the program believe that companies benefiting from venture capital ownership are essentially not small businesses and should therefore not be entitled to access the small percentage of funds set aside for small businesses, i.e., the SBIR Program. They believe further that including venture-backed firms would decrease support for high-risk innovative research in favor of low-risk product development often favored by venture funds.57
-
-
Limits on Venture Funding. The ultimate impact of the 2004 SBA ruling remains uncertain. What is certain is that no empirical assessment of its impact was made before the ruling was implemented. At the same time, the claims made by proponents and opponents of the change appear overstated.
-
Preliminary research indicates that approximately 25 percent of the most prolific NIH SBIR Phase II winners have received VC funding; that some of these are now graduates of the program (having grown too large or left for other reasons), and some are also not excluded by the
-
-
-
-
-
ruling because they are still less than 50 percent VC owned. Yet it is important to recognize that these companies may be disproportionately among the companies most likely to succeed—such as previous highly successful SBIR companies that were simultaneously recipients of VC funding.58 What is not known is how many companies are excluding themselves from the SBIR program as a result of the ruling.
-
For firms seeking to capitalize on the progress made with SBIR awards, venture funding may be the only plausible source of funding at the levels required to take a product into the commercial marketplace. Neither SBIR nor other programs at NIH are available to provide the average of $8 million per deal currently characterizing venture funding agreements.59
-
For firms with venture funding, SBIR may allow the pursuit of high-risk research or alternative path development that is not in the primary commercialization path, and hence is not budgeted for within the primary development path of the company.60
-
-
An Empirical Assessment. As noted above, the SBA ruling concerning eligibility alters the way the program has operated during the period of this review (1992-2002) and, presumably, from the program’s origin. Anecdotal evidence and initial analysis indicate that a limited number of venture-backed companies have been participating in the program. To better understand the impact of the SBA exclusion of firms receiving venture funding (resulting in majority ownership), the NIH recently commissioned an empirical analysis by the National Academies. This is a further positive step towards an assessment culture and should provide data necessary to illuminate the ramifications of this ruling.61
-
II.
NRC STUDY RECOMMENDATIONS
The recommendations in this section are designed to improve the operation of the nation’s SBIR program. They complement the core findings that the program is largely addressing its legislative goals—that significant commercialization is occurring, that the awards are making valuable additions to nation’s stock of scientific and technical knowledge, and that SBIR is developing products that apply this knowledge to agency missions, and hence to the nation’s needs.
-
Preserve Program Flexibility. Agencies, SBA, and the Congress should seek to ensure that any program adjustments made should not reduce the program’s flexibility.
-
The SBIR program is effective across the agencies partly because a “one-size-fits-all” approach has not been imposed.
-
This flexible approach should be continued, subject to appropriate monitoring, across the departments and agencies in order to adapt the program to their evolving needs and to improve its operation and output.
-
-
Regular Evaluations Are Needed.
-
The SBIR Program Is Currently Not Sufficiently Evidence-based. Some aspects of the SBIR program have been subject to internal and external evaluation.62 Nonetheless, insufficient data collection, analytic capability, and reporting requirements, together with the decentralized character of the program, mean that there is limited ability to make connections between program outcomes and program management and practices. There are several issues. These include:
-
Limited Collection of Data and Tracking of Outcomes. There is insufficient tracking of awards, as well as a widespread lack of systemic data collection and analysis, needed to support an evidence-based program culture and to enable program managers to be more effective in their stewardship of the program.
-
Limited Analyses and Use of Metrics. Changes to the program need to be better understood and the impact tracked. Because of the factors listed above, decisions that affect the operation of the program are necessarily taken with limited data and analysis, and without clear
-
-
-
-
-
benchmarks or metrics for assessing the success or failure of a given initiative.63
-
Inadequate Management Funding. Agencies administering SBIR can improve the management of the program by providing greater support for data collection and analysis. The SBIR program involves a significant amount of public funding, some $1.851 billion in FY2005. The necessary resources to manage the program and evaluate outcomes should be provided.64
-
-
Single Benchmarks of Achievement Are Problematic. The current study represents a major research effort in an area with limited previous research. Differences in both program structures and agency objectives mean that no simple benchmark for success is applicable across the entire program. Existing studies, such as those by GAO, have generally been limited in scope, focusing in most cases only on a limited aspect of the program.65 This is not surprising, given the size and diversity of the program and the difficulties in developing the comprehensive data required for accurate assessments of program outcomes (described above). To address this assessment challenge, regular, multidimensional, multiagency evaluation is required.
-
-
-
Developing a Culture of Evaluation. Recent agency initiatives toward more and better evaluation, e.g., at DoD, NIH, and NSF, represent positive steps. However, considerable further assessment is needed. The lack of an adequate assessment effort is, in some cases, the result of shortfalls in the resources needed to carry out assessments, and in others because agencies do not have the support of agency leadership for a sustained effort of evaluation and assessment.
-
Recommendations. More evaluations of the SBIR program’s outcomes are needed. Such assessment can provide the data needed to inform program management decisions and to enhance understanding of the program’s contributions.
-
Annual Reports. Top agency management should make a direct annual report to Congress on the state of the SBIR program at their agency. The report should describe management goals and commitment to the program, and measures taken to enhance the implementation of the program. This report should include a statistical appendix, which would provide data on awards, processes, outcomes, and survey information.
-
Internal Evaluation. Agencies should be encouraged—and funded—to develop improved data collection technologies and evaluation procedures. Where possible, agencies should be encouraged to develop interoperable standards for data collection and dissemination.66
-
External Evaluation. Agencies should be directed to commission an external evaluation of their SBIR programs on a regular basis.
-
Special Topic Studies. A number of further studies on special topics might help to improve program outcomes. These include:
-
The role of venture capital funding in the SBIR program.
-
The linkages and synergies between state programs and SBIR.
-
Improving links between SBIR and the acquisition community at DoD: Effects of matching fund programs.
-
Tools for reducing cycle time.
-
Gap funding mechanisms and results.
-
Phase III transition incentives.
-
Impact of increased award size.
-
-
-
-
-
-
-
Assessment of selection mechanisms.
-
Evaluation of professional commercialization consultancy services.
-
-
-
-
Additional Management Resources Are Needed.
-
To enhance program utilization, management, and evaluation, the program should be provided with additional funding for management and evaluation.
-
Effective management and evaluation requires adequate funding.67 An evidence-based program requires high quality data and systematic assessment. As noted above, sufficient resources are not currently available for these functions.
-
Increased funding is needed to provide effective oversight, including site visits, program review, systematic third-party assessments, and other necessary management activities.
-
In considering how to provide additional funds for management and evaluation, there are three ways that this might be done:
-
Additional funds might be allocated internally, within the existing budgets of the services and agencies, as the Navy has done.
-
Funds might be drawn from the existing set-aside for the program to carry out these activities.
-
The set-aside for the program, currently at 2.5 percent of external research budgets, might be marginally increased, with the goal of providing management resources necessary to maximize the program’s return to the nation.68 At the same time, increased resources should
-
-
-
-
-
not be used to create new or separate management. Increased resources should go to enable existing management to enhance their efforts with respect to the program. The Committee recommends this third option.
-
-
-
Improve Program Processes.
-
The processing periods for awards vary substantially by agency, and appear to have significant effects on recipient companies.69
-
Cycle time is a concern for small high-technology businesses with limited resources. Such small companies often have a substantial “burn rate” (outgoing monthly cash flow), and the speed with which projects are approved and funded can have a material impact on small business ventures.
-
Numerous entrepreneurs interviewed for this study identified long cycle time—that is between initial application and Phase I awards, and between Phases—as a significant problem. Some case study interviewees noted that these gaps discourage otherwise excellent companies from applying to the SBIR program.70
-
Processing schedules that are adequate for managing larger-scale research projects or academic research may not be appropriate for the small, high-technology firms that apply for SBIR awards.
-
-
Agencies Should Be Encouraged to Employ a Full Range of Tools to Reduce the Time Between Applications and Awards.
-
Monitoring and Reporting. As part of a new and expanded annual report, agencies should closely monitor and report on cycle times for each element of the SBIR program: topic development and publication, solicitation, application review, contracting, Phase II application and selection, and Phase III contracting. Agencies should also specifically report on initiatives to shorten decision cycles.
-
Best Practice Adoption. Agencies should seek to ensure that they adopt best practice in all areas of the program. Specifically, agencies should consider agency procedures that the Committee has identified that are expected to shorten the time from idea to implementation.
-
-
69 |
Drawn from case study interviews and gap data from the NRC Phase II Survey. |
70 |
See case studies at all agencies. In addition, the NRC Phase II Survey indicated that most companies had stopped work in between phases, and many had experienced other funding gaps. See Chapter 5: Program Management. |
-
-
-
Below we have provided a list of potential initiatives. Not all of these options would be appropriate for every agency, and agencies should be encouraged to experiment and assess the impact of such changes. Identified practices include:
-
Multiple annual solicitations.
-
Broader topic definitions.
-
Solicitations where no topic is necessary, or where some funding is available to projects outside the topic framework.
-
Rapid review and decisions on applications.
-
Clear and tight deadlines for completing reviews.
-
Permitting resubmission, and providing rejected applicants with reviews fast enough to ensure that they can apply again during the next cycle.
-
Use of electronic tools to augment or replace traditional selection meetings.
-
Rapid electronic delivery of results to applicants.
-
Clear and tight deadlines for contracts and grants negotiations to be completed. This may require more resources and better training for awards management staff.
-
Alignment of Phase I/Phase II applications, so that completion of Phase I comes at an appropriate time for Phase II application.
-
Fast Track mechanisms such as that operating at NIH, where Phase I and Phase II are considered as a single application.
-
Phase IIB awards which add additional agency resources on the condition that equal, or larger, amounts of private funding are also obtained.
-
Further training of technical points of contact (TPOCs) and contracting officer’s technical representatives (COTARs) to help ensure that they understand the importance of shortening cycle times.
-
-
-
-
Agencies Should Take Steps to Increase the Participation and Success Rates of Woman- and Minority-owned Firms in the SBIR Program.
-
Encourage Participation. Develop targeted outreach to improve the participation rates of woman- and minority-owned firms, and strategies to improve their success rates. These outreach efforts and other strategies
-
-
-
should be based on causal factors determined by analysis of past proposals and feedback from the affected groups.71
-
Encourage Emerging Talent. The number of women and, to a lesser extent, minorities graduating with advanced scientific and engineering degrees has been increasing significantly over the past decade, especially in the biomedical sciences. This means that many of the woman and minority scientists and engineers with the advanced degrees usually necessary to compete effectively in the SBIR program are relatively young and may not yet have arrived at the point in their careers where they own their own companies. However, they may well be ready to serve as principal investigators (PIs) and/or senior co-investigators (Co-Is) on SBIR projects. Over time, this talent pool could become a promising source of SBIR participants.
-
Improve Data Collection and Analysis. The Committee also strongly encourages the agencies to gather the data that would track woman and minority firms as well as principal investigators (PIs), and to ensure that SBIR is an effective road to opportunity. The success rates of woman and minority PIs and Co-Is is the traditional measure of their participation in the non-SBIR research grants funded by agencies like NIH and NSF. It can also be a measure of woman and minority participation in the SBIR program.
-
-
Agencies that Do Not Now Have an Independent Advisory Board that Draws Together Senior Agency Management, SBIR Managers, and Other Stakeholders as Well as Outside Experts Should Consider Creating a Board to Review Current Operations and Achievements and Recommend Changes to the SBIR Program.
-
The purpose of such an advisory board is to provide a regular monitoring and feedback mechanism that would address the need for upper management attention, and encourage internal evaluation and regular assessment of progress towards definable metrics.
-
Each agency’s annual program report could be presented to the board. The board would review the report that would include updates on program progress, management practices, and make recommendations to senior agency officials in charge.
-
The board could be assembled on the model of the Defense Science Board
-
BOX 2-1 Some Examples of Agency Best Practices A major strength of the SBIR program is its flexible adaptation to the diverse objectives, operations, and management practices at the different agencies. In some cases, however, there are examples of best practice that should be examined for possible adoption by other agencies. Examples of these best practices include: DoD: The Pre-Release Period. DoD announces the contents of its upcoming solicitations some time before the official start date of the solicitation. By attaching detailed contact information, prospective applicants can talk directly to the technical officers in charge of specific topics. This helps companies determine whether they should apply and gives the prospective applicant a better understanding of the agency needs and objectives. This informal approach provides an efficient mechanism for information exchange. Federal Acquisition Regulations prevent such discussion after formal release. DoD: Help Desk and Web Support. DoD maintains an extensive and effective Web presence for the SBIR program, which can be used by companies to resolve questions about their proposals. In addition, DoD staffs a help desk aimed at addressing nontechnical questions. This is appreciated by companies, and is strongly supported by program staff because it reduces the burden of process oriented calls on technical staff. DoD: Commercialization Tracking. DoD’s approach requires companies with previous Phase II awards to enter data into a commercialization tracking database each time these companies apply for SBIR awards at DoD. The database captures outcomes (both financial, such as sales and additional funding by source, and other benefits resulting from SBIR; e.g., public health, cost savings, improved weapon system capability, etc.) from these companies for all their previous SBIR awards, including those at other agencies. It also captures information on firm size and growth since entering the SBIR program, as well as the percent of annual revenue derived from SBIR awards. These historical results of prior awards are then used in proposal evaluation. Non-DoD agencies should consider adapting both this approach and the DoD technology and contributing to the DoD database. This would provide a unified tracking system. Adaptations could be made to track additional data for specific agencies, but this would provide a cost-effective approach to enhance data collection on award outcomes. NIH: Resubmission. In many cases, changes in topics between successive solicitations may not allow for resubmission of Phase I proposals. However, agencies should |
-
-
(DSB) or perhaps the National Science Foundation’s Advisory Board.72 In any case, it should include senior agency staff and the Director’s Office on an ex officio basis, and bring together, inter alia, representatives
-
consider selective use of resubmission of Phase II as a useful way of improving application quality. NIH: Investigator-driven Topics. While DoD and NASA have focused primarily on topics designed to address specific agency needs, NIH accepts applications that are not directly linked to an explicit topic. Other agencies might consider putting aside some portion of SBIR funds to encourage investigator driven research that does not fall within the “standard” solicitations. Multiple Agencies: Gap-reduction Strategies. The agencies have, to different degrees, recognized the importance of reducing funding gaps. While details vary, best practice would involve the development of a formal gap-reduction strategy with multiple components covering application, selection, contract negotiation, the Phase I-Phase II gap, and support after Phase II. NSF: Phase IIB, DoD Phase II Enhancement. The matching fund approach adopted by NSF for Phase IIB and by DoD for Phase II Enhancement might be explored at other agencies. The NSF matching requirement represents an important tool for helping companies to enter Phase III at non-procurement agencies. The DoD funding match by acquisition programs provides a transition link into Phase III contracts with the agency. DoD-Navy: Technology Assistance Program. The Navy has developed the most comprehensive suite of support mechanisms for companies entering Phase III, and has also developed new tools for tracking Phase III outcomes. These are important initiatives, and other components and agencies should consider them carefully. NIH: Program Flexibility. NIH is unique in the range of award size, the use of supplementary funding, the competing continuation awards (now called Phase II Competing Renewals Awards) that provide funding for post-Phase II activities, and the use of no cost extensions for awards. While this flexibility may require additional review at NIH, as recommended in the NIH Report, it is also an effective way for agencies to meet the varying needs of small research companies. DoE: Commercialization Assistance. DoE has the longest running commercial assistance program, and a broad menu of possible services. These have now been widened to include Phase I recipients. This extended range of offerings seems well adapted to improve fits between the varied Departmental needs of small business capabilities. NASA: Innovative Electronic Interface. NASA has pioneered the development of an innovative electronic interface designed to help applicants navigate the complex process of applying for awards and managers track and assess program activities. |
-
-
from industry (including award recipients), academics, and other experts in program management.
-
-
Preserve the Basic Program Structure.
-
The Phase I Bypass Proposal. Some agency staff and recipient companies have suggested that promising research has been excluded from
-
-
-
Phase II funding because all Phase II recipients must first receive a Phase I award.73 To address this concern, some program participants and agency staff have suggested that consideration be given to changing the requirement that SBIR recipients apply for and receive a Phase I award before applying for Phase II. They suggest that the application of this requirement excludes promising research that could help agencies meet their congressionally mandated goals.
-
Recommendation. The Committee recommends that fundamental changes to the Phase I, Phase II program structure should not be made.
-
Permitting companies to apply directly to Phase II could significantly change the program. In particular, it could shift the balance of both awards and funding significantly away from Phase I toward Phase II.
-
Every additional Phase II award represents funding approximately equivalent to 7.5 Phase I awards. If “direct to Phase II” were as attractive to applicants as proponents suggest, it might become a significant component of the program. This, in turn, could make a very substantial difference to funding patterns in SBIR, to the detriment of Phase I.74
-
This change could be detrimental to the program and should not be undertaken. Phase I is an important component in achieving congressional objectives and deemphasizing it should—at a minimum—require prior review and assessment.
-
-
-
Encourage Program Experimentation.
-
The Fast Track Program, whose principal objective is to reduce the funding gap between Phase I and Phase II of the SBIR award process, began as an experiment at the Department of Defense. It was subsequently evaluated by the NRC, which found that the Fast Track Program increases the effectiveness of the SBIR program at DoD by encouraging the commercialization of new technologies. That report, published in 2000, recommended that DoD consider expanding the Fast Track Program within appropriate services, organizations, and agencies within DoD. That report also called for cross-agency comparisons of the impact of Fast Track, noting that it could prove useful for the continued refinement of the program.75 DoD has since commissioned the NRC to conduct a follow up study of the Fast Track Program.
-
-
-
Funding Beyond Phase II. SBIR is designed with two funded Phases (I and II) and a third nonfunded Phase III. Projects successfully completing Phase II are expected to be well placed to obtain Phase III funding from non-SBIR sources, e.g., through procurement contracts, to attract private investment, or to deploy products directly into the marketplace.
-
While this three-phase approach has often proved successful, there are widely recognized difficulties in obtaining funding for Phase III, and consequently in achieving external commercial success or agency take-up in the agencies with major procurement responsibilities.76
-
Some agencies have sought, with the approval of SBA, to experiment with SBIR funding beyond Phase II in order to improve the commercialization potential of SBIR-funded technologies. NIH is now experimenting with tools for supporting projects through the early stages of regulatory review.
-
The NSF Phase IIB initiative and the NIH Competing Continuation Awards are positive examples that might well be adapted elsewhere.
-
-
Other Improvements. The agencies should be encouraged to develop program modifications to address possible improvements to the SBIR program, including but not limited to the pilot projects suggested above. (See Box 2-1.) SBA should make every effort to accommodate agency initiatives.
-
Evaluation of Change. Agencies should equally ensure that program modifications are designed, monitored, and evaluated, so that positive and negative results can be effectively evaluated. Where feasible and appropriate, the agencies should conduct scientifically rigorous experiments to evaluate their most promising SBIR approaches—experiments in which SBIR program applicants, awardees, and/or research topics are randomly assigned to the new approach or to a control group that participates in the agency’s usual SBIR process.77
-
-
Readjust Award Sizes.
-
Erosion of Award Value.
-
The real value of SBIR awards, last increased in 1995, has eroded due to inflation. It is now 14 years since the Congress increased the standard limits on the size of Phase I and Phase II awards. Many agency staff and award recipients have noted that in the face of continuing low but steady inflation, the amount of research funded by these awards has declined. Calculated using the NIH BioMedical Research and Development Price Index, the real value of the awards has declined by just over 35 percent.78
-
Given that Congress did not indicate that the real value of awards should be allowed to decline, this erosion in the value of awards needs to be addressed. However, many recipients contacted over the course of this study expressed concern that increases in the size of award would lead to a corresponding decline in the number of awards, if the SBIR budget is held constant.79
-
Steady increases in the number of applications for SBIR awards at most agencies (NIH is a notable recent exception) may be evidence that declining real award size has not led to a corresponding decline in interest among potential applicants.
-
-
Recommendations.
-
Phase I Increase. In order to restore the program to the approximate initial levels, adjusted for inflation, the Congress should consider making a one-time adjustment that would give the agencies latitude to increase the guidance on standard Phase I awards to $150,000.
-
Phase II Increase. In order to restore the program to the approximate initial levels, adjusted for inflation and able to sustain quality applications, the Congress should consider making a one-time adjustment that
-
-
Institute of Medicine National Roundtable on Health Care Quality, Journal of the American Medical Association, 280(11):1003, September 16, 1998; American Psychological Association, “Criteria for Evaluating Treatment Guidelines,” American Psychologist, 57(12):1052-1059, December 2002; Society for Prevention Research, Standards of Evidence: Criteria for Efficacy, Effectiveness and Dissemination, April 12, 2004, at <http://www.preventionresearch.org/softext.php>; Office of Management and Budget, What Constitutes Strong Evidence of Program Effectiveness, pp. 4-8, accessed at <http://www. whitehouse.gov/omb/part/2004_program_eval.pdf, 2004>. |
78 |
Accessed at <http://officeofbudget.od.nih.gov/PDF/BRDPI_2_5_07.xls>. |
79 |
For example, a 25 percent increase in the size of Phase II awards implies a 25 percent reduction in the number of Phase II awards, all other things being equal—or a much larger reduction in the number of Phase I awards. |
-
-
-
would give the agencies latitude to increase the guidance on standard Phase II awards to approximately $1,000,000.80
-
Flexibility in Size of Awards. It should be stressed that recommendations are intended as guidance for standard award size. The SBA should continue to provide the maximum flexibility possible with regard to award size and the agencies should continue to exercise their judgment in applying the program standard. Recognizing agencies’ need for flexibility to meet new technical or mission challenges expeditiously—such as countermeasures for biological threats or Improvised Explosive Devices—strict limits on the minimum or maximum amount for awards should be avoided. The agencies, as well, should consider whether pilot programs offering larger (or indeed smaller) awards might be useful in some cases, and whether close evaluation of large awards made in the past could help guide future practice.81
-
Flexibility—Duration. Contracting agencies might wish to experiment with a limited increase in flexibility, especially for Phase I where radical changes in technical direction are more likely to be required. Innovation implies uncertainty, so a more flexible timeline might help to attract approaches that are more technically innovative.
-
Flexibility in Additional Funding. Agencies might consider providing supplementary awards to small businesses with promising technologies at the discretion of the program manager. Supplementary awards are made at NIH, and the NSF has, for some time, implemented a Phase IIB and Phase IIB+ program. The results of these initiatives should be assessed, not least because the results may be of significant value to other agencies.82
-
-
-
-
-
This de facto focus on the Phase III transition is especially important to those agencies that generally do not acquire the products of the firms receiving their awards.
-
Transition. Larger award sizes should be phased in over two to three years in order to avoid a sudden reduction in the number of awards. The impact of the larger awards may be mitigated by the larger R&D budgets currently under consideration in the Congress. As suggested above, the transition in award sizes should be left to the individual agencies to implement in light of their current needs, capabilities, and priorities.
-
-
-
Understanding and Managing Firms Winning Multiple Awards.
-
Unproductive Multiple Award Winners Do Not Appear to Be a Major Problem, Although Continued Monitoring Is Necessary.
-
The common perception about the prevalence of mills in the SBIR program—i.e., that they have captured a large percentage of the awards, that they rarely commercialize, and that they do not meet agency research needs—is not substantiated by the evidence. Nonetheless, belief that the phenomenon of unproductive “mills,” that win repeated awards, is widespread. This perception has led to calls for a limit on the number of awards that a single firm can win.
-
The NRC data suggest that firms that repeatedly win SBIR awards but fail to commercialize are not a significant problem at any of the granting agencies.83 As the list of the top five Phase II winners (those with projects with $10 million or more in sales or investment) found in Table 2-1 further illustrates, most companies with multiple awards (but not all) are actually producing significant commercial products, often with substantial sales.
-
-
Characteristics of Multiple Award Winners. Companies that do receive large numbers of awards at the contracting agencies (notably DoD) share two characteristics:
-
If the award process is relatively objective, then one can argue that multiple award winners are high performers, able to address and meet agency needs, just as leading universities and major defense companies repeatedly win contracts for work based on a proven track record.
-
As they grow in size, the dependence of frequent award winners on SBIR declines. They typically cease to fit the SBIR-dependent model of the mills outlined above.
-
-
83 |
See Section 3.8 and Section 4.4.4.2 for related data and discussion. |
-
-
Recommendations. Because there is little evidence to suggest that multiple award winners are, in themselves, a problem, efforts to limit their participation seem misplaced. Limiting participation by unproductive winners is, of course, appropriate and already occurs. Setting a necessarily arbitrary limit on the number of awards seems neither necessary nor desirable in light of the contributions made by these firms, nor is it likely to be effective as firms could easily change names or spin out subsidiaries.
-
Improved Monitoring. This phenomenon, however, should be monitored. As agencies improve their evaluation capacity, one area of focus could be multiple award winners. DoD, for example, should continue its efforts to track both awards to multiple winners and outcomes to ensure that they generate sufficient value to the defense mission, recognizing the varied contributions made by program participants. Better information on numbers of awards and especially on outcomes would, of course, enable management to make judgments that are more informed.
-
The Use of Quotas to Reduce Applications by Multiple Winners Should Be Discontinued.
-
In the case of multiple award winners who qualify in terms of the selection criteria, the acceptance/rejection decision should be based on both the strength of their applications and their past performance rather than on the number of grants received or applications made. Firms able to provide quality solutions to solicitations should not be excluded, a priori, from the program except on clear and transparent criteria (e.g., quality of research and/or past performance).
-
Agencies should avoid imposing quotas unless there are compelling reasons to do so. Arbitrary limitations run the risk of limiting innovative ideas and of unnecessarily restricting opportunities among
-
-
-
-
-
-
-
prospective principal investigators in the larger, eligible small companies to provide high quality solutions to the government.84
-
-
-
III.
SUMMARY: AN EFFECTIVE PROGRAM
In summary, the program is proving effective in meeting congressional objectives. It is increasing innovation, encouraging participation by small companies in federal R&D, providing support for small firms owned by minorities and women, and resolving research questions for mission agencies. Should the Congress wish to provide additional funds for the program in support of these objectives, with the programmatic changes recommended above, those funds could be employed effectively by the nation’s SBIR program.
TABLE 2-1 Top Five Phase II Winners: Projects with $10 Million or More in Sales or Investment
Firm Name |
Number of Phase II Awards |
Commercialization Achievement Index (CAI) |
Agency |
Award Year |
Project Title |
Total Sales ($) |
Total Investment ($) |
Foster-Miller, Inc. |
260 |
80 |
DARPA |
1994 |
Lemmings-A Swarming Approach to Shallow Water Mine Field Clearance |
63,190,000 |
1,150,000 |
|
|
ARMY |
1996 |
Hard Faced Lightweight Composite Armor Materials for Helmets |
45,110,015 |
1,500,000 |
|
|
|
Air Force |
1992 |
Novel Aircraft Door/Panel Fastening System |
30,379,559 |
0 |
|
|
|
NASA |
1992 |
Liquid Crystal Polymers for CTE Matched PWBs |
11,203,111 |
2,295,421 |
|
|
|
Air Force |
1996 |
In Situ Biological Remediation of Hydrazine Spills Using Gel Encapsulated Enzymes |
3,000,000 |
17,466,000 |
|
|
|
Air Force |
1996 |
Integrated In-Line Wear and Lubricant Condition Sensor |
1,868,000 |
12,126,105 |
|
|
|
Air Force |
1994 |
Novel Infrared F/O Sensor for In Field Identification of Hazardous Waste Solutions |
267,505 |
49,545,842 |
|
|
|
NAVY |
1997 |
Electric Energy Absorption System for Aircraft Recovery (EARS) |
0 |
25,388,571 |
|
Physical Optics Corporation |
234 |
70 |
NASA |
1994 |
Embedded Distributed Moisture Sensor for Nondestructive Inspection of Aircraft Lap Joints |
$11,040,000 |
$1,732,000 |
|
|
AF |
1996 |
Visually Transparent Films |
$10,120,000 |
$962,000 |
|
|
|
NSF |
1997 |
The Photochemical Generation of Planar Waveguides in Sol-Gel Glasses |
$1,077,000 |
$10,500,000 |
Firm Name |
Number of Phase II Awards |
Commercialization Achievement Index (CAI) |
Agency |
Award Year |
Project Title |
Total Sales ($) |
Total Investment ($) |
Physical Sciences, Inc. |
205 |
30 |
Air Force |
1985 |
Space Shuttle Plasma Interactions |
20,711,803 |
1,409,755 |
|
|
Air Force |
1997 |
Airborne Hyperspectral Imager |
14,570,673 |
999,525 |
|
|
|
Air Force |
1997 |
Micromachined Sensor Array for Optical Meas. of Surface Pressure |
44,872 |
18,614,672 |
|
Creare, Inc. |
190 |
100 |
NASA |
1986 |
Reliable Long-Lifetime Closed-Cycle Cryocooler for Space |
1,212,529 |
39,417,294 |
|
|
NASA |
1989 |
Three-Phase Inverter for High Speed Motor Drive |
37,403 |
15,662,926 |
|
Radiation Monitoring Devices, Inc. |
140 |
95 |
HHS/NIH |
1991 |
Intraoperative Imaging Probe for Delineation of Tumors |
132,500,000 |
1,600,000 |
|
|
HHS/NIH |
1993 |
A New Portable XRF System for Lead Paint Analysis |
91,343,990 |
4,339,000 |
|
|
|
NASA |
1985 |
Portable Nuclear Cardiac Ejection Fraction Monitor |
26,600,000 |
4,500,000 |
|
|
|
HHS/NIH |
1992 |
Emboli Monitor to Reduce Surgical Neurological Deficits |
25,500,000 |
470,000 |
|
|
|
HHS/NIH |
1986 |
Sensitive Probe for Intra-Operative Bone Scanning |
19,700,000 |
1,000,000 |
|
|
|
DoE |
1984 |
Advanced Avalanche Photodiode for Positron Emission Tomography |
588,000 |
21,870,000 |
|
SOURCE: U.S. Small Business Administration, Tech-Net Database. |