2
Findings and Recommendations

I.
NATIONAL RESEARCH COUNCIL (NRC) STUDY FINDINGS

  1. The Small Business Innovation Research (SBIR) Program Is Making Significant Progress in Achieving the Congressional Goals for the Program. The SBIR program is sound in concept and effective in practice. With the programmatic changes recommended here, the SBIR program should be even more effective in achieving its legislative goals.

    1. Overall, the Program Has Made Significant Progress in Achieving its Congressional Objectives by:

      1. Stimulating Technical Innovation. By a variety of metrics, the program is contributing to the nation’s stock of new scientific and technical knowledge. (See Finding F.)

      2. Using Small Businesses to Meet Federal Research and Development Needs. SBIR program objectives are aligned with, and contribute significantly to fulfilling the mission of each studied agency.1 In some cases, closer alignment and greater integration should be possible. (See Finding C).

      3. Fostering and Encouraging Participation by Minority and Disadvantaged Persons in Technological Innovation. The SBIR program supports the growth of a diverse array of small businesses, including minority- and woman-owned business, by providing market access, funding, and recognition. To better assess this support, enhanced efforts to collect better data and to monitor outcomes more closely are required at some agencies. Also, more analysis is needed to improve understanding of the obstacles faced by minority- and woman-owned

1

The Department of Defense (DoD), National Institutes of Health (NIH), Department of Energy (DoE), National Aeronautics and Space Administration (NASA), and National Science Foundation (NSF).



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 54
2 Findings and Recommendations I. NATIONAL RESEARCH COUNCIL (NRC) STUDY FINDINGS A. The Small Business Innovation Research (SBIR) Program Is Making Significant Progress in Achieving the Congressional Goals for the Pro- gram. The SBIR program is sound in concept and effective in practice. With the programmatic changes recommended here, the SBIR program should be even more effective in achieving its legislative goals. 1. Overall, the Program Has Made Significant Progress in Achieving its Congressional Objectives by: a. Stimulating Technical Innovation. By a variety of metrics, the pro- gram is contributing to the nation’s stock of new scientific and techni- cal knowledge. (See Finding F.) b. Using Small Businesses to Meet Federal Research and Develop- ment Needs. SBIR program objectives are aligned with, and contribute significantly to fulfilling the mission of each studied agency.1 In some cases, closer alignment and greater integration should be possible. (See Finding C). c. Fostering and Encouraging Participation by Minority and Disad- vantaged Persons in Technological Innovation. The SBIR program supports the growth of a diverse array of small businesses, including minority- and woman-owned business, by providing market access, funding, and recognition. To better assess this support, enhanced ef- forts to collect better data and to monitor outcomes more closely are required at some agencies. Also, more analysis is needed to improve understanding of the obstacles faced by minority- and woman-owned 1The Department of Defense (DoD), National Institutes of Health (NIH), Department of Energy (DoE), National Aeronautics and Space Administration (NASA), and National Science Foundation (NSF). 5

OCR for page 54
55 FINDINGS AND RECOMMENDATIONS businesses and the nature of the measures needed to address these obstacles. (See Findings D and E.) d. Increasing Private Sector Commercialization of Innovation De- rived from Federal Research and Development. The program en- ables small businesses to contribute to the commercialization of the nation’s R&D investments, both through private commercial sales, as well as through government acquisition, thereby enhancing American health, welfare, and security through the introduction of new products and processes. (See Finding B.) 2. SBIR Is Meeting Federal R&D Needs. SBIR plays an important role in introducing innovative, science-based solutions that address the diverse mission needs of the federal agencies. a. The program has the potential to make systematic investments in high-potential, high-risk technologies that can meet specific mission needs and/or offer the potential of significant commercial development. Balancing the potential gains from high risk, but potentially high pay- off, against technologies with more promising immediate commer- cial benefits, represents a challenge for management and appropriate evaluation. 3. Improving SBIR. As structured, the management of SBIR exhibits con- siderable flexibility, allowing program experimentation needed to address varied agency missions and technologies. This flexibility is commendable and has enabled the program to contribute to multiple agency missions. At the same time, more regular assessment is required, both internal and ex- ternal, to inform agency management of program outcomes and to improve performance through analysis of program outcomes and regular adoption of agency “best practices.” (See Section II on Recommendations.) B. Commercialization. 1. Commercialization Despite High Risk. Small technology companies use SBIR awards to advance projects, develop firm-specific capabilities, and ultimately create and market new commercial products and services. a. Given the very early stage of SBIR investments, and the high degree of technical risk often involved (reflected in risk assessment scores developed during agency selection procedures), the fact that a high proportion of projects reach the market place in some form is signifi- cant, even impressive. b. Although the data vary by agency, responses to the NRC Phase II Survey indicate that just under half of the projects do reach the marketplace. 2 2 See Figure 4-1. In addition, the response to the survey may reflect a degree of self-selection bias,

OCR for page 54
5 AN ASSESSMENT OF THE SBIR PROGRAM 2. Multiple Indicators of Commercial Activity. The full extent of com- mercialization for both government and private use is insufficiently cap- tured by simply recording sales of a product or service. The Committee’s analysis uses five measures of commercial activity, although sales in the marketplace are obviously of particular importance. These indicators are: a. Sales (using a range of different benchmarks to indicate different de- grees of commercial activity) b. Additional non-SBIR research funding and contracts c. Licensing revenues d. Third-party investment (including both venture funding and other sources of investment) e. Additional SBIR awards for related work 3. A Skewed Distribution of Sales. SBIR awards result in sales numbers that are highly skewed, with a small number of awards accounting for a very large share of the overall sales generated by the program.3 This is to be expected in funding early-stage technological innovation and is broadly consistent with the general experience of other sources of early technol- ogy financing by angel investors.4 Most projects, however, do not achieve significant commercial success; a few companies do.5 e.g., the successful companies (those still in existence) are more likely to respond. Survey bias can manifest itself in several ways, however, and in some cases understate returns. For a fuller discussion of this important caveat, see Box 4-1 in Chapter 4: SBIR Program Outputs. 3 See Figure 4-2. 4 Even venture capital—a proximate referent group not appropriate for direct comparison because venture investments do not provide the same kind of project financing—shows a high returns skew. Nonetheless, returns on venture funding tend to show a similar high skew as characterize commer- cial returns on the SBIR awards. See John H. Cochrane, “The Risk and Return of Venture Capital,” Journal of Financial Economics, 75(1):3-52, 2005. Drawing on the VentureOne database, Cochrane plots a histogram of net venture capital returns on investments that “shows an extraordinary skew- ness of returns. Most returns are modest, but there is a long right tail of extraordinary good returns. 15 percent of the firms that go public or are acquired give a return greater than 1,000 percent! It is also interesting how many modest returns there are. About 15 percent of returns are less than 0, and 35 percent are less than 100 percent. An IPO or acquisition is not a guarantee of a huge return. In fact, the modal or ‘most probable’ outcome is about a 25 percent return.” See also Paul A. Gompers and Josh Lerner, “Risk and Reward in Private Equity Investments: The Challenge of Performance Assessment,” Journal of Priate Equity, 1(Winter 1977):5-12. Steven D. Carden and Olive Darragh, “A Halo for Angel Investors,” The McKinsey Quarterly, 1, 2004, also show a similar skew in the distribution of returns for venture capital portfolios. 5This is expected with research that pushes the state-of-the-art where technical failures are expected and when companies face the vagaries of the procurement process. A small proportion (3-4 percent) of projects generate more than $5 million in cumulative revenues. At NIH for example, only 6 of 450 projects in the recent NIH survey were identified as generating more than $5 million in revenues.

OCR for page 54
57 FINDINGS AND RECOMMENDATIONS a. As an example, just eight of the NSF-supported projects—each of which had $2.3 million or more in sales—accounted for over half of the total sales dollars reported by respondents. b. Similarly, one firm selected for the NRC Phase II Survey accounted for over half of all licensing income that the 162 respondents reported they earned from the NSF SBIR-supported projects.6 4. SBIR Is an Input, Not a Panacea. SBIR can be a key input to encourage small business commercialization, but most major commercialization suc- cesses require substantial post-SBIR research and funding from a variety of sources. SBIR awards will have been in many cases a major, even criti- cal input—but only one of many inputs. 5. SBIR Projects Attract Significant Additional Funding. SBIR funded re- search projects enable small businesses to develop the technical know-how needed to attract third-party interest from a variety of public and private sources, including other federal R&D funds, angel investors, and venture funds. The NRC survey revealed that 56 percent of surveyed projects were successful in attracting additional funding from a variety of sources. 7 a. Federal Funding. Responses to the NRC Phase II Survey indicate that about 10 percent of Phase II projects were eventually supported by other federal research funding; over half received at least one ad- Not all valuable technology innovations lead directly to high revenues, however, and SBIR has led to valuable technology innovations across the spectrum of work funded by NIH. Technologies such as the bar-coding developed by Savi using SBIR funds have been of great importance to DoD, enabling massive savings in logistics. This degree of skew underscores the limitations of random surveys that can easily miss important outcomes in terms of mission and/or sales. SBIR companies that achieve significant sales and show promise of more are often acquired. Once acquired, they normally do not respond to surveys. For example, one firm, Digital Systems Resources, Inc., was acquired by General Dynamics in September of 2003. At the time of acquisition, DSR had received 40 Phase II SBIR awards and had reported $368 million in resultant sales and investment. See Findings and Recommen- dations chapter of National Research Council, An Assessment of the SBIR Program at the Department of Defense, Charles W. Wessner, ed., Washington, DC: The National Academies Press, 2009. 6 See National Research Council, An Assessment of the SBIR Program at the National Science Foun- dation, Charles W. Wessner, ed., Washington, DC: The National Academies Press, 2008, p. 214. 7 See NRC Phase II Survey, Questions 22 and 23, in Appendix A. SBIR awards are related to a range of other funding events for recipients, and interviews strongly suggest that SBIR does provide a posi- tive validation effect for third party funders. This perspective is confirmed by some angel investors who see SBIR awards as a valuable step in providing funds to develop proof of concept and encourage subsequent angel investment. Steve Weiss, Personal Communication, December 12, 2006. Weiss, an angel investor, cites SBIR awards as an example of how the public and private sectors can collaborate in bringing new technology to markets. Further, see National Research Council, An Assessment of the SBIR Program at the National Institutes of Health, Charles W. Wessner, ed., Washington, DC: The National Academies Press, 2009.

OCR for page 54
5 AN ASSESSMENT OF THE SBIR PROGRAM ditional related Phase I SBIR award, and more than half received at least one additional related Phase II award;8 b. Licensing. Sixteen percent of projects report licensing agreements in place for their technologies, with an additional 16 percent engaged in licensing negotiations at the time of the NRC Phase II Survey.9 Some companies are making substantial use of the technology.10 c. Venture Funding. Venture capital (VC) funding has, in some cases, also played a significant role for SBIR award winners. At NIH, for example, some 25 percent of the 200 companies that received the high- est number of SBIR awards had also obtained funding from venture capital, totaling some $1.59 billion.11 d. Acquisition. In some cases, the SBIR funded technology developed sufficient commercial potential that investors bought the grantee com- pany outright. For example, in 2000, Philips bought out SBIR recipient Optiva, reportedly for the sum of $1 billion.12 6. Better Documentation of Commercialization Is Needed. While the recorded successes of the program are significant, commercialization outcomes are not yet adequately documented in most of the agencies re- viewed. Indeed, some agencies have made very limited efforts to track the evolution of firms having received SBIR awards. Understanding outcomes and the impact of program modifications on those outcomes is essential for effective program management. 8 See response to NRC Phase II Survey, Question 23. 9 See response to NRC Phase II Survey, Question 12. 10While 5.6 percent of respondents to the NRC Phase II Survey reported licensee sales greater than $0, three licenses reported more than $70 million in sales each and accounted for more than half of all reported sales, with one project alone reporting more than $200 million in licensee sales. See Section 4.2.3.4. 11 See Figure 4-5. The impact of the 2004 SBA ruling that excluded firms with majority ownership by venture firms is to be empirically assessed in a follow-on study by the National Research Council. On December 3, 2004, the Small Business Administration issued a final rule saying that to be eligible for an SBIR award “an entity must be a for-profit business at least 51 percent owned and controlled by one or more U.S. individuals.” U.S. Congress, House, Committee on Science, Subcommittee on Environment, Technology, and Standards, Hearing on “Small Business Innovation Research: What is the Optimal Role of Venture Capital,” Hearing Charter, June 28, 2005. 12With SBIR funding, David Giuliani, together with University of Washington professors David Engel and Roy Martin, formed a company in 1987, eventually known as Optiva Corporation Inc., to promote a new, innovative dental hygiene device based on sonic technology. In October 2000, Royal Philips Electronics, acquired Optiva Corporation, Inc., now known as Philips Oral Healthcare. The final terms were not disclosed. Royal Philips Electronics, accessed at .

OCR for page 54
59 FINDINGS AND RECOMMENDATIONS C. Effective and Flexible Alignment of SBIR with Agency Mission. 1. Flexible Program Management. The effective alignment of the program with widely varying mission objectives, needs, and modes of operation is a central challenge for an award program that involves a large number of departments and agencies. The SBIR program has been adapted effectively by the management of the individual departments, services, and agencies, albeit with significant operational differences, reflecting distinct missions and operational cultures. This flexibility in program management is one of the great strengths of the program. 2. Mission Support. Each of the agency SBIR programs is effectively sup- porting the mission of the respective funding agency:13 a. Defense.14 At DoD, considerable progress has been made in aligning SBIR-funded research with the strategic objectives of agency research and acquisition. In some parts of DoD, e.g., some Program Executive Officers within the Navy, significant success has been achieved in improving the insertion of SBIR-funded technologies into the acquisi- tion process. The commitment of upper management to the effective operation of the program and the provisioning of additional funding appear to be a key element of success. Teaming among the agency, the SBIR awardees, and the prime contractors is important in the transition of technologies to products to integration in systems. The growing importance of the SBIR program within the defense acquisi- tion system is reflected in the increasing interest of primes, who are seeking opportunities to be involved with SBIR projects—a key step toward acquisition.15 b. NIH.16 NIH and other public health agencies increasingly see SBIR as an important element in the agency’s translational strategy—designed to move technologies from the lab into the marketplace.17 The NIH 13This assessment is based both on procedures for selecting topics and awardees aligned with agency mission, as well as outcomes analysis. Each of the five separate volumes prepared by the Committee on the SBIR program at the five individual agencies under review explores this issue in considerable depth. The Outcomes chapter of this volume summarizes these findings in more detail. 14 National Research Council, An Assessment of the SBIR Program at the Department of Defense, op. cit. 15The growing interest of Defense prime contractors is recorded in National Research Council, SBIR and the Phase III Challenge of Commercialization, Charles W. Wessner, ed., Washington, DC: The National Academies Press, 2007. 16 National Research Council, An Assessment of the SBIR Program at the National Institutes of Health, op. cit. 17The importance of this strategy to NIH is reflected in the NIH RoadMap, announced in Septem- ber 2003. Recent interest at NIH, led by the National Cancer Institute in conjunction with the SBIR program offices, has underscored the growth in interest in the role and potential of the program. See .

OCR for page 54
0 AN ASSESSMENT OF THE SBIR PROGRAM awards cases examined focused on public health and biomedical sci- ence and technology. The NRC’s review of the SBIR program at NIH documents that SBIR program funds projects have a significant impact on public health: NIH considers a project’s impact on public health during the selection process, and recipients and NIH staff note that examining potential impacts is an important component in every ap- plication review.18 The FDA’s Critical Path Initiative is focused on similar concerns.19 c. NASA.20 At NASA, the primary metric for project success is the de- ployment of SBIR-funded technologies on space missions, where the agency can point to a number of significant impacts.21 d. Energy.22 At DoE, as at NASA, SBIR topics and project selection are heavily influenced by the agency research staff’s wider R&D re- sponsibilities. At DoE, all technical topics in the SBIR solicitation are constructed to support the overall mission of each of the agency’s technical program areas. e. NSF.23 At NSF, SBIR-funded research meets the agency’s primary mission of expanding scientific and technical knowledge.24 However, the SBIR program does so by tapping the scientific and engineering capabilities of an important sector—small businesses—that is almost entirely unserved by the remainder of NSF’s programs.25 18The NIH Mission is “science in pursuit of fundamental knowledge about the nature and behavior of living systems and the application of that knowledge to extend healthy life and reduce the burdens of illness and disability.” 19The FDA describes its Critical Path Initiative as “an effort to stimulate and facilitate a national effort to modernize the scientific process through which a potential human drug, biological product, or medical device is transformed from a discovery or “proof of concept” into a medical product.” Accessed at . 20 National Research Council, An Assessment of the SBIR Program at the National Aeronautics and Space Administration, Charles W. Wessner, ed., Washington, DC: The National Academies Press, 2009. 21 NASA SBIR managers report that several SBIR projects relate to the Mars Rover. While these are considered to be valuable contributions to that mission, by definition they do not represent opportuni- ties for high-volume sales. Interview with NASA Management, December 21, 2006. 22 National Research Council, An Assessment of the SBIR Program at the Department of Energy, Charles W. Wessner, ed., Washington, DC: The National Academies Press, 2008. 23 National Research Council, An Assessment of the SBIR Program at the National Science Founda- tion, op. cit. 24 NSF’s mission is “To promote the progress of science; to advance the national health, prosperity, and welfare; to secure the national defense.” Accessed at . 25 Nearly 40 percent of U.S. scientific and technical workers are employed by small businesses. U.S. Small Business Administration, Office of Advocacy (2005) data drawn from U.S. Bureau of the Census; Advocacy-funded research by Joel Popkin and Company (Research Summary #211); Federal Procurement Data System; Advocacy-funded research by CHI Research, Inc. (Research Summary

OCR for page 54
 FINDINGS AND RECOMMENDATIONS D. The SBIR Program Provides Significant Support for Small Business. 1. Substantial Benefits. The NRC Phase II Survey and NRC Firm Survey show that the SBIR program has provided substantial benefits for par- ticipating small businesses at all agencies, in a number of different ways. These benefits include: a. Encouraging Company Foundation. According to the NRC Firm Survey, just over 20 percent of companies indicated that they were founded entirely or partly because of an SBIR award;26 b. Supporting the Project Initiation Decision. Over two-thirds of SBIR projects reportedly would not have taken place without SBIR fund- ing.27 This finding reflects the known difficulties in funding com- mercial applications for early-stage technologies. The NRC Phase II Survey shows that SBIR seems to provide funding necessary to initiate early stage projects. c. Providing an Alternative Development Path. Companies often use SBIR to fund alternative development strategies, exploring technologi- cal options in parallel with other activities. d. Partnering and Networking. SBIR funding helps small businesses access outside resources, especially academic consultants and partners, helping to create networks and facilitating the transfer of university knowledge to the private sector. e. Commercializing Academic Research. In addition to fostering part- nerships between academic institutions and private firms, SBIR plays an important role in encouraging academics to found new firms that can commercialize their research.28 2. SBIR Support Is Widely Distributed. SBIR provides support for innova- tion activity that is widely distributed across the small business research community. a. Large Number of Firms. During the fourteen years between 1992 and #225); Bureau of Labor Statistics, Current Population Survey; U.S. Department of Commerce, Inter- national Trade Administration. 26 For the breakout, see Table 4-7. Chapter 4 also includes a case study of Sociometrics, Inc., which illustrate the role of SBIR in firm formation. 27 See Figure 4-19 for a breakdown of the survey findings. Thirty-eight percent reported that they definitely would not have proceeded with the project without SBIR funding and an additional 33 per- cent reported that they probably would not have proceeded with the project without SBIR funding. 28A significant number of firms involved in the program report that the founder was previously an academic. See Finding F-3.

OCR for page 54
 AN ASSESSMENT OF THE SBIR PROGRAM 2005, inclusive, more than 14,800 different firms received at least one Phase II award, according to the SBA Tech-Net database.29 b. Many New Participants. The agencies tracking new winners indicate that at least a third of awards at all agencies go to companies that had not previously won awards at that agency.30 This steady infusion of new firms is a major strength of the program and suggests that SBIR is encouraging innovation across a broad spectrum of firms, creating additional competition among suppliers for the agencies with pro- curement responsibilities, and providing agencies with new mission- oriented research and solutions. E. Benefits for Minority- and Woman-owned Small Businesses. 1. A Key Program Objective. One of the four congressional objectives for the SBIR program is to enhance opportunities for woman- and minority- owned businesses. SBIR’s competitive awards provide a source of capital for small innovative firms with pre-prototype technologies—the phase where funding is most difficult to obtain. The certification effect that SBIR awards generate can be especially valuable as a signal to the early-stage capital markets of the potential of the project and hence of the firm. 2. A Mixed Record. Woman- and minority-owned firms face substantial challenges in obtaining early-stage finance.31 Recognizing these chal- lenges, the legislation calls for fostering and encouraging the participation of women and minorities in SBIR. Given this objective, some current trends are troubling. Agencies do not have a uniformly positive record in funding research by woman- and minority-owned firms. Efforts to docu- ment and monitor their participation have been inadequate at, for example, NIH. It is also true, however, that developing appropriate measures in this area is complex given, inter alia, the variations in the demographics of the applicant pool.32 a. Trends in Support for Woman-owned Businesses Vary Across Agencies. 29 See U.S. Small Business Administration, Tech-Net Database, accessed at . 30 See Section 4.5.3 for a discussion of the incidence of new entrants. See also Figures 4-21, 4-22, 4-23, and 4-24 for data on new entrants at NSF, NIH, and DoD. 31Academics represent an important future pool of applicants, firm founders, principal investigators, and consultants. Recent research shows that owing to the low number of women in senior research positions in many leading academic science departments, few women have the chance to lead a spinout. “Under-representation of female academic staff in science research is the dominant (but not the only) factor to explain low entrepreneurial rates amongst female scientists.” See Peter Rosa and Alison Dawson, “Gender and the commercialization of university science: academic founders of spinout companies,” Entrepreneurship & Regional Deelopment, 18(4):341-366, July 2006. 32 See Chapters 3, 4, and 5 for additional discussion.

OCR for page 54
 FINDINGS AND RECOMMENDATIONS At DoD, which accounts for over half of the SBIR program fund- i. ing, the share of Phase II awards going to woman-owned businesses increased from 8 percent at the time of the 1992 reauthorization (1992-1994) to 9.5 percent in the most recent years covered by the NRC Phase II Survey (1999-2001).33 This percentage increase oc- curred in a program that expanded significantly over the same time period, both in terms of the R&D base and the SBIR allocation. ii. At NSF, in recent years, woman-owned businesses have submit- ted an average of 213 Phase I proposals annually, and they have received an average of 26.6 Phase I awards annually. With the exception of the bump-up in 2002 and 2003, there is no upward trend. In other words, while woman-owned businesses do partici- pate in the NSF SBIR program, that participation does not seem to be increasing over time. iii. At NIH, awards to woman-owned businesses have increased, and their share of all awards is trending upward.34 However, the per- centage of female life scientists has been growing much faster. 35 b. Support for Minority-owned Firms Has Not Increased Proportionately. DoD. The share of Phase I awards to minority-owned firms at i. DoD has declined quite substantially since the mid 1990s and fell below 10 percent for the first time in 2004 and 2005.36 Data on Phase II awards suggest that the decline in Phase I award shares for minority-owned firms is reflected in Phase II. ii. NSF minority firm participation is higher than that of woman- owned firms.37 However, application rates have not been rising in line with changes in the demographics of the scientific and techni- cal workforce, and success rates for woman- and minority-owned companies continue to lag those of other small businesses.38 iii. NIH. Data only recently provided by NIH raises significant con- 33 See Figure 2-4 and Figure 3-12 in National Research Council, An Assessment of the SBIR Pro- gram at the Department of Defense, op. cit. 34 See Figure 2-1 in National Research Council, An Assessment of the SBIR Program at the National Institutes of Health, op. cit. 35 See Section 3.2.6.4 in National Research Council, An Assessment of the SBIR Program at the National Institutes of Health, op. cit. 36This statistic is drawn from the DoD Awards Database. See Figure 4-13. 37 See Figure 4.2-15 and Figure 4.2-16 in National Research Council, An Assessment of the SBIR Program at the National Science Foundation, op. cit. 38 See Figure 4.2-14 in National Research Council, An Assessment of the SBIR Program at the National Science Foundation, op. cit.

OCR for page 54
 AN ASSESSMENT OF THE SBIR PROGRAM cerns about the shares of awards being made to minority-owned firms. The numbers of awards and applications, as well as suc- cess rates, have all declined for minorities, for both Phase I and Phase II. The lack of accurate and complete data suggests insuf- ficient management attention to this component of the program’s objectives.39 F. Stimulating Scientific and Technological Knowledge Advances. SBIR companies have generated many patents and publications, the traditional measures of activity in this area. 1. Multiple Knowledge Outputs. SBIR projects yield a variety of knowl- edge outputs. These contributions to knowledge are embodied in data, scientific and engineering publications, patents and licenses of patents, presentations, analytical models, algorithms, new research equipment, reference samples, prototypes products and processes, spin-off companies, and new “human capital” (enhanced know-how, expertise, and sharing of knowledge). 2. The NRC Phase II Survey found that 34 percent of NIH projects surveyed generated at least one patent, and just over half of NIH respondents pub- lished at least one peer-reviewed article.40 3. Linking Universities to the Public and Private Markets. The SBIR program supports the transfer of research into the marketplace, as well as the general expansion of scientific and technical knowledge, through a wide variety of mechanisms. With regard to SBIR’s role in linking uni- versities to the market, about a third of all NRC Phase II and Firm Survey respondents indicated that there had been involvement by university fac- ulty, graduate students, and/or a university itself in developed technolo- gies. This involvement took a number of forms.41 Among the responding companies— 39 See Figures 2-1, 2-2, 3-10, and 3-11 in National Research Council, An Assessment of the SBIR Program at the National Institutes of Health, op. cit. 40 See Table 4-23 in National Research Council, An Assessment of the SBIR Program at the National Institutes of Health, op. cit. The patenting activity of SBIR firms has also been highlighted in recent Congressional testimony. Representatives of the Small Business Technology Council claimed that SBIR-related companies produced some 45,000 patents, compared to just over 2,900 patents pro- duced by universities in 2006. The same testimony, citing the National Science Foundation, reported that some 32 percent of U.S. scientists and engineers are employed by small businesses. (Testimony of Michael Squillante before the Senate Committee on Small Business and Entrepreneurship on July 12, 2006) The Small Business Administration reports that small businesses employ 41 percent of high-tech workers, including scientists, engineers, and computer workers. U.S. Small Business Administration, Frequently Asked Questions, June 2006, accessed at . 41 See Section 4.6.3.1 on university faculty and company formation. Also see Table 4-13 on uni- versity involvement in SBIR projects.

OCR for page 54
0 AN ASSESSMENT OF THE SBIR PROGRAM BOX 2-1 Some Examples of Agency Best Practices A major strength of the SBIR program is its flexible adaptation to the diverse objec- tives, operations, and management practices at the different agencies. In some cases, however, there are examples of best practice that should be examined for possible adoption by other agencies. Examples of these best practices include: DoD: The Pre-Release Period. DoD announces the contents of its upcoming solicita- tions some time before the official start date of the solicitation. By attaching detailed contact information, prospective applicants can talk directly to the technical officers in charge of specific topics. This helps companies determine whether they should ap- ply and gives the prospective applicant a better understanding of the agency needs and objectives. This informal approach provides an efficient mechanism for informa- tion exchange. Federal Acquisition Regulations prevent such discussion after formal release. DoD: Help Desk and Web Support. DoD maintains an extensive and effective Web presence for the SBIR program, which can be used by companies to resolve ques- tions about their proposals. In addition, DoD staffs a help desk aimed at addressing nontechnical questions. This is appreciated by companies, and is strongly supported by program staff because it reduces the burden of process oriented calls on technical staff. DoD: Commercialization Tracking. DoD’s approach requires companies with previ- ous Phase II awards to enter data into a commercialization tracking database each time these companies apply for SBIR awards at DoD. The database captures outcomes (both financial, such as sales and additional funding by source, and other benefits resulting from SBIR; e.g., public health, cost savings, improved weapon system capa- bility, etc.) from these companies for all their previous SBIR awards, including those at other agencies. It also captures information on firm size and growth since entering the SBIR program, as well as the percent of annual revenue derived from SBIR awards. These historical results of prior awards are then used in proposal evaluation. Non-DoD agencies should consider adapting both this approach and the DoD tech- nology and contributing to the DoD database. This would provide a unified tracking system. Adaptations could be made to track additional data for specific agencies, but this would provide a cost-effective approach to enhance data collection on award outcomes. NIH: Resubmission. In many cases, changes in topics between successive solicita- tions may not allow for resubmission of Phase I proposals. However, agencies should (DSB) or perhaps the National Science Foundation’s Advisory Board.72 In any case, it should include senior agency staff and the Director’s Of- fice on an ex officio basis, and bring together, inter alia, representatives 72The intent here is to use the DSB or the NSF Board as a model, not something necessarily to be copied exactly.

OCR for page 54
 FINDINGS AND RECOMMENDATIONS consider selective use of resubmission of Phase II as a useful way of improving ap- plication quality. NIH: Investigator-driven Topics. While DoD and NASA have focused primarily on topics designed to address specific agency needs, NIH accepts applications that are not directly linked to an explicit topic. Other agencies might consider putting aside some portion of SBIR funds to encourage investigator driven research that does not fall within the “standard” solicitations. Multiple Agencies: Gap-reduction Strategies. The agencies have, to different de- grees, recognized the importance of reducing funding gaps. While details vary, best practice would involve the development of a formal gap-reduction strategy with multiple components covering application, selection, contract negotiation, the Phase I-Phase II gap, and support after Phase II. NSF: Phase IIB, DoD Phase II Enhancement. The matching fund approach adopted by NSF for Phase IIB and by DoD for Phase II Enhancement might be explored at other agencies. The NSF matching requirement represents an important tool for help- ing companies to enter Phase III at non-procurement agencies. The DoD funding match by acquisition programs provides a transition link into Phase III contracts with the agency. DoD-Navy: Technology Assistance Program. The Navy has developed the most comprehensive suite of support mechanisms for companies entering Phase III, and has also developed new tools for tracking Phase III outcomes. These are important initiatives, and other components and agencies should consider them carefully. NIH: Program Flexibility. NIH is unique in the range of award size, the use of supple- mentary funding, the competing continuation awards (now called Phase II Competing Renewals Awards) that provide funding for post-Phase II activities, and the use of no cost extensions for awards. While this flexibility may require additional review at NIH, as recommended in the NIH Report, it is also an effective way for agencies to meet the varying needs of small research companies. DoE: Commercialization Assistance. DoE has the longest running commercial assis- tance program, and a broad menu of possible services. These have now been widened to include Phase I recipients. This extended range of offerings seems well adapted to improve fits between the varied Departmental needs of small business capabilities. NASA: Innovative Electronic Interface. NASA has pioneered the development of an innovative electronic interface designed to help applicants navigate the complex pro- cess of applying for awards and managers track and assess program activities. from industry (including award recipients), academics, and other experts in program management. G. Preserve the Basic Program Structure. 1. The Phase I Bypass Proposal. Some agency staff and recipient com- panies have suggested that promising research has been excluded from

OCR for page 54
 AN ASSESSMENT OF THE SBIR PROGRAM Phase II funding because all Phase II recipients must first receive a Phase I award.73 To address this concern, some program participants and agency staff have suggested that consideration be given to changing the require- ment that SBIR recipients apply for and receive a Phase I award before applying for Phase II. They suggest that the application of this requirement excludes promising research that could help agencies meet their congres- sionally mandated goals. 2. Recommendation. The Committee recommends that fundamental changes to the Phase I, Phase II program structure should not be made. a. Permitting companies to apply directly to Phase II could significantly change the program. In particular, it could shift the balance of both awards and funding significantly away from Phase I toward Phase II. b. Every additional Phase II award represents funding approximately equivalent to 7.5 Phase I awards. If “direct to Phase II” were as attrac- tive to applicants as proponents suggest, it might become a significant component of the program. This, in turn, could make a very substantial difference to funding patterns in SBIR, to the detriment of Phase I. 74 c. This change could be detrimental to the program and should not be un- dertaken. Phase I is an important component in achieving congressio- nal objectives and deemphasizing it should—at a minimum—require prior review and assessment. H. Encourage Program Experimentation. 1. The Fast Track Program, whose principal objective is to reduce the funding gap between Phase I and Phase II of the SBIR award process, began as an experiment at the Department of Defense. It was subsequently evaluated by the NRC, which found that the Fast Track Program increases the effectiveness of the SBIR program at DoD by encouraging the com- mercialization of new technologies. That report, published in 2000, rec- ommended that DoD consider expanding the Fast Track Program within appropriate services, organizations, and agencies within DoD. That report also called for cross-agency comparisons of the impact of Fast Track, noting that it could prove useful for the continued refinement of the pro- gram.75 DoD has since commissioned the NRC to conduct a follow up study of the Fast Track Program. 73 Discussions with NIH SBIR program managers, June 13, 2006. 74 Phase I awards may have particular importance in meeting noncommercial objectives of the program, for example, helping academics to transition technologies out of the lab into startup companies. 75 National Research Council, The Small Business Innoation Research Program: An Assessment of the Department of Defense Fast Track Initiatie, op. cit., pp. 33-39.

OCR for page 54
 FINDINGS AND RECOMMENDATIONS 2. Funding Beyond Phase II. SBIR is designed with two funded Phases (I and II) and a third nonfunded Phase III. Projects successfully completing Phase II are expected to be well placed to obtain Phase III funding from non-SBIR sources, e.g., through procurement contracts, to attract private investment, or to deploy products directly into the marketplace. a. While this three-phase approach has often proved successful, there are widely recognized difficulties in obtaining funding for Phase III, and consequently in achieving external commercial success or agency take-up in the agencies with major procurement responsibilities. 76 b. Some agencies have sought, with the approval of SBA, to experiment with SBIR funding beyond Phase II in order to improve the commer- cialization potential of SBIR-funded technologies. NIH is now experi- menting with tools for supporting projects through the early stages of regulatory review. c. The NSF Phase IIB initiative and the NIH Competing Continuation Awards are positive examples that might well be adapted elsewhere. 3. Other Improvements. The agencies should be encouraged to develop program modifications to address possible improvements to the SBIR program, including but not limited to the pilot projects suggested above. (See Box 2-1.) SBA should make every effort to accommodate agency initiatives. 4. Evaluation of Change. Agencies should equally ensure that program modifications are designed, monitored, and evaluated, so that positive and negative results can be effectively evaluated. Where feasible and appro- priate, the agencies should conduct scientifically rigorous experiments to evaluate their most promising SBIR approaches—experiments in which SBIR program applicants, awardees, and/or research topics are randomly assigned to the new approach or to a control group that participates in the agency’s usual SBIR process.77 76 For a discussion of Phase III commercialization issues, see National Research Council, SBIR and the Phase III Challenge of Commercialization, op. cit. 77 Randomized evaluations are recognized as a highly rigorous means for evaluating the effective- ness of a strategy or approach across many diverse fields. When properly conceived and executed, they enable one to determine to a high degree of confidence whether the new approach itself, as op- posed to other factors, causes the observed outcomes. Such randomized evaluations are recognized as an effective means for evaluating an intervention’s effectiveness across many diverse fields. When properly conceived and executed, they enable one to determine with some confidence whether the intervention itself, as opposed to other factors, causes the observed outcomes. See, for example, U.S. Department of Education, “Scientifically-Based Evaluation Methods: Notice of Final Priority,” Federal Register, 70(15):3586-3589, January 25, 2005; the Food and Drug Admin- istration’s standard for assessing the effectiveness of pharmaceutical drugs and medical devices, at 21 C.F.R. §314.12; “The Urgent Need to Improve Health Care Quality,” Consensus statement of the

OCR for page 54
 AN ASSESSMENT OF THE SBIR PROGRAM Readjust Award Sizes. I. 1. Erosion of Award Value. a. The real value of SBIR awards, last increased in 1995, has eroded due to inflation. It is now 14 years since the Congress increased the stan- dard limits on the size of Phase I and Phase II awards. Many agency staff and award recipients have noted that in the face of continuing low but steady inflation, the amount of research funded by these awards has declined. Calculated using the NIH BioMedical Research and Devel- opment Price Index, the real value of the awards has declined by just over 35 percent.78 b. Given that Congress did not indicate that the real value of awards should be allowed to decline, this erosion in the value of awards needs to be addressed. However, many recipients contacted over the course of this study expressed concern that increases in the size of award would lead to a corresponding decline in the number of awards, if the SBIR budget is held constant.79 c. Steady increases in the number of applications for SBIR awards at most agencies (NIH is a notable recent exception) may be evidence that declining real award size has not led to a corresponding decline in interest among potential applicants. 2. Recommendations. a. Phase I Increase. In order to restore the program to the approximate initial levels, adjusted for inflation, the Congress should consider mak- ing a one-time adjustment that would give the agencies latitude to increase the guidance on standard Phase I awards to $150,000. b. Phase II Increase. In order to restore the program to the approximate initial levels, adjusted for inflation and able to sustain quality applica- tions, the Congress should consider making a one-time adjustment that Institute of Medicine National Roundtable on Health Care Quality, Journal of the American Medi- cal Association, 280(11):1003, September 16, 1998; American Psychological Association, “Criteria for Evaluating Treatment Guidelines,” American Psychologist, 57(12):1052-1059, December 2002; Society for Prevention Research, Standards of Eidence: Criteria for Efficacy, Effectieness and Dis- semination, April 12, 2004, at ; Office of Manage- ment and Budget, What Constitutes Strong Eidence of Program Effectieness, pp. 4-8, accessed at . 78Accessed at . 79 For example, a 25 percent increase in the size of Phase II awards implies a 25 percent reduction in the number of Phase II awards, all other things being equal—or a much larger reduction in the number of Phase I awards.

OCR for page 54
5 FINDINGS AND RECOMMENDATIONS would give the agencies latitude to increase the guidance on standard Phase II awards to approximately $1,000,000.80 c. Flexibility in Size of Awards. It should be stressed that recommen- dations are intended as guidance for standard award size. The SBA should continue to provide the maximum flexibility possible with regard to award size and the agencies should continue to exercise their judgment in applying the program standard. Recognizing agencies’ need for flexibility to meet new technical or mission challenges expedi- tiously—such as countermeasures for biological threats or Improvised Explosive Devices—strict limits on the minimum or maximum amount for awards should be avoided. The agencies, as well, should consider whether pilot programs offering larger (or indeed smaller) awards might be useful in some cases, and whether close evaluation of large awards made in the past could help guide future practice.81 d. Flexibility—Duration. Contracting agencies might wish to experi- ment with a limited increase in flexibility, especially for Phase I where radical changes in technical direction are more likely to be required. Innovation implies uncertainty, so a more flexible timeline might help to attract approaches that are more technically innovative. e. Flexibility in Additional Funding. Agencies might consider provid- ing supplementary awards to small businesses with promising tech- nologies at the discretion of the program manager. Supplementary awards are made at NIH, and the NSF has, for some time, implemented a Phase IIB and Phase IIB+ program. The results of these initiatives should be assessed, not least because the results may be of significant value to other agencies.82 80 Recognizing that these values are not identical, the erosion by inflation of the amounts available for Phase I appear more constraining than for Phase II. One may argue that, for parity, the Phase II should be raised to approximately $1,125,000; the trade-offs involve attracting adequate numbers of quality proposals, and providing sufficient resources to enable firms to actually carry out the necessary work, while also recognizing the impact on the number of awards. While recognizing that there is necessarily an arbitrary element in fixing these amounts, the Committee is confident that a $1,000,000 Phase II award will maintain the program’s attraction to innovative small businesses. 81 Program innovations that offer flexibility in award size introduced at NIH appear promising, though there are currently no data to determine whether this approach has resulted in more success- ful projects. Several agencies, NIH, NSF, and NASA, have successfully made their own judgments as to the most appropriate award sizes to meet their agency needs and continue to draw adequate “deal flows.” Agencies with smaller SBIR programs, not reviewed here, have adopted much smaller award sizes. 82 See National Research Council, An Assessment of the SBIR Program at the National Institutes of Health, op. cit., Awards chapter. See also the discussion of the Phase IIB program in National Re- search Council, An Assessment of the SBIR Program at the National Science Foundation, op. cit.

OCR for page 54
 AN ASSESSMENT OF THE SBIR PROGRAM This de facto focus on the Phase III transition is especially important to those agencies that generally do not acquire the products of the firms receiving their awards. f. Transition. Larger award sizes should be phased in over two to three years in order to avoid a sudden reduction in the number of awards. The impact of the larger awards may be mitigated by the larger R&D budgets currently under consideration in the Congress. As suggested above, the transition in award sizes should be left to the individual agencies to implement in light of their current needs, capabilities, and priorities. J. Understanding and Managing Firms Winning Multiple Awards. 1. Unproductive Multiple Award Winners Do Not Appear to Be a Major Problem, Although Continued Monitoring Is Necessary. a. The common perception about the prevalence of mills in the SBIR program—i.e., that they have captured a large percentage of the awards, that they rarely commercialize, and that they do not meet agency research needs—is not substantiated by the evidence. Nonetheless, belief that the phenomenon of unproductive “mills,” that win repeated awards, is widespread. This perception has led to calls for a limit on the number of awards that a single firm can win. b. The NRC data suggest that firms that repeatedly win SBIR awards but fail to commercialize are not a significant problem at any of the granting agencies.83 As the list of the top five Phase II winners (those with projects with $10 million or more in sales or investment) found in Table 2-1 further illustrates, most companies with multiple awards (but not all) are actually producing significant commercial products, often with substantial sales. 2. Characteristics of Multiple Award Winners. Companies that do receive large numbers of awards at the contracting agencies (notably DoD) share two characteristics: a. If the award process is relatively objective, then one can argue that multiple award winners are high performers, able to address and meet agency needs, just as leading universities and major defense companies repeatedly win contracts for work based on a proven track record. b. As they grow in size, the dependence of frequent award winners on SBIR declines. They typically cease to fit the SBIR-dependent model of the mills outlined above. 83 See Section 3.8 and Section 4.4.4.2 for related data and discussion.

OCR for page 54
7 FINDINGS AND RECOMMENDATIONS 3. Recommendations. Because there is little evidence to suggest that mul- tiple award winners are, in themselves, a problem, efforts to limit their participation seem misplaced. Limiting participation by unproductive win- ners is, of course, appropriate and already occurs. Setting a necessarily arbitrary limit on the number of awards seems neither necessary nor desir- able in light of the contributions made by these firms, nor is it likely to be effective as firms could easily change names or spin out subsidiaries. a. Improved Monitoring. This phenomenon, however, should be moni- tored. As agencies improve their evaluation capacity, one area of focus could be multiple award winners. DoD, for example, should continue its efforts to track both awards to multiple winners and outcomes to ensure that they generate sufficient value to the defense mission, recognizing the varied contributions made by program participants. Better information on numbers of awards and especially on outcomes would, of course, enable management to make judgments that are more informed. b. The Use of Quotas to Reduce Applications by Multiple Winners Should Be Discontinued. i. In the case of multiple award winners who qualify in terms of the selection criteria, the acceptance/rejection decision should be based on both the strength of their applications and their past per- formance rather than on the number of grants received or applica- tions made. Firms able to provide quality solutions to solicitations should not be excluded, a priori, from the program except on clear and transparent criteria (e.g., quality of research and/or past performance). ii. Agencies should avoid imposing quotas unless there are compelling reasons to do so. Arbitrary limitations run the risk of limiting in- novative ideas and of unnecessarily restricting opportunities among

OCR for page 54
 AN ASSESSMENT OF THE SBIR PROGRAM prospective principal investigators in the larger, eligible small com- panies to provide high quality solutions to the government.84 III. SUMMARY: AN EFFECTIVE PROGRAM In summary, the program is proving effective in meeting congressional objec- tives. It is increasing innovation, encouraging participation by small companies in federal R&D, providing support for small firms owned by minorities and women, and resolving research questions for mission agencies. Should the Congress wish to provide additional funds for the program in support of these objectives, with the programmatic changes recommended above, those funds could be employed effectively by the nation’s SBIR program. 84 Creare, Inc., has received a significant number of SBIR awards and now has 21 patents resulting from SBIR-funded work. Staff members have published dozens of papers. The firm has licensed tech- nologies including high-torque threaded fasteners, a breast cancer surgery aid, corrosion preventative coverings, an electronic regulator for firefighters, and mass vaccination devices (pending). Products and services developed at Creare include thermal-fluid modeling and testing, miniature vacuum pumps, fluid dynamics simulation software, network software for data exchange. A significant techni- cal accomplishment was achieved with the NCS Cryocooler used on the Hubble Space Telescope to restore the operation of the telescope’s near-infrared imaging device. In some cases, the company has developed technical capabilities that have remained latent for years until a problem arose for which those capabilities were required. The cryogenic cooler for the Hubble telescope is an example. The technologies that were required to build that cryogenic refrigera- tor started being developed in the early 1980s as one of Creare’s first SBIR projects. Over 20 years, Creare received over a dozen SBIR projects to develop the technologies that ultimately were used in the cryogenic cooler.

OCR for page 54
TABLE 2-1 Top Five Phase II Winners: Projects with $10 Million or More in Sales or Investment Number of Commercialization Total Phase II Achievement Award Total Sales Investment Firm Name Awards Index (CAI) Agency Year Project Title ($) ($) Foster- 260 80 DARPA 1994 Lemmings—A Swarming Approach to Shallow Water 63,190,000 1,150,000 Miller, Inc. Mine Field Clearance ARMY 1996 Hard Faced Lightweight Composite Armor Materials for 45,110,015 1,500,000 Helmets Air Force 1992 Novel Aircraft Door/Panel Fastening System 30,379,559 0 NASA 1992 Liquid Crystal Polymers for CTE Matched PWBs 11,203,111 2,295,421 Air Force 1996 In Situ Biological Remediation of Hydrazine Spills Using 3,000,000 17,466,000 Gel Encapsulated Enzymes Air Force 1996 Integrated In-Line Wear and Lubricant Condition Sensor 1,868,000 12,126,105 Air Force 1994 Novel Infrared F/O Sensor for In Field Identification of 267,505 49,545,842 Hazardous Waste Solutions NAVY 1997 Electric Energy Absorption System for Aircraft Recovery 0 25,388,571 (EARS) Physical 234 70 NASA 1994 Embedded Distributed Moisture Sensor for $11,040,000 $1,732,000 Optics Nondestructive Inspection of Aircraft Lap Joints Corporation AF 1996 Visually Transparent Films $10,120,000 $962,000 NSF 1997 The Photochemical Generation of Planar Waveguides in $1,077,000 $10,500,000 Sol-Gel Glasses 9 continued

OCR for page 54
TABLE 2-1 Continued 90 Number of Commercialization Total Phase II Achievement Award Total Sales Investment Firm Name Awards Index (CAI) Agency Year Project Title ($) ($) Physical 205 30 Air Force 1985 Space Shuttle Plasma Interactions 20,711,803 1,409,755 Sciences, Air Force 1997 Airborne Hyperspectral Imager 14,570,673 999,525 Inc. Air Force 1997 Micromachined Sensor Array for Optical Meas. of 44,872 18,614,672 Surface Pressure Creare, Inc. 190 100 NASA 1986 Reliable Long-Lifetime Closed-Cycle Cryocooler for 1,212,529 39,417,294 Space NASA 1989 Three-Phase Inverter for High Speed Motor Drive 37,403 15,662,926 Radiation 140 95 HHS/NIH 1991 Intraoperative Imaging Probe for Delineation of Tumors 132,500,000 1,600,000 Monitoring HHS/NIH 1993 A New Portable XRF System for Lead Paint Analysis 91,343,990 4,339,000 Devices, Inc. NASA 1985 Portable Nuclear Cardiac Ejection Fraction Monitor 26,600,000 4,500,000 HHS/NIH 1992 Emboli Monitor to Reduce Surgical Neurological Deficits 25,500,000 470,000 HHS/NIH 1986 Sensitive Probe for Intra-Operative Bone Scanning 19,700,000 1,000,000 DoE 1984 Advanced Avalanche Photodiode for Positron Emission 588,000 21,870,000 Tomography SOURCE: U.S. Small Business Administration, Tech-Net Database.