Cover Image

HARDBACK
$73.75



View/Hide Left Panel

III

RECOMMENDATIONS AND FINDINGS



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 29
The Small Business Innovation Research Program: AN ASSESSMENT OF THE DEPARTMENT OF DEFENSE FAST TRACK INITIATIVE III RECOMMENDATIONS AND FINDINGS

OCR for page 29
The Small Business Innovation Research Program: AN ASSESSMENT OF THE DEPARTMENT OF DEFENSE FAST TRACK INITIATIVE This page in the original is blank.

OCR for page 29
The Small Business Innovation Research Program: AN ASSESSMENT OF THE DEPARTMENT OF DEFENSE FAST TRACK INITIATIVE Evaluation of the Fast Track Initiative for the Department of Defense SBIR Program Summary of Findings General Findings As noted in the Introduction, the Findings and Recommendations summarized below have important attributes and equally important limitations. A major attribute of the study is that the Board commissioned a significant series of studies to better understand the SBIR program. The case studies, the survey, and the empirical analyses were conducted independently, generally by established researchers who had not previously studied the program. Secondly, the research findings were publicly presented and discussed in an open forum and subsequently subjected to internal review procedures. While the papers and their conclusions are the responsibility of the investigators, the Findings and Recommendations are the responsibility of the Committee. As such, they reflect the results of the field research, the tacit knowledge acquired about the SBIR program during this phase of the project, and of course the substantial and diverse expertise of the Committee itself. However, as noted in the Introduction, and as the investigators themselves note, there are important caveats and limitations to this research. The first limitation concerns the relatively short time that the Fast Track Program has been in place. This necessarily limits our ability to assess the impact of the program. Secondly, although the case studies and surveys constitute what is clearly the largest independent assessment of the SBIR Fast Track Program at the Department of Defense, the study is nonetheless constrained by the limitations of the case-study approach and the size of the survey sample.1 Research results are 1   The relatively high rate of response to the survey is a positive feature of this research. Still, as with any research effort, there are a number of potential sources of bias. For example, the survey may be subject to a form of response bias if subjects have an incentive to make favorable comments about the program (Lerner raises this point in his paper, but also notes other factors which may, in fact, limit the willingness of entrepreneurs to emphasize the impact of the program). Some point out that a self-selection bias can be present insofar as the better firms may “self-select” for the Fast Track Program, although this criticism has a circular element. And as noted above, there is a sampling bias in that all of the Fast Track award recipients had not completed their projects before being asked to evaluate the Fast Track Program. The researchers and the Committee were sensitive to these concerns from the outset. The Committee attempted to indemnify against interpretive implications of these biases from the beginning by supporting a portfolio of methodologies to evaluate Fast Track as well as a portfolio of researchers with heterogeneous skills. While this effort does not eliminate all potential sources of bias, where possible the researchers controlled statistically for these types of biases. Because this multi-faceted approach to the research provides a useful means to search for common themes, the researchers felt that on balance the gains to be derived from this approach outweighed the methodological limitations. The statistical analysis prepared by Professors Audretsch, Link, and Scott specifically addresses these potential limitations of the data. Their empirical results are both robust and complementary to the case-based analyses by other researchers. See “Statistical Analysis of the National Academy of Sciences Survey of SBIR Awardees: Analyzing the Influence of the Fast Track Program ,” especially pp. 298-304.

OCR for page 29
The Small Business Innovation Research Program: AN ASSESSMENT OF THE DEPARTMENT OF DEFENSE FAST TRACK INITIATIVE necessarily preliminary. It is also important to keep in mind that this analysis of the SBIR—and the findings—are limited to the program of the Department of Defense. It is for this reason that the Committee has underscored (in Section IV below) the need for additional research. Additional assessment is required to confirm these results as technologies and firms continue to mature and as new firms increasingly take advantage of the SBIR Program. Moreover, regular assessment of the SBIR Program results and their comparison with the results of the Fast Track, both at Defense and other participating agencies, would provide a valuable means of understanding the operation of this important cooperative program, which currently operates at approximately $1.2 billion annually. As the resources allocated to federal research increase, under current law the SBIR program is destined to increase as well, underscoring the need for careful assessment of the program’s efficiency and effectiveness. Although this research represents a significant step in improving our understanding of the SBIR program, these findings should be appreciated for what they are; that is, a preliminary and limited effort by independent researchers and an informed Committee to understand the operation of an important government-industry partnership. Program Wide Findings The SBIR Program is contributing to the achievement of Department of Defense mission goals. Valuable innovative projects are being funded by the SBIR. It appears that a significant portion of these projects would not have been undertaken in the absence of the SBIR funding.2 2   See especially the papers in this volume of Robert Archibald and David Finifter, “Evaluation of the Department of Defense Small Business Innovation Research Program and the Fast Track Initiative” and the paper of Albert N, Link and John T. Scott “Estimates of the Social Returns to Small Business Innovation Research Projects.” This finding is consistent with the case studies conducted by Reid Cramer, David Audretsch, and Albert Link in their contributions to this volume.

OCR for page 29
The Small Business Innovation Research Program: AN ASSESSMENT OF THE DEPARTMENT OF DEFENSE FAST TRACK INITIATIVE The Fast Track Program increases the effectiveness of the SBIR Program at the Department of Defense by encouraging the commercialization of new technologies.3 Case studies from various regions in the United States found that the Department of Defense SBIR Program facilitates the development and utilization of human capital and technological knowledge.4 This holds true even for firms that exit the market. The experience and human capital generated by the program have economic value and can be applied by other firms. The receipt of an SBIR award can serve a certification function with regard to the firm’s technology and eventual market, thus encouraging private sector investment.5 Fast Track Findings: Survey and Case Study Results The Fast Track Program, whose principal objective is to reduce the funding gap between Phase I and Phase II of the SBIR award process, works. Among the companies surveyed, over half the Fast Track firms experienced no funding gap and the average funding gap was 2.4 months. By contrast, over 80 percent of companies not participating in the Fast Track Program confronted a funding gap between the end of Phase I funding and the beginning of Phase II funding; the average gap was 4.7 months.6 In particular, The average additional development funding received by Fast Track firms was five times greater than for a control group.7 This suggests, as intended, that the Fast Track Program is both attracting firms with a greater potential for commercial success and, with its promise of virtu 3   Peter Cahill, “Fast Track: Is It Speeding Commercialization of DOD’s SBIR?” pp. 64-65, and Figure 7 and Figure 8. 4   The paper of Audretsch, Weigand, and Weigand, “Does the Small Business Innovation Research Program Foster Entrepreneurial Behavior? Evidence from Indiana,” focuses particularly on how SBIR encourages the development of human capital. 5   See the papers in this volume by Reid Cramer “Participation in the SBIR Program in the Southwestern and Mountain States,” p. 152 and John Scott “Assessment of the Small Business Innovation Research Program in New England: Fast Track Compared with Non-Fast Track Projects,” pp. 116-118. Harvard Business School’s Josh Lerner initially identified this effect. See “Public Venture Capital”: The Small Business Innovation Research Program: Challenges and Opportunities, Washington, D.C.: National Academy Press, 1999, pp. 120-122. See also Kegler and Lerner, “Evaluating the SBIR: A Literature Review,” p. 321 in this volume. 6   Peter Cahill, “Fast Track: Is It Speeding Commercialization of DOD’s SBIR?” pp. 66-67. 7 Ibid., p. 44.

OCR for page 29
The Small Business Innovation Research Program: AN ASSESSMENT OF THE DEPARTMENT OF DEFENSE FAST TRACK INITIATIVE ally continual funding through Phase I and Phase II, is aiding in attracting capital to SBIR Fast Track firms. Actual and expected commercialization is greater for Fast Track firms. Fast Track firms forecast significantly greater sales than do non-Fast Track firms.8 Two-thirds of the technical representatives of agencies within the Department of Defense evaluated Fast Track projects as more effective than other projects in advancing their research goals, with only 11.8 percent rating Fast Track as less effective9 The Fast Track Program appears useful in assisting companies in attracting outside investment.10 However, individual companies have to weigh the benefits of Fast Track against its requirement for early outside investment.11 Neither the Fast Track Program, nor SBIR as a whole, appear to compensate for regional weaknesses in capital markets or university-industry interactions.12 The quality of research carried out under the Fast Track Program compares favorably with the research carried out under the regular SBIR award process.13 Fast Track attracts additional firms new to the program that are younger and smaller than non-Fast Track firms. Fifty-eight percent of Fast Track firms had no prior Phase II awards, compared with 30 percent with no prior Phase II awards for non-Fast Track firms.14 Attracting new community-oriented firms for the SBIR program is one of the objectives of the Fast Track approach. Case Studies Summaries Case studies were undertaken relying on detailed interviews with the founders, owners, and employees of over fifty firms. All of the case study firms 8   Ibid. 9   Archibald and Finifter, op. cit. p. 243. 10   Cahill, op. cit. pp. 68-70 and Reid Cramer, “Participation in the SBIR Program in the Southwestern and Mountain States,” p. 152. Both authors add that Fast Track tends to attract firms with a commercial orientation and this is a factor beyond the Fast Track tool that facilitates attracting outside investment. 11   See Cramer, op. cit. for a discussion of benefits and costs of Fast Track participation, pp. 149-150. 12   For example, a recent GAO report notes that the “SBA has found that the distribution of SBIR awards generally resembles the distribution of non-SBIR expenditures for research and development, venture capital investments, and academic research funds.” Evaluation of Small Business Innovation Research Can Be Strengthened, GAO (T-RCED-99-198), p. 4. See Kegler and Lerner, p. 313 in this volume. The authors underscore the role of clusters in encouraging geographic concentration of firms and therefore awards. 13   Archibald and Finifter, op. cit., p. 231. 14   Cahill, op. cit., p. 61.

OCR for page 29
The Small Business Innovation Research Program: AN ASSESSMENT OF THE DEPARTMENT OF DEFENSE FAST TRACK INITIATIVE had received SBIR assistance. They are dispersed across the United States and span a broad range of technologies, products, and industries. While some are new start-ups, others are established firms. These case studies examined the impact of the SBIR in a broad context. In particular, the case studies found that: The benefits of the SBIR extend beyond the impact on the individual recipient firm. Analysis of 44 projects shows that the social rate of return from SBIR-funded research to be 84 percent, well in excess of the 25 percent average expected rate of return from the projects without SBIR funding.15 This means that there are positive spillovers (i.e., net benefits to society) from SBIR-funded projects. The social rate of return, which incorporates this external positive impact, exceeds the private rate of return. The magnitude of the difference between social and private returns does not vary significantly, on average, between Fast Track and non-Fast Track projects.16 By definition, Fast Track firms attract additional financial capital. They also have higher rates of employment growth. Both suggest a greater likelihood of survival.17 Firms not participating in the Fast Track Program had nearly six times as many projects interrupt work due to a funding gap.18 There are three different types of firms involved with the SBIR. The first category is comprised of firms that are contractors, that is, firms whose mission is to conduct R&D on a contract basis for clients. The second category focuses on the development of a specific product or promising technology. The third category involves firms pursuing basic research outside of a university setting. The Fast Track Program seems to be the most effective with firms in the second category, which are involved in commercializing a new product.19 Specific Findings by Research Topic Impact of SBIR Awards on University Researchers There is evidence that SBIR induces scientists and engineers to change their career path and apply their technological knowledge to the development of a new firm.20 15   Link and Scott, “Estimates of the Social Returns to Small Business Innovation Research Projects,” p. 285. 16   Ibid., p. 275. 17   Audretsch, Link, and Scott, “Statistical Analysis of the SBIR Awardees,” pp. 293-305, especially p. 304 (Table 6). 18   Cahill, op. cit., p. 67. 19   Cramer, op. cit. pp. 143-145. 20   Audretsch, Weigand, and Weigand “Does the Small Business Innovation Research Program Foster Entrepreneurial Behavior? Evidence from Indiana” elaborate on this; see especially the case of Star Enterprises, p. 166.

OCR for page 29
The Small Business Innovation Research Program: AN ASSESSMENT OF THE DEPARTMENT OF DEFENSE FAST TRACK INITIATIVE The SBIR awards provide a source of funding for researchers to launch start-up firms that otherwise would not have had access to alternative sources of funding.21 SBIR awards can have a powerful demonstration effect. Scientists commercializing research results by starting companies induce colleagues to consider applications and the commercial potential of their research. The awards also encourage other scientists to submit their research to the award process for review.22 Biotechnology Awards Department of Defense SBIR awards to biotechnology firms have made a significant contribution to biotechnology R&D.23 Many Department of Defense funded bioscience projects have dual civilian and military uses.24 Department of Defense and National Institutes of Health bioscience funding complement each other.25 The SBIR award can have a positive impact on innovation even when the firm fails. The knowledge created sometimes has value for other firms and in some cases leads to successful commercial products. From a public viewpoint, this has positive economic value.26 Regional Findings Northeast27 Innovative activity in the small business sector would have been lower in the absence of SBIR. 21   Ibid., pp. 176-178. Ten of 12 firms interviewed said that they would not have founded the firms without SBIR funds. See also particularly the discussion of Genetic Models. See also John Scott, “SBIR in New England,” p. 122. 22   Ibid., p. 176; four of eight firms interviewed in Indiana said their SBIR experience encouraged colleagues to start a firm. 23   See Maryann Feldman, “The Role of DoD in Building Biotech Expertise,” in this volume. She reaches this conclusion based on her overall assessment of DoD’s impact on the biotechnology industry; she also reports that DoD spent $241 million (1997 dollars) on biotech research in the SBIR program from 1983-1997; p. 255. 24   Ibid., pp. 268-271 in the discussion of HT Medical Systems; moreover, nearly all of the DoD SBIR-funded biotech companies sell to commercial markets as well as the military. 25   Ibid. See especially the Martek case, pp. 266-268. 26   See Feldman, op. cit. 27   See John Scott’s paper in this volume.

OCR for page 29
The Small Business Innovation Research Program: AN ASSESSMENT OF THE DEPARTMENT OF DEFENSE FAST TRACK INITIATIVE Fast Track projects required less time for development subsequent to Phase II than did non-Fast Track Firms. This suggests that Fast Track firms are more oriented towards commercialization. Southeast28 Fast Track firms proceed to Phase II faster than non-Fast Track firms. Fast Track firms develop commercialization strategies sooner than non-Fast Track firms. Mountain states and southwest29 SBIR is perceived as being more useful to new firms oriented towards bringing new products to market than to other firms. Multiple Award Winners As noted above, firms approached the SBIR award process at different stages of development and with different objectives. Some firms are developing technology concepts; some firms see their vocation as contract research organizations; others actively seek to develop commercial products, either for public agencies or for the marketplace. Discussions with SBIR managers and awardees as well as field research suggest that in light of these different objectives: Investigator-led firms, limited in size and focused on a single concept, may seek several awards as they advance the research.30 For firms that carry out research as a core activity, success is necessarily measured in multiple contract awards. Even with many awards, there is nothing intrinsically wrong with a process that provides high-quality research at a lower cost than might otherwise be available to the Department of Defense. Inexpensive exploration of new technological approaches can be valuable, particularly if they limit expenditure on technological dead ends. For research-oriented firms, the key issue is the quality of the research. Those firms that seek to develop commercial products may, in an initial phase, seek multiple awards to rapidly develop a technology. Normally, this period is limited in time, before private investment becomes the principal source of funding.31 Responsibility for recognizing qualitative differences among multiple 28   See Albert N. Link’s paper in this volume. 29   See Reid Cramer’s paper in this volume. 30   See Cramer, op. cit. p. 151, who makes the point about the incremental nature of technical advance, which sometimes necessitates several awards and Scott, op. cit., in his discussion of Foster-Miller on p. 109. 31   See Feldman, op. cit., pp. 266-268, in her discussion of Martek as an example; see also Cramer, op. cit., pp. 146-147, who discusses several firms that realized commercial success after several awards.

OCR for page 29
The Small Business Innovation Research Program: AN ASSESSMENT OF THE DEPARTMENT OF DEFENSE FAST TRACK INITIATIVE award winners and ensuring congruency of awards with agency interests and goals lies with the managers of SBIR programs. Senior management, in consultation with SBIR managers, should develop meaningful guidance to ensure maximum returns to the program.32 As noted below, additional research with respect to multiple award winners might well reveal differences by industry and by frequency of awards. Established companies that successfully seek large numbers of awards and fledgling companies that seek few awards clearly have different objectives. It is the responsibility of the agency management to determine if the mandated goals of the SBIR program and the needs of the agency are being achieved. Recommendations for the Fast Track Program The Fast Track Program initiative at the Department of Defense has proved effective at increasing commercialization. As intended, the program has attracted more commercially-oriented firms to the program, has substantially reduced the funding gap between Phase I and Phase II of SBIR, and has served as a tool to attract third party investors to SBIR awardees. On this basis, it should be continued. This broadly positive assessment of Fast Track suggests that DoD should consider expanding the Fast Track Program within appropriate services, organizations, and agencies within the Department of Defense. This expansion should be undertaken with the recognition that the Fast Track process should not be applied to the entire award process. Other significant program goals, such as research and concept development, might otherwise be reduced or excluded. In light of its recent implementation, continued research on the impact of the Fast Track Program is required, as noted below. Cross-agency comparisons of the impact of Fast Track could prove useful for the continued refinement of the Program. Consideration should be given to allowing greater spread in the size of funding of awards, with larger awards reserved for commercially-oriented projects. The case studies and survey reveal that a variety of firm types participate in Fast Track, including start-up firms new to the program. This growing diversity suggests that greater flexibility in award size is warranted. 32   For a discussion of DoD measures to ensure quality research, see DOD’s Small Business Innovation Research Program, GAO (RCED-97-122), p. 3.

OCR for page 29
The Small Business Innovation Research Program: AN ASSESSMENT OF THE DEPARTMENT OF DEFENSE FAST TRACK INITIATIVE Recommendations for Future Research Regular assessment of SBIR program results and their comparison with Fast Track results would provide a valuable means of improving our understanding of program operations. While this study represents the first systematic assessment of the Fast Track Program carried out to date, additional assessment is required to confirm these results as technologies and firms continue to mature. The impact of SBIR awards in inducing researchers to undertake commercialization should be assessed through a larger study including more regions. Consideration should also be given to assessing this impact by technology. An assessment of the costs and benefits of better integrating SBIR awards in the development of “clusters” around universities and technology parks should be undertaken. An assessment of the costs and benefits of integrating particular SBIR awards to meet pressing agency needs through the development of focus programs, directed towards a particular technology or system, should be undertaken. Further analysis is required concerning the need and effectiveness of out-reach programs designed to publicize the opportunities of the SBIR program, to facilitate application to the program, and to reduce the application burden on first-time applicants. The Steering Committee* * For the Committee membership, see page vii.

OCR for page 29
The Small Business Innovation Research Program: AN ASSESSMENT OF THE DEPARTMENT OF DEFENSE FAST TRACK INITIATIVE This page in the original is blank.