Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 125
An Assessment of the SBIR Program at the National Science Foundation 7 Contributions to Knowledge 7.1 RESEARCH PROGRAM PERSPECTIVES By nature of their principal activity, Federal research programs share the goal of generating scientific and technical knowledge. They differ, however, in the importance they place on pushing the technology envelope, overcoming high-risk technical barriers, and performing high quality research. They also differ in the importance they attach to disseminating knowledge widely outside the innovator. In addition, they differ in the importance placed on developing performance measures for research quality, knowledge creation, and knowledge dissemination. 7.1.1 Attention to Research Quality and Knowledge Creation In its statement of program purpose, the Small Business Innovation Research (SBIR) program of the National Science Foundation (NSF) emphasizes that it intends “to increase the incentive and opportunity for small firms to undertake cutting-edge, high-risk, high quality research.” In its statement of goals, the NSF’s SBIR program states firstly that it aims to promote development of intellectual capital, making awards for research which builds on recent discoveries in basic sciences and engineering. As would be expected for consistency, the NSF’s SBIR program emphasizes intellectual merit as one of two merit review criteria for selecting proposals for funding. Thus, the NSF’s SBIR program in its statements of purpose, goals, and criteria—hence, intentions—consistently assigns importance to research quality and knowledge creation. In light of its stated purpose, goals, and criteria, and given that the peer-review selection process appears to have integrity, this study found nothing that would suggest the program is falling down on research quality or knowledge creation. At the same time, it
OCR for page 126
An Assessment of the SBIR Program at the National Science Foundation is recognized that research quality is difficult to measure and that the value of knowledge created is difficult to predict, takes time to be realized, and is highly variable among projects. 7.1.2 Attention to Knowledge Dissemination and Spillover Effects Technology development programs, in contrast to basic science programs, generally view knowledge gains as a means to a desired end, not the end point itself. Knowledge gains are seen as capacity building, as providing answers to questions impeding innovation, and in some cases as a means of broadening the scope of program benefits beyond those accruing directly to funding recipients and their customers. In the longer run, the social impacts of technology development programs come to reflect both the direct effects and the spillover effects of public R&D investment.1 Agencies that emphasize the value of research to accomplish their own mission-driven goals, such as the Department of Defense (DoD), tend to place less emphasis on measures of knowledge dissemination as an important program output. Programs that emphasize the generation of broad-based economic benefits, such as the Advanced Technology Program (ATP), emphasize both the value of their programs achieved through knowledge dissemination to others and the direct economic impact achieved by the innovator. The NSF’s SBIR program would appear at first glance to be close to the ATP in emphasizing knowledge dissemination and spillovers, in that its second major merit review criterion is “broader impacts.” However, the details of the NSF’s broader-impacts criterion appear to put less emphasis on broadening economic benefits of the proposed activity to society than ATP’s. More important, in practice the NSF’s SBIR program has largely ignored potential spillover benefits from knowledge dissemination. “Broader impacts” appears to be defined by the program largely as commercial results.2 Patenting was the only knowledge-related measure found in NSF evaluation studies of its SBIR program,3 and it appears that patent data were collected to signal commercial activity.4 No effective pro- 1 In contrast, basic research programs view knowledge creation as their primary goal. For these programs, most of the agency’s output/outcome/impact measures will likely focus on knowledge creation and dissemination. 2 NRC Program Manager Survey and discussions with program managers. 3 It should be acknowledged that the required interim, final, and post-grant annual commercialization reports (see Appendix F) each include a request for the reporting, if applicable, of publications, including the reporting of “scientific articles or papers appearing in scientific, technical, or professional journals … and any publication that will be published in a proceedings of a scientific society, a conference, or the like.” However, no program reporting or use of this information on knowledge creation and dissemination has thus far been discovered in either program management or performance metrics. And the requirement for postgrant annual reports has recently been dropped. 4 Patents preserve ownership rights to innovations that may be critical to being able to exploit them commercially.
OCR for page 127
An Assessment of the SBIR Program at the National Science Foundation gram reporting or use of publication information to indicate knowledge creation and dissemination was uncovered in either program management or performance metrics. In fact, it was concluded that the NSF SBIR program management sends mixed messages to grantees about the importance of knowledge dissemination, telling them at conferences to “forget about publishing and focus on commercializing.” At the same time, grantees appear in fact to be producing knowledge outputs, including publications, presentations, networking, and patents. There appear to be opportunities for the program to provide a more consistent message that may stimulate knowledge spillovers and to compile a more comprehensive set of indicators of knowledge creation and dissemination for evaluation purposes. 7.2 NRC STUDY FINDINGS ON KNOWLEDGE CREATION AND DISSEMINATION BY THE NSF’S SBIR PROGRAM The National Research Council’s study, through its surveys and case studies, investigated the outputs commonly used to assess knowledge creation and dissemination. The following sections present results for patents, copyrights, trademarks, and scientific publications; licensing agreements; sales of equity; partnering relationships with other companies and investors; and relationships of grantees with universities. The result of another type of analysis that would have been potentially useful, citation analysis, was not performed. In addition, survey results pertaining to the risk profile of projects funded are given. 7.2.1 Patents, Copyrights, Trademarks, and Scientific Publications Patents, copyrights, trademarks, and scientific publications are important indicators that knowledge has been both created and disseminated. The NRC Phase II Survey grants provided information on intellectual property, including patents, copyright, trademarks, and scientific publications. The responses for the 151 grants (or projects which they represent) reported the knowledge outputs shown in Table 7.2-1 as of the time of the survey. Note that the average number of patents granted (or received) per Phase II SBIR grants in the survey is 0.67. This is more than double the average number reported in the NRC Phase I Survey per Phase I SBIR grant not followed by a Phase II grant.5 7.2.2 Licensing Licensing agreements depend on the protection of the intellectual property. They are another indicator of the creation and dissemination of knowledge. 5 As noted in Section 5.2.1, the reported average number of patents per SBIR grant in the survey is much lower than the reported average number of patents per firm resulting at least in part to SBIR awards.
OCR for page 128
An Assessment of the SBIR Program at the National Science Foundation TABLE 7.2-1 Phase II Survey Results on Intellectual Property Type Number Applied for/ Submitted Average Number Received/ Published Average Patents 159 1.05 101 0.67 Copyrights 49 0.32 42 0.28 Trademarks 42 0.28 33 0.22 Scientific publications 266 1.76 250 1.66 SOURCE: NRC Phase II Survey. TABLE 7.2-2 Licensing Activities of Phase II Surveyed Grantees with U.S. and Foreign Companies and Investors Focus of Interactions Finalized Agreements (%) Ongoing Negotiations (%) Interactions with U.S. companies and investors 20 21 Interactions with foreign companies and investors 10 7 SOURCE: NRC Phase II Survey. Respondents reported licensing as the predominant activity they engaged in with other companies and investors both in the United States and abroad. Table 7.2-2 shows the frequency with which respondents said they had finalized or were negotiating licensing agreements to commercialize technologies resulting from the referenced grants. Respondents appear to form licensing agreements with foreign companies and investors approximately half as often as they form them with domestic companies and investors. The intense use of licensing signals the underlying importance of intellectual property protection to high-tech small businesses. Case study results also highlight the importance of intellectual protection and licensing activities as a major commercialization strategy of the small businesses. For example, consider the case study of Language Weaver. The company describes itself as “a core technology house based on licensing its software” directly to customers and indirectly through partners who license Language Weaver’s technology and incorporate it into their own products. Licensing activities tend to increase the diffusion of a technology’s effect, and as noted by Jaffe, licensing tends to increase spillover effects, particularly market spillovers.6 6 Adam Jaffe, Economic Analysis of Research Spillovers: Implications for the Advanced Technology Program, NIST GCR 97-708, pp. 42–44.
OCR for page 129
An Assessment of the SBIR Program at the National Science Foundation 7.2.3 Tracking Knowledge Dissemination by Citation Analysis Citation studies have been used extensively to show the transfer of knowledge from federally funded projects to others outside the walls of the funded projects, thereby demonstrating the wider potential impact of the federal funds. In the case of paper-to-patent citations, this is done by examining references to scientific and engineering papers on the front pages of U.S. patents. References are also made to previously issued patents. Both sets of patent and nonpatent references comprise the “prior art” of patents. Citation analysis has been used at various times by the U.S. Department of Energy, the National Institute of Standards and Technology, the Agricultural Research Service, the National Science Foundation,7 and other federal agencies to show the movement of knowledge from scientific research programs—where impacts are difficult to measure—to industrial technology—where impact measurement is more easily tracked.8 Patent citation trees are routinely used by ATP, for example, to show the dissemination of technical knowledge via patents from completed projects to other companies and other organizations.9 No evidence was found, however, of publication or patent citation analysis by the NSF’s SBIR program. Further, no evidence was found of the systematic collection by OII of the detailed publication and patent data from SBIR projects needed to support citation studies. With regard to publications, evidence was found of NSF SBIR program managers sending mixed signals to grantees regarding the importance, or lack thereof, of publications as a project output.10 Yet, as indicated by the results of the NRC Phase II Survey conducted for this study, patents and scientific publications are being produced by the NSF’s SBIR program. Hence, opportunities exist to encourage program participants to publish when it will not compromise their ability to commercialize. Both publication and patent citation analysis could be used to demonstrate and track knowledge dissemination from NSF SBIR projects to others. 7 The referenced use of citation analysis by the NSF lies outside the NSF’s SBIR program. The NSF supported extensive work by CHI Research, Inc., to develop and “clean” databases needed to perform publication citation analysis. 8 For an example of a citation study performed for a federal R&D program, see J. S. Perko and Francis Narin, CHI Research, Inc., “The Transfer of Public Science to Patented Technology: A Case Study in Agricultural Science,” Journal of Technology Transfer 22, no. 3 (1997):65–72. 9 Advanced Technology Program, Performance of 50 Completed ATP Projects: Status Report Number 2, NIST Special Publication 950-2, Gaithersburg, MD: National Institute of Standards and Technology, pp. 266–270. 10 This statement is based on observation of program manager oral responses to proposed questions at an SBIR conference regarding the importance of publications.
OCR for page 130
An Assessment of the SBIR Program at the National Science Foundation TABLE 7.2-3 Equity Sales of Phase II Grantees to U.S. and Foreign Companies and Investors Focus of Interactions Company Merger Sale of Technology Rights Partial Sale of Company Sale of Company Final (%) Ongoing (%) Final (%) Ongoing (%) Final (%) Ongoing (%) Final (%) Ongoing (%) Interactions with U.S. companies and investors 0 4 5 16 2 6 2 3 Interactions with foreign companies and investors 2 1 4 3 2 2 0 1 SOURCE: NRC Phase II Survey. 7.2.4 Equity Sales Sales of equity by NSF SBIR grantees to others represent transfers of knowledge. While licensing leaves ownership of the technology with the SBIR firm at the same time that it allows the licensee use of it, a sale of equity transfers ownership to a new owner. Among the NRC Phase II Survey respondents, activities to transfer equity centered on sales of technology rights to other domestic companies and investors rather than sales abroad. Table 7.2-3. shows that much of this activity was still in process at the time of the survey. Although at the time of the survey, none of the grantee companies had been sold to foreign companies or investors, there was indication that this activity was under way to some extent. Equity sales are sometimes an essential element in a commercialization strategy. In some cases, companies with the ability to commercialize are located outside the United States, and they may require ownership as a condition for commercializing.11 7.2.5 Partnerships of Small Firms with Other Companies and Investors Partnering with other organizations and people also accomplishes knowledge transfer. For small companies, the formation of partnerships with other compa- 11 For example, according to Brodd there are no volume lithium-ion battery manufacturers in the United States and this may influence commercialization strategies of small companies performing R&D in lithium-ion batteries. Ralph J. Brodd, Factors Affecting U.S. Production Decisions: Why Are There No Volume Lithium-Ion Battery Manufacturers in the United States? ATP Working Paper Series, no. 05-01, June 2005.
OCR for page 131
An Assessment of the SBIR Program at the National Science Foundation TABLE 7.2-4 Percentage of Phase II Surveyed Grantees Forming Partnerships with U.S. and Foreign Companies and Investors Partnering for: With U.S. Companies and Investors With Foreign Companies and Investors Finalized (%) Ongoing Negotiations (%) Finalized (%) Ongoing Negotiations (%) Licensing Agreement(s)a 20 21 10 7 R&D Agreement(s) 17 17 5 7 Marketing/Distribution Agreement(s) 16 12 8 2 Customer Alliance(s) 12 18 3 4 Manufacturing Agreement(s) 8 10 3 2 Joint Venture Agreement(s) 3 10 1 2 SOURCE: NRC Phase II Survey. NOTE: (a) Licensing agreements may or may not entail close partnering, whereas the other listed forms generally do require close alliances and partnering. nies is often an essential strategy for commercializing a technology. The larger companies they partner with often have manufacturing capacity, marketing know-how, and distribution paths in place. Grantees whose technology is far upstream of consumer goods may need to: partner with other companies for the additional research needed to integrate their technologies into larger systems; partner with Original Equipment Manufacturers (OEM) who purchase the grantees’ output as intermediate goods; and form alliances with customers to more effectively reach markets. The NRC Phase II Survey provided insight about the kinds of partnerships being formed by SBIR recipients. As shown in Table 7.2.4, partnerships for R&D, for marketing and distribution, with customers, and for manufacturing were found to be formed by these grantees. The case studies also illustrate how partnering as a business strategy transfers knowledge among partnering companies and their researchers. For example, NRT developed a metals processing technology in a strategic collaboration with another company, wTe Corporation, which has an automobile shredder division and specialized knowledge in this application. The joint objective of the companies was to develop an optoelectronic, ultra-high-speed process for sorting metals into pure metals and alloys and apply it to automobile shredding. 7.2.6 Small Firms and Universities Many of the companies also have a variety of relationships with universities by which knowledge is created and disseminated. Many funded projects have
OCR for page 132
An Assessment of the SBIR Program at the National Science Foundation involvement by university faculty, graduate students, and/or university-developed technologies. University faculty and students establish small businesses. Faculty members serve as proposal reviewers. Universities assist firms with proposal preparation and serve as consultants or subcontractors on projects. They also sometimes provide facilities and equipment to assist projects. Even high-school teachers and students sometimes work on projects through a supplemental program provided by the NSF to encourage such opportunities. Sixty-eight percent of companies responding to the NRC Firm Survey had at least one founder with an academic background. Of these, 40 percent of company founders had been employed by a college or university prior to founding the company. The NRC Phase II Survey showed that nearly half (47 percent) of the referenced projects involved some form of university involvement. The survey data show the prime mode of involvement to be faculty members or adjunct faculty members working on the referenced project in a role other than as principal investigator—as a consultant, for example. The next most frequent modes of involvement were those of graduate students working on the project and university or college facilities or equipment being used on the project. To a lesser extent, universities/colleges were subcontractors on the projects. In some instances, project technologies were originally developed in universities or colleges by one of the participants in the referenced projects. On occasion, the technologies for the referenced projects were licensed from a university or college. Table 7.2-5 indicates the extent to which each type of university involvement occurred in the sample Phase II projects. The NRC Phase II Survey results show that the NSF’s SBIR program helps move research concepts out of the university. Of the Phase II projects surveyed, 14 percent involved technology that was originally developed at a university by a project participant. Five percent of the technologies within the Phase II survey projects were licensed from a university. Some of the case study firms were found to have ongoing affiliations with universities. For example, NVE Corporation—formerly known as Nonvolatile Electronics and, now, simply as NVE—has maintained ongoing affiliations with the University of Minnesota, Iowa State University, and the University of Alabama. Established by a retired Honeywell executive, NVE developed substantial intellectual property in MRAM technology, nonvolatile computer memory. As NVE pursued MRAM development, it saw related potential applications, such as magnetic field sensors, and has developed several business lines in magnetic field sensors. 7.2.7 Risk Profile Survey results for funded projects suggest that they have high technical risk—an indicator that they are taking on challenging scientific problems. The NRC Phase I Survey reported technical difficulties as the chief reason for dis-
OCR for page 133
An Assessment of the SBIR Program at the National Science Foundation Box A Language Weaver: Validating University Research Like a newly born gazelle, Language Weaver found its legs early. It has rapidly developed a fully functional commercial software that uses a statistics-based translation technology from a prototype developed by the company’s founders—professors and researchers at the University of Southern California’s Information Sciences Institute. While the company founders still have their university posts, the company, under professional management, has grown to 35 employees, with several millions of dollars in revenues. The founders’ insight that they had a marketable product came just as the Internet bubble burst, making private venture capital difficult to secure. “When we were trying to start the company,” related Mr. Wong, a founder, “it was a little before 9/11 and no one cared about languages. There were no Senate hearings about languages. Then 9/11 happened, and at the time, we had already submitted a proposal to the NSF. But we didn’t hear back until November, and by then the NSF was able to bootstrap us to get us working quickly, moving code from the university to the company…. It was after the SBIR grant that everything happened. We started getting government interest as it became apparent that we had something interesting. But we would not have been positioned to move quickly to respond to the need if it hadn’t been for that first small amount of NSF funding and the confirmation of the technology.” The Language Weaver translation technology uses a code-breaking approach rather than the standard rule-based approach of existing translation software. The result is very high quality, near simultaneous translation of languages, including Arabic, Farsi, and Somali where, given the United States’ current national security requirements, the demand for translation has outstripped the supply of available translators. continuing Phase I projects. The survey of Phase I projects also suggested high technical risk. Of the Phase I projects that did not get a follow-on Phase II grant, a leading reason was technical barriers. An example of a technical barrier that one of the case study companies, MicroStrain, Inc., is successfully addressing is how to make microminiature, digital, wireless sensors that can autonomously and automatically collect and report data in a variety of applications. These applications range from protecting the Liberty Bell when it was moved, to determining the safety of a high-traffic bridge, to tracking damage in Navy aircraft. Another example of a technical barrier overcome is provided by the efforts of Language Weaver, Inc., which has boot-strapped a statistical machine translation technology with national security and economic potential out of a university into use on a fast-track basis. It is being used now to translate Arabic, Farsi, Chinese, and other languages into English, and English into other languages with reportedly far greater speed and accuracy than other techniques.
OCR for page 134
An Assessment of the SBIR Program at the National Science Foundation TABLE 7.2-5 Involvement by Universities and Colleges in Phase II Survey Projects Type of Relationship Between Referenced Project and Universities/Colleges Respondents Reporting the Relationship (%) Faculty members or adjunct faculty member worked on the project in a role other than principal investigator 37 Graduate students worked on the project 27 University/College facilities and/or equipment were used on the project 25 A university or college was a subcontractor on the project 17 The technology for this project was originally developed at a university or college by one of the participants in the referenced project 14 The technology for the project was licensed from a university or college 5 The principal investigator for the project was at the time of the project an adjunct faculty member 5 The principal investigator for the project was at the time of the project a faculty member 1 SOURCE: NRC Phase II Survey. Box B MicroStrain Inc.: Real-Time Monitoring and the Liberty Bell How do you move the Liberty Bell without turning its famous crack into an infamous one? That was the dilemma that National Park Service curators faced in 2003 when they moved the delicate American icon from its longtime home at the Bicentennial Pavilion in Philadelphia to a new display space at the Liberty Bell Center. Casting impurities make the Liberty Bell prone to cracking, but thanks to new microsensor technology developed by MicroStrain Inc., with SBIR awards from the NSF, the bell reached its destination without incident. The Liberty Bell was moved safely, thanks in part to a technology developed by MicoStrain that uses a network of wireless sensors to autonomously detect and report motion as small as one-hundredth the width of a human hair. Such networks of wireless sensors can be used to monitor in real time the structural health of bridges, roads, trains, dams, buildings, ground vehicles, aircraft, and ships, alerting those responsible to potential failures before they become disasters. Both private manufacturers and government customers—from local municipalities to the Navy—are drawing on this technology to reduce the costs of equipment maintenance and replacement while improving both the safety and the reliability of the nation’s infrastructure. Steve Arms started MicroStrain after completing his graduate degree in engineering. He found the NSF’s SBIR program to be particularly helpful in the early stages when he was building the company’s technological and commercial capacity. The NSF’s relatively more “open topics allowed the company to pursue the technical development that best fit its know-how,” he explained, adding that as a result, the company is better able to respond to the needs of its commercial and government customers.
OCR for page 135
An Assessment of the SBIR Program at the National Science Foundation 7.3 INDICATORS, NOT MEASURES OF BENEFIT Economic benefits from knowledge efforts become apparent when they are actually used by others to develop new and improved products, processes, and services. This means that collecting data about knowledge generation and dissemination activities provides only an indirect measure of impacts. Such data can, however, help us construct indicators of potential economic impacts. Examples of possible indicators include the number of patents per research dollar, characteristics of collaborative networks formed, and sales of commercialized goods and services. Trends in these and other indicators may indicate that developments are occurring along an indirect path—as would be expected for projects that are progressing toward the generation of broad impacts. It is apparent from the NRC Phase II Survey results that it would be possible to compile multiple indicators of knowledge generation and dissemination and early commercialization achievements from NSF SBIR projects, as well as to track them over time. Thus far, however, it appears that such indicators have been developed only partially and on an ad hoc basis. It also appears that more could be done to systematically compile and track indicators of knowledge generation and dissemination if desired.