5
Program Management

5.1
INTRODUCTION

Each agency’s Small Business Innovation Research (SBIR) program operates under legislative constraints that set out program eligibility rules, the program’s three-phase structure, and (to a considerable extent) phase-specific funding limitations. Beyond these basic structural characteristics, programs have flexibility to decide how they select, manage, and track awards.

The similarities and differences across the agencies are discussed in detail in the following sections.

5.2
TOPIC GENERATION AND UTILIZATION

All applications for SBIR funding are made in response to published documents—termed solicitations—that describe what types of projects will be funded and how funding decisions will be made. Each agency publishes its solicitation separately. Subject areas eligible for SBIR funding are referred to as “topics” and “subtopics,” and each agency chooses its topics and subtopics in a different fashion with different goals in mind.

We have identified three distinct models of topic use at the SBIR agencies:

  1. Acquisition-oriented Topic Procedures. The Department of Defense (DoD) and the National Aeronautics and Space Administration (NASA) solicit for topics that channel applications toward areas that are high acquisition priorities for the awarding agency. For these agencies, the goal is to target R&D spending towards projects that will provide technolo-



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 170
5 Program Management 5.1 INTRODUCTION Each agency’s Small Business Innovation Research (SBIR) program operates under legislative constraints that set out program eligibility rules, the program’s three-phase structure, and (to a considerable extent) phase-specific funding limi- tations. Beyond these basic structural characteristics, programs have flexibility to decide how they select, manage, and track awards. The similarities and differences across the agencies are discussed in detail in the following sections. 5.2 TOPIC GENERATION AND UTILIZATION All applications for SBIR funding are made in response to published docu- ments—termed solicitations—that describe what types of projects will be funded and how funding decisions will be made. Each agency publishes its solicitation separately. Subject areas eligible for SBIR funding are referred to as “topics” and “subtopics,” and each agency chooses its topics and subtopics in a different fashion with different goals in mind. We have identified three distinct models of topic use at the SBIR agencies: (1) Acquisition-oriented Topic Procedures. The Department of Defense (DoD) and the National Aeronautics and Space Administration (NASA) solicit for topics that channel applications toward areas that are high ac- quisition priorities for the awarding agency. For these agencies, the goal is to target R&D spending towards projects that will provide technolo- 70

OCR for page 170
7 PROGRAM MANAGEMENT gies that can eventually be acquired by the agency for agency use. Appli- cations are not accepted outside topic areas defined in the solicitation. (2) Management-oriented Topic Procedures. At the National Science Foundation (NSF) and the Department of Energy (DoE), topics are used to serve the needs of agency management, not as a method of acquiring technology. NSF primarily focuses on ensuring that the research agen- das of the various NSF directorates (divisions) are closely served. This enhances agency staff buy-in for SBIR. At DoE, topics are also used as a screen to reduce the number of applications so that the agency’s lim- ited SBIR staff is able to manage the program effectively. As a result, neither agency will accept applications outside topic areas defined in the solicitation. (3) Guideline-oriented Topic Procedures. The National Institutes of Health (NIH) is the only agency that uses topics as guidelines and indicators, not hard delimiters. NIH issues annual topic descriptions, but emphatically notes that it primarily supports investigator-driven research. Topics are defined to show researchers areas of interest to the NIH Institutes and Centers (ICs), not to delimit what constitutes ac- ceptable applications. Applications on any topic or subject are therefore potentially acceptable. 5.2.1 Acquisition-oriented Approaches Acquisition-oriented approaches are designed to align long range research with the likely needs of specific agencies and programs. How well this works depends to a great degree on the specific topic development mechanisms used at each agency. 5.2.1.1 Topic Development In DoD, topics originate in Service Laboratories1 or in Program Acquisition Offices. Many awarding units within DoD do not have their own laboratories, and depend on the Service Laboratories for “in-house” expertise. They request top- ics from these experts, who thus become topic authors. These authors frequently become the technical monitors for the contracts that are awarded based on their topics. Since 1999, DoD has made a number of efforts to ensure that topics are closely aligned with the needs of acquisition programs. For example, each major acquisition program has a SBIR liaison officer, who works with SBIR program managers within DoD, and with the SBIR contractor community. Their job is to provide a mechanism through which contractors can communicate with end cus- 1The laboratories develop technologies to meet long-term agency needs.

OCR for page 170
7 AN ASSESSMENT OF THE SBIR PROGRAM tomers in acquisition programs. The liaison officers may author topics, or cause them to be authored. 5.2.1.2 Topic Review The topic review process at an acquisition-oriented agency is stringent and extensive. Alignment with mission and with anticipated technology needs is crucial. After each individual DoD component has completed its own internal topic review process, the results go through the department-wide DoD review process. This multistage topic approval process takes a considerable amount of time, sometimes more than a year. Intracomponent review varies in duration, somewhat correlated by size (smaller components with fewer proposed topics take less time). The Army topic review is a centralized online process that takes 4 months, while Air Force review is less centralized (but also online) and takes up to 15 months. 5.2.1.3 Broad versus Narrow Topics There is a tension between the need to write topics tightly to ensure that they align well with acquisition programs, and the desire to write them broadly enough to fund novel mechanisms for resolving agency problems. Components have tended to adopt one or other of these approaches. For example, the Missile Defense Agency (MDA) has until recently had only a small number of broad topics, evolving only gradually from year to year, which provides great flexibility to proposing firms. DoD is addressing this tension by calling for all topics to be written to allow a bidder “significant flexibility” in achieving the technical goals of the topic.2 For example, the Navy might need corrosion-resistant fastenings for use on ships. Instead of writing a topic that specifies the precise kinds of technology to be utilized, managers are now pushing for topics that simply state the problem and leave the technology itself up to applicant. 5.2.1.4 Quick Response Topics Particularly since 9/11, and the start of the conflict in Iraq, DoD has had acquisitions needs with short timeframes. Quick response topics offer a new way to short-cut the often lengthy topic review process. As part of the DoD’s SBIR FY-2004.2 solicitation, the Navy added three special SBIR Quick Response Topics. The rules for these topics were slightly different: They offered a three- to six-month Phase I award of up to $100,000 (with no option), with successful Phase I firms being invited to apply for a Phase 2This is a correction of the text in the prepublication version released on July 27, 2007.

OCR for page 170
7 PROGRAM MANAGEMENT II award of up to $1,000,000.3 The topics were released more rapidly because they did not go through the DoD level approval process. 5.2.1.5 Topic/Funding Allocation The allocation of topics effectively controls the allocation of money. There are two basic models for agency funding allocation via topics: • Percentage of the Gross. All SBIR programs struggle with the prob- lem of agency buy-in. Many agencies have traditionally viewed SBIR as a “small business tax” on their research budgets. This is even more important where the agency is itself the most important market for SBIR products and services, and acquisitions officers are the key customers. SBIR is funded through a set-aside of 2.5 percent of agency extramural research, and some senior agency staff have claimed that they could find better ways to spend that money. To address these concerns, some agen- cies and some components have designed their SBIR programs to tighten the link between the results of the SBIR research and the agency unit that has been “taxed” to fund the SBIR program. One way to do this is by allotting a specific number of topics to each funding unit, which then ef- fectively controls the content of the research funded by “its” SBIR money. For example, in the Army and Air Force, the number of topics is allocated to agency laboratories specifically on the basis of each lab’s overall R&D budget—and hence its contribution to the SBIR program funding pool. 4 • Technology-driven. Topic decisions can be more centralized or other- wise divorced from funding unit control. This allows more flexible admin- istration and more rapid response to urgent needs (via easier reallocation of funding between technical areas), but risks reduced unit buy-in. 5.2.1.6 DoD Pre-release Federal Acquisition Regulations prohibit any SBIR applicant contact with the relevant agency after a solicitation opens, other than through written ques- tions to the contracting officer, who must then make the question and the answer available to all prospective bidders. Under its pre-release program, DoD posts the entire projected solicitation on the Internet about two months before the solicitation is to open. Each topic in- cludes the name and contact information for the topic author. Firms may contact 3The Navy’s quick review approach is discussed further in National Research Council, An Assess- ment of the SBIR Program at the Department of Defense, Charles W. Wessner, ed., Washington, DC: The National Academies Press, 2009. 4The Air Force also allocates the number of funded Phase I projects.

OCR for page 170
7 AN ASSESSMENT OF THE SBIR PROGRAM the authors, and discuss the problem that the government wants to solve as well as their intended approach. Often, a short private discussion with the topic author can help a company avoid the cost of a proposal, or give the company a better idea of how to approach the SBIR opportunity. Small firms can learn of possible applications of their technology in acquisition programs, and determine which primes are involved in those programs. Firms new to SBIR often get procedural information, and are steered to the appropriate DoD Web sites for further information. 5.2.1.7 Company Influence on Topic Development In some sense, private company influence on topic development is both positive and sought after. Companies often understand cutting edge technologies better than the agencies, and may be able to suggest innovative lines of research. At DoD, involvement of prime contractors in topic development is one further way to help build linkages between the major acquirers of technology and the SBIR program. There have been some criticisms that topics are “wired”—that they are writ- ten for a specific company. There is no simple way to determine whether topics are wired, at DoD or elsewhere. Since about 40 percent of DoD Phase I winners are new to the program does, however, suggest that new or unconnected firms do have significant opportunities to participate in the SBIR program. It is also worth noting that while firms with long connections to DoD are best placed to help design topics friendly to their specific capacities, those long connections are in many cases, according to agency staff,5 the result of effective performance building to a strong track record in the course of previous SBIR awards. 5.2.2 Management-oriented Approaches to Topic Utilization Management-oriented approaches differ from the acquisition-driven pro- grams in that they do not evaluate applicants based on the government agency’s acquisitions needs. Instead, there are management priorities. The two agencies using this ap- proach differ substantially in the origins of these priorities. At NSF, SBIR funding decisions are driven by the technical decisions of the major funding directorates. Each directorate provides one annual list of topics, and their objective is explicitly to direct SBIR funding into targeted research areas. DoE’s priorities are described in the case study below. 5 See National Research Council, SBIR and the Phase III Challenge of Commercialization, Charles W. Wessner, ed., Washington, DC: The National Academies Press, 2007.

OCR for page 170
75 PROGRAM MANAGEMENT BOX 5-1 Topics at DoE Technical Areas For the 2003 competition, the 11 program areas at DoE were as follow: • Defense Nuclear Nonproliferation (3 Topics: 82 applications received); • Biological and Environmental Research (6: 140 apps.); • Environmental Management (2 Topics 10-11: 38 apps); • Nuclear Energy (1 Topic: 15 apps); • Basic Energy Sciences (8 Topics: 222 apps); • Energy Efficiency (6 Topics: 287 apps); • Fossil Energy (6 Topics: 169 applications received); • Fusion Energy Sciences (3 Topics: 66 applications received); • Nuclear Physics (4 Topics: 56 applications received); • A dvanced Scientific and Computing Research (2 Topics: 62 applications re- ceived); and • High Energy Physics (6 Topics: 87 applications received). SOURCE: Department of Energy. 5.2.2.1 Case Study: DoE DoE attempts to provide each of its technical program areas with a “return on investment” equal to its SBIR contribution. Each year, for each technical program area, the SBIR office attempts to ensure that the dollar amount of all awards (Phase I plus Phase II) awarded to proposals submitted to that program area’s technical topics is equal to that program area’s contribution to the SBIR set-aside. Although the dollar amount is the primary consideration, the number of awards each program area gets—in both Phase I and Phase II—is also roughly proportional to the funding it provides SBIR awards. To generate a “fair” return, each program area is provided with a number of technical topics that is proportional to its share of the total contribution. For example, if Fossil Energy (FE), provided 10 percent of the DoE’s set-aside, FE would receive 10 percent of the technical topics, and also would receive Phase I and Phase II awards (from proposals submitted to those topics) whose dollar value equaled 10 percent of the set-aside. About six months before the fall publication of the annual program solici- tation, the DoE SBIR Program Manager sends a “call for topics” memo to all Portfolio Managers responsible for technical program areas within DoE. This call also goes to all technical topic managers (TMs) from the prior year. The call includes the National Critical Technologies List,6 as well as guidelines for topic development. 6As conveyed to DoE from the U.S. Small Business Administration.

OCR for page 170
7 AN ASSESSMENT OF THE SBIR PROGRAM Topics and subtopics are submitted to or generated by the portfolio manag- ers, who determine which topics to forward to the SBIR Program Office. Portfolio managers develop a topic list that matches the amount of funding allocated to them, and this list is 100 percent funded by the SBIR Office, which performs no further review. The DoE SBIR Program Manager then edits the topics into a common for- mat, and may seek to narrow topics to limit the number of applications. 7 About 50 percent of DoE topics change from year to year. 5.2.3 Investigator-driven Approaches NIH is proud of its investigator-driven approach. The agency believes its approach is the best way to fund the best science, because it substitutes the deci- sions of applicants and peer reviewers for the views of agency or program staff. A small but growing percentage of NIH SBIR funding, however, goes to Program Announcements (PAs) and Requests for Applications (RFAs), which are Institute-driven requests for research proposals on specific subjects, covering the remainder. Discussions with agency staff suggest that this percentage may increase, perhaps substantially.8 5.2.3.1 Standard Procedure at NIH The NIH Web site and the documents guiding SBIR applications are both replete with statements that investigators are free to propose research on any technical subject, and that the topics published in the annual solicitation are only guides to the current research interests of the Institutes and Centers. Topics are developed by individual ICs for inclusion in the annual omnibus solicitation. Typically, the Program Coordinator’s office sends a request to the individual program managers (SBIR Point of Contact) at each IC. These PMs in turn meet with division directors, the locus of research assignments within the IC. Division directors review the most recent omnibus solicitation (with their staff), and suggest changes and new topics based on recent developments in the field, areas of particular interest to the IC, and agency-wide initiatives with im- plications within the IC. The revised topics are then resubmitted for publication by the Office of Extramural Research (OER), which does not appear to vet the topics further. 7 Interview with Bob Berger, former DoE SBIR Program Manager, March 18, 2005. 8 Interview with National Cancer Institute staff on March 6, 2007.

OCR for page 170
77 PROGRAM MANAGEMENT 5.2.3.2 Procedures for Program Announcements and Requests for Proposals (RFAs) PAs and RFAs occupy a position closer to management-oriented approaches. Essentially, they add a parallel structure within the NIH SBIR program through which the ICs and the agency as a whole can fund their own research priorities. PAs/RFAs are announcements of research funding areas that the IC expects to prioritize. PAs are areas of special interest that are still evaluated within the broad applicant pool for SBIR. RFAs are evaluated separately, and its applicants compete for a separate pool of SBIR funding set aside by the awarding IC. In both cases, the announcement is published by one or more ICs as a reflec- tion of top research priorities at the IC. It offers applicants a more directed path, aligned with agency priorities. NIH does still try to ensure that while PAs and RFAs define a particular problem, they are written broadly enough to encompass multiple technical solutions to the problem. PAs and RFAs appear to be an effort to develop a middle ground between topic-driven and investigator-driven research. Essentially, by layering PA/RFA announcements on top of the standard approach, NIH seeks to focus some re- sources on problems that it believes to be of especially pressing concern, while retaining the standard approach as well. 5.2.4 Topics: Conclusions There are some obvious advantages to an investigator-driven approach: • Applications are likely to respond to current concerns more rapidly (given the long lags involved in topics developed with other mechanisms). • Multiple deadlines are easier because an official solicitation does not have to be adjusted. • Better science may be attracted, as promising and truly innovative new technologies are not excluded. Acquisition- and management-driven models have their own advantages: • Agencies can much more easily impose their own research agendas. • Agencies can better ensure that SBIR research is aligned with eventual agency acquisition needs. • Narrow technical “windows” mean that the agency can limit the technolo- gies being addressed by a particular solicitation. • Narrower topic options mean fewer applications, limiting the amount staff and reviewer time required. The case for better science may be the most important point here. By defini- tion, if topics are designed to sharply limit/focus research on particular problems,

OCR for page 170
7 AN ASSESSMENT OF THE SBIR PROGRAM or in some cases on particular technologies, they are excluding other kinds of research. And, simple mathematics would suggest, if possible applicants are ex- cluded a priori, the average quality of successful applications is likely to fall. This suggests the following conclusions: • The acquisition-oriented model seems appropriate at the agencies where eventual technology acquisition by the agency is the primary objective. • The management-oriented model seems much less immediately defensible. It is difficult to see how the narrowing of potential topics at NSF best serves the interest of science: in conjunction with the annual funding cycle, this ensures that excellent science may have to wait for several years before its number comes up in the NSF topics lottery. It is even more difficult to defend the narrowing of selection areas on the basis of administrative convenience, as at DoE. Limited resources mean that all agencies limit the number of SBIR awards that are made, but it is hard to understand how doing so through a deliberate decision to reduce the scope of possible applications makes much sense. • The NIH investigator-driven model seems to be working effectively for NIH. • Hybrid models are worth further assessment. NIH appears to be evolving toward a hybrid model, as the share of awards going to RFAs and PAs is growing. This suggests possible options with which other agencies could experiment. For example, DoD components might reserve a small percent- age of funding for “open” topics that are essentially investigator-driven. It is worth noting that in other countries, hybrid models are used. For exam- ple, in Sweden, approximately half of Vinnova’s9 research funding is distributed to agency programs focused on the research needs of specific industries. The other half is allocated to proposals for research initiated by companies. 10 5.3 OUTREACH MECHANISMS AND OUTCOMES 5.3.1 Introduction As the program has become larger and more important to early-stage funding for high-tech companies, outreach activities by the programs have increasingly come to be complemented by outreach initiatives at the state level. At the different agencies, outreach programs seem to share three key objectives: 9VINNOVA is the Swedish national technology agency. 10VINNOVA Web site, accessed at .

OCR for page 170
79 PROGRAM MANAGEMENT • Attracting new applicants. • Ensuring geographical diversity in applications by attracting new appli- cants from under-represented regions. • Expanding opportunities for disadvantaged and woman-owned companies. These objectives are implicit, rather than explicit, in the outreach activities of all the programs. With the growth in electronic access, it is unclear whether greater outreach efforts could be cost-effective. 5.3.2 Outreach Mechanisms At the program level, all the agencies conduct outreach activities, though those efforts vary in kind and degree. Important types of outreach activities include: • National SBIR Conferences, which twice a year bring together repre- sentatives from all of the agencies, often at locations far from the big- gest R&D hubs (e.g., the Spring 2005 national conference took place in Omaha, Nebraska). • Agency-specific Conferences, usually held annually, which focus on awardees from specific agencies (e.g., the NIH conference, which is usu- ally held in Bethesda, Maryland). • Phase III Conferences, focused on bringing together awardees, potential partners, such as defense industry prime contractors, and potential inves- tors (e.g., the Navy Opportunity Forum, held annually in the Washington, DC area). • The SWIFT Bus Tour, which makes an annual swing through sev- eral “under-represented” states, with stops at numerous cities along the way. Participants usually include some or all of the agency program managers/directors/coordinators.11 • Web Sites and Listservs are maintained by all of the agencies. These Web sites contain application and support information. Some, such as the DoD’s site, are elaborate and comprehensive, and most allow users to sign up for a news listserv. • Agency Publications are used to spread information about the SBIR program. NASA, for example, has a variety of print and electronic publi- cations devoted to technology and research promotion. The other depart- ments use this publicity vehicle to a lesser degree. • Other, non-SBIR-focused Publications are sometimes used by the agen- 11Agencies have different titles for their primary SBIR program manager.

OCR for page 170
0 AN ASSESSMENT OF THE SBIR PROGRAM cies. For example, DoD publishes solicitation pre-publication announce- ments in its Commerce Daily. • Demographic-focused Outreach. Agencies acknowledge the need to ensure that woman- and minority-owned businesses know about SBIR and are attracted to apply. They have organized a series of conferences to encourage participation in the program by woman- and minority-owed business, often run in conjunction with other groups. For example, the DoD Southeastern Small Business Council held a conference for Woman- owned Small Businesses in Tampa, Florida, in June 1999. In general, these activities are the responsibility of the SBIR Program Co- ordinator at the agency, rather than other agency staff. They are also funded out of general agency funds, as the governing SBIR legislation does not permit pro- grams to use SBIR funds for purposes other than awards (and a limited amount of commercialization assistance. See below.). Generally, agencies do not provide much funding for outreach, though there are pronounced differences between the agencies. NIH, for example, has devel- oped a popular annual SBIR conference, whereas DoE has no funds for such an initiative. Within DoD, the Navy has provided significant extra operating funds for its SBIR program, which has therefore been able to take on outreach and other initiatives not open to the programs at other DoD components. 5.3.3 Outreach Outcomes 5.3.3.1 New Winners As SBIR programs have become larger, with longer track records and better online support, it is increasingly less likely that potential candidate companies will remain unaware of the program, or that such companies will have difficulty understanding how to apply. Still, new applicants remain an important component of the applicant pool (see Figure 5-1, for example). This may suggest that the current outreach activi- ties are having the intended effect. Much learning about new opportunities such as SBIR is conveyed directly. Another interpretation of the data is that outreach activities have become increasingly unnecessary in a world where a wide range of economic development institutions at the state and local level automatically point small high-tech companies toward SBIR as a funding source, and where public information about SBIR is fairly widespread and online. Anecdotal evidence from the SWIFT bus tour participants suggests that outreach activities do encourage new firms to enter the program. The program coordinator for NIH has noted that there is usually an upturn in applications from

OCR for page 170
 AN ASSESSMENT OF THE SBIR PROGRAM • All Phase II companies must attend the TAP orientation, which kicks off the program and provides detailed information about the program and its possible benefits. • The program itself is optional—a service provided by a contractor (Dawn- breaker, Inc.) on behalf of the Navy. The DoD components do not offer a comparable commercialization as- sistance program, although change is clearly under way in some areas. Defense Advanced Research Projects Agency (DARPA) has teamed with Larta Institute, a technology commercialization assistance organization, to develop and deliver a pilot program designed to assist DARPA’s SBIR Phase II awardees in the com- mercialization of their technologies. The program began in May 2006 and will comprise individual mentoring, coaching sessions, and a showcase event where SBIR awardees will present their technologies to the investment community, to potential strategic partners and licensees—an approach similar to the Navy’s. The program is available to current DARPA Defense Sciences Office SBIR Phase II Awardees.38 DARPA is also funding technical assistance through the Foundation for Enterprise Development (FED).39 In FY2005, following the NRC meeting on this subject, Congress estab- lished a new commercialization pilot program (CPP), which permits DoD to use 1 percent of SBIR funding (about $11.3 million in FY2006) to pilot various approaches that will further support commercialization of successful Phase II research.40 • The CPP was established in the 2006 Defense Bill and is still being de- fined by DoD. • The goal is to provide additional resources and emphasis on SBIR inser- tion into Acquisition Programs. • New incentives are to be established and CPP projects will receive ad- ditional assistance. Under guidance from the Under Secretary of Defense for Acquisition, Tech- nology & Logistics, USD(AT&L), the Army, Navy and Air Force began estab- lishing commercialization pilot programs in FY2006. Subsequent USD(AT&L) guidance in FY2007 encouraged all remaining DoD components to establish CPP activities.41 38 LartaInstitute Web site, accessed at . 39This is a correction of the text in the prepublication version released on July 27, 2007. 40 For a summary of the key points raised on this topic at the conference, see National Research Council, SBIR and the Phase III Challenge of Commercialization, op. cit. 41This is a correction of the text in the prepublication version released on July 27, 2007.

OCR for page 170
5 PROGRAM MANAGEMENT 5.8.2.2 NSF NSF’s program emphasizes commercialization and begins support for com- mercialization during Phase I. All grantees are required to attend a business development and training workshop during Phase I. Phase II grantees meet an- nually, where they are briefed on Phase IIB opportunities and requirements and are provided with workshops intended to assist with commercialization. The Phase IIB program provides additional funding predicated on the companies obtaining third-party funding. NSF’s SBIR program operates a “MatchMaker” program, which seeks to bring together SBIR recipient companies and potential third-party funders such as venture capitalists. Just recently, NSF has begun to support participation of its grantees in DoE’s SBIR Opportunity Forum, which brings grantees together with potential investors and partners. 5.8.2.3 NASA Like other agencies, NASA does not formally market its SBIR-derived technologies. However, because the agency understands the difficulties facing companies as they approach Phase III, it has developed a range of mechanisms through which to publicize the technologies developed using SBIR (and other NASA R&D activities). These include: • Spin-off (an annual publication). • Technology Innoation (quarterly). • Technology Briefs (monthly). • Success Stories publications. • TechFinder, NASA’s heavily used electronic portal and database. • Other materials such as a 2003 DVD portfolio of then-current NASA SBIR projects. NASA also sponsors nine small business incubators in different regions of the country. Their purpose is to provide assistance in creating new businesses based on NASA technology. 5.8.2.4 NIH NIH has managed a successful annual conference for some years, with grow- ing attendance and positive attendee feedback. NIH has also completed two pilot programs, one at the National Cancer In- stitute and one agency wide. The positive response to these programs has led the agency to roll out a larger Commercialization Assistance Program (CAP).

OCR for page 170
 AN ASSESSMENT OF THE SBIR PROGRAM Larta Institute (Larta) of Los Angeles, California42 was selected by a com- petitive process to be the contractor for this program.43 The Larta contract began in July 2004, and will run for five years. During the first three years, three cohorts of SBIR Phase II winners will receive assistance. Years four and five will cover follow-up work, as each cohort is tracked for 18 months after completion of the assistance effort. CAP Program Details. The assistance process for each group typically includes: • Provision of consultant time for business planning and development. • Business presentation training. • Development of presentation materials. • Participation in a public investment event organized by Larta. • 18 months for follow-up and tracking. It is still too soon to draw conclusions about the effectiveness of this effort, although NIH reports that of the 114 companies participating in the CAP program in 2004-2005, 22 have received investments totaling $22.4 million.44 5.8.2.5 DoE To aid Phase II awardees that seek to speed the commercialization of their SBIR technology, the DoE has sponsored a Commercialization Assistance Pro- gram (CAP). Formally launched in 1989 by Program Manager Samuel Barish with a $50,000 contribution from 11 DoE research departments, the CAP is now the oldest commercialization focused program among all federal SBIR programs.45 The CAP provides, on a voluntary basis, individual assistance in developing business plans and in preparing presentations to potential investment sponsors. The CAP is operated by a contractor, Dawnbreaker, Inc., a private firm based in Rochester, New York. In order to make the forum a more attractive event for the potential partners/investors (i.e., more business opportunities), the DoE SBIR program often partners with other agencies or with other DoE offices. For the FY2006 Forum DoE SBIR has agreed to partner with the National Science Foundation’s SBIR Program and with DoE’s Office of Industrial Technologies. 42 Larta Institute Web site, accessed at . 43 Larta was founded by Rohit Shukla who remains as its chief executive officer. It assists technol- ogy oriented companies by bringing together management, technologies, and capital to accelerate the transition of technologies to the marketplace. 44 Jo Anne Goodnight, NIH SBIR Coordinator, Personal Communication, March 16, 2007. 45 Interview with Sam Barish, DoE Director of Technology Research Division, May 6, 2005.

OCR for page 170
7 PROGRAM MANAGEMENT BOX 5-5 NASA’s Sponsored Business Incubators • Business Technology Development Center • Emerging Technology Center • Florida/NASA Business Incubation Center • Hampton Roads Technology Incubator • Lewis Incubator for Technology • Mississippi Enterprise for Technology • NASA Commercialization Center/California State Polytechnic University • University of Houston/NASA Technology Commercialization Incubator • NASA Illinois Commercialization Center 5.9 CONCLUSIONS The wide variety of organizational arrangements used to implement the SBIR program at the various agencies—and the differences even within programs at the two largest programs, at DoD and NIH—mean that generalized conclusions about the program should be treated with some caution,. Nonetheless, the NRC research permits us to draw the following conclusions, some which form the basis for recommendations. 5.9.1 Differentiation and Flexibility It is difficult to overstress the importance of diversity within the SBIR program. In fact, given the differences, it is a mistake to talk about “the” SBIR program at all: There are agency SBIR programs, and they are—and should be—different enough that they must be considered separately, as the NRC has done through the companion agency volumes. These differences—outlined above and emerging through a comparison of the agency volumes—stem of course partly from the differing agency missions and histories, and partly from the differing cultures within them. It is not our mandate to disentangle the sources of these complexities. The point here is to assert that the diversity of the program—which reflects the flexibility permitted within the SBIR structure, one largely enhanced by SBA interpretations of its guidelines—is fundamental to the program’s success. The fact is that DoD pro- grams are managed to generate technologies for use within DoD. It would make little sense to manage them like NIH programs, which aim to provide technolo- gies that will eventually find their way largely to private sector providers in health care. The different objectives impose different needs and constraints and hence different strategies.

OCR for page 170
 AN ASSESSMENT OF THE SBIR PROGRAM The inescapable conclusion from this is that efforts to standardize the opera- tion of different aspects of the program are likely to prove counterproductive. A further corollary is that managers need substantial flexibility to adapt the program to the context that they confront within their own agency and to their own technical domains. Nothing in the following points should therefore be taken as asserting the need for a single one-size-fits-all approach. We believe that every agency will find value in considering the suggestions and recommendations in this report—but also that each agency must judge those recommendations in light of its own needs, objectives, and programs. 5.9.2 Fairness, Openness, and Selection Procedures The evidence suggests that the programs are by and large operated in a fair and open manner. Opportunities for funding are widely publicized, and the pro- cess for applying and for selection is largely transparent at all agencies. It is also true that at some agencies—in particular NIH, where peer reviewers from outside the agency have a predominant role in selection—concerns about possible conflict of interest were widely reported by interviewees. Conflicts of interest may occur in some areas, but the different selection procedures used by the agencies require that this issue be addressed at the agency level, as it is recom- mended, for example, in the NIH volume. It is by no means a general problem. While open, the agency selection procedures do differ, yet none of the agencies has made any effort to assess the success of their particular selection methodologies, or to determine ways of piloting improvements in them. Linking outcomes to selection has been done only at DoD, through the use of the Com- mercialization Index, but efforts to integrate this into selection procedures have not been systematically analyzed. 5.9.3 Topics In agencies where topics define the proposal boundaries, the topic develop- ment process is also an important part of outsider perceptions of the program, and here there is less transparency: in none of the agencies is the process of topic development spelled out clearly on the Web site, which at least at DoD results in perceptions that some topics are designed—wired—for specific companies. As noted above, there is no available evidence to support this claim, not least because some companies have developed expertise, giving them a competitive advantage. Increased transparency in this area might enhance perceptions of the program’s fairness more generally. At DoD in particular, the agency began making efforts to reduce cycle times and to increase the relevancy of topics, starting in 1997 if not before. These ef- forts have had an important impact on relevancy, where a growing number of topics—now more than 50 percent DoD-wide—are sponsored by acquisition of-

OCR for page 170
9 PROGRAM MANAGEMENT fices. At NASA, DoE, and NSF, topics are all drawn from technical experts with area responsibilities in the agencies. However, two other topic issues have emerged: • Cycle Time. The DoD process does reduce duplication and enhance relevance, but at the cost of timeliness. Recent initiates to develop quick response topics are therefore a promising way of balancing the needs for effectiveness and speed. • Breadth. At least two factors seem to drive agencies toward narrow defi- nitions of topics, including definitions that mandate the technical solution to be used as well as the problem to be solved. At DoD, the effort to gain interest from acquisition and other non-research communities could lead to the use of SBIR as a form of off-budget contract research. At NSF and DoE, topics have been narrowed in an effort to reduce the flow of applications to what the agency staff believes to be manageable propor- tions. Neither of these motives seems justified, and we would suggest that agencies seek to keep their topics as broadly defined as possible, at least in terms of the technical solutions that might be acceptable. 5.9.4 Cycle Time In interviews with awardees and other observers of the program, concerns about cycle time surfaced regularly, at every agency. The fact is that it takes time to develop topics, publish a solicitation, assess applications, make awards, and in the end sign a contract. And that is just for Phase I. Despite efforts at some agencies—including DoD, and in other ways DoE— the cycle time issue has not been fully addressed. Agencies have not in general fully committed to a “gap reduction” model where every effort is made to squeeze days out of the award cycle. For small businesses, especially those with few awards and few other resources, cycle time is a huge disincentive to applying for SBIR, and constitutes a significant problem for companies in the program. Every aspect of cycle time—from lags between Phases to the number of annual solicitations, the breadth of topics and the permeability of topic borders, to contracting procedures—should be monitored, assessed, and, where possible, improved. Currently, agencies are in general insufficiently focused on this issue, as detailed in the agency volumes. The additional funding recommended below should, in part, be used to ad- dress this problem. 5.9.5 Award Size The fact that award size has not been officially increased since the 1992 reauthorization, and has therefore not kept pace with inflation, in itself raises

OCR for page 170
0 AN ASSESSMENT OF THE SBIR PROGRAM the question of whether the size of awards should be increased. But this is not a simple question. Related questions include: by what amount? One-time only or possibly tied to inflation? Both Phases? At all agencies? The primary justification for raising the nominal limits is that the cost of research has increased with inflation, and hence that these limits do not buy the agencies the same amount of research results that the Congress intended when this guidance on award size was first introduced. Agency staff have offered a range of additional justifications for larger awards. At NIH, which has been most active in experimenting with larger awards, justifications include the need to focus on the highest quality research, the like- lihood that more funding will lead to more commercial success, the impact of inflation, the need to support companies through regulatory hurdles, and the pos- sibility that higher funding levels will expand the applicant pool by attracting, in particular, high-quality applicants who currently believe SBIR is too small to justify the effort to apply. Finally, and not insignificantly, there is a relatively higher overhead cost of administering more, smaller awards. The latter is tied to the minimal administrative funding for SBIR, discussed below. The other NIH-specific points are discussed in the NIH volume. Awardees in interviews have also favored larger awards—until they are asked to make an explicit trade-off between the size of awards and the number of awards. At that point, awardees often become less supportive of larger awards. There are also arguments against larger (and fewer) awards. Because it is ex- tremely difficult to predict which awards will generate large returns (commercial or otherwise), it may be wise to spread the awards as widely as possible. SBIR awards also play a critical role (described in Chapter 4: SBIR Program Outputs) in supporting the transition of research from the academy to the market place. This kind of motivation may not require more than the existing level of support. There is as yet also no evidence to support the assertion that larger awards gener- ate larger returns, although further analysis at NIH might test that connection. If we conclude—as we do—that the SBIR programs at the agencies do work as intended by Congress, and do generate significant benefits, we should recom- mend change only with caution. It therefore seems that while there is a case to increase award size, there are risks involved, and it would be prudent for agencies taking this step to increase the awards incrementally over, perhaps, three years to avoid a sharp contraction of the program and to allow hope for increases in R&D funding to mitigate the impact on applicant success rates of increasing award sizes. 5.9.6 Multiple-Award Winners Multiple-award winners do not appear to constitute a problem for the SBIR program at any agency. At all agencies except DoD, only a limited number of

OCR for page 170
 PROGRAM MANAGEMENT companies win a sufficiently large number of awards to meet even the loosest definition of a “mill.” Even at DoD, we find arguments aimed at limiting a company’s participation in SBIR to be unconvincing, for a number of reasons: (1) Successful Commercialization. Aggregate data from the DoD com- mercialization database indicates that the basic charge against “mills,” i.e., no commercialization, is simply incorrect. Companies winning the most awards are on average more successful commercializers than those winning fewer awards. • While data from this source are not comprehensive, they do cover the vast majority of MAWs—and the data indicate that on average, firms with the largest number of awards commercialize as much or more than all other groups of awardees; that in the aggregate, there is no MAW problem of companies living off SBIR awards. (2) For some multiple winners, at least, even though they continue to win a considerable number of awards, the contribution of SBIR to overall revenues has declined.46 (3) Case studies show that some of the most prolific award winners have successfully commercialized, and have also in other ways met the needs of sponsoring agencies. (4) Graduation. Some of the biggest Phase II winners have graduated from the program either by growing beyond the 500-employee limit or by being acquired—in the case of Foster-Miller, for example, by a foreign- owned firm. Legislating to solve a problem with companies that are in any event no longer eligible seems inappropriate. (5) Contract Research. This can be valuable in and of itself. Agency staff indicate that SBIR fills multiple needs, many of which do not show up in sales data. For example, efficient probes of the technological frontier, conducted on time, on budget, to effectively test technical hypotheses, may save extensive time and resources later, according to agency staff. (6) Spin-offs Some MAWs spin off companies—like Optical Sciences, Creare, and Luna. Creating new firms can be a valuable contribution. (7) Valuable Outputs. Some MAWs have provided the highly efficient and flexible capabilities needed to solve pressing problems rapidly. (8) Compared to What? Agency programs do not impose limits. It is hard to see why small businesses should be subjected to limits on the number of awards annually when successful universities and prime contractors are not subject to such limits. 46At Radiation Monitoring, for example, SBIR has fallen steadily and is now only 16 percent of total firm revenues.

OCR for page 170
 AN ASSESSMENT OF THE SBIR PROGRAM All these points suggest that while there have been companies that depend on SBIR as their primary source of revenue for a considerable period to time, and there are some who fail to develop commercial results, the evidence strongly supports the conclusion that there is no multiple winner problem. Moreover, those who advocate a limit on the annual number of awards to a given company should explain how this limit is to be addressed across multiple agencies, and why technologies that may be important and unique to a given company should be excluded on this basis. Given that SBIR awards meet multiple agency needs and multiple congres- sional objectives, it is difficult to see how the program might be enhanced by the imposition of an arbitrary limit on the number of applications per year, as is currently the case at NSF. However, if agencies continue to see issues in this area, they should consider adopting some version of the DoD “enhanced surveillance” model, in which multiple winners are subject to enhanced scrutiny in the context of the award process. 5.9.7 Information Flows The shift toward Web-based information delivery has occurred unevenly at the different agencies. DoD has perhaps moved farthest; along with NASA, it was the first agency to require electronic submission of applications, and the online support for applicants is strong. It is, moreover, well integrated with non- electronic information sources, with two innovations in particular being well- received by awardees: • The Pre-release Period, during which topics are released on the Web along with contact information for the topic authors. This enables poten- tial applicants to directly determine how well their proposed research will fit with the agency’s needs, and provides opportunities for tuning appli- cations, so they are a better fit. This innovation also connects applicants specifically to the technical officers running a particular topic, who will, in the end, also make finding decisions. • The Help Desk, which is staffed by contractors and is designed to divert non-technical questions (for example about contracts and contracting) to staff with relevant experience in the SBIR program, who may well know these materials better than topic authors for example. Program managers at NIH and NASA have complained in interviews that SBIR applicants require much more help than academic applicants for other kinds of awards. Some of that burden might be alleviated with a better resourced help desk function.47 47This is a correction of the text in the prepublication version released on July 27, 2007.

OCR for page 170
 PROGRAM MANAGEMENT Overall, the growing size of the program, the clear interest exhibited by state economic development staff, and the increasingly positive view of SBIR at many universities, suggest that knowledge about the program is increasingly being dif- fused to potential applicants. The rise of the Internet—and the high quality Web sites developed by the agencies—mean that general interest can be translated into specifics quickly and inexpensively for both applicants and agencies. This con- clusion is buttressed by the continuing flow of new companies into the program; these new companies account for more than 30 percent of all Phase I awards at every agency every year. Thus it appears that the general outreach function historically fulfilled by the SBIR Agency Coordinators/Program Managers may now be changing toward a more nuanced and targeted role, focused on enhancing opportunities for under- served groups and underserved states, or on specific aspects of the SBIR program (e.g., the July 2005 DoD Phase III meeting in San Diego)—while relying on the Web and other mechanisms to meet the general demand for information. This seems entirely appropriate. 5.9.8 Commercialization Support While some agencies have been working to support the commercialization activities of their companies for a number of years, this has clearly become a higher priority at most agencies in the recent past. Congress has always permit- ted agencies to spend a small amount per award ($4,000) on commercialization support, and most agencies have done so. Commercialization support appears likely to have a significant pay-off for the agencies, partly because many SBIR firms have limited commercial experi- ence. They are often founded by scientists and engineers who are focused on the technology, and interviews with awardees, agency staff, and commercialization contractors all indicate that the business side of commercial activities is often where companies experience the most difficulty. Important recent initiatives include the extensive set of services provided at Navy through the TAP, and the NIH commitment to roll out the CAP program. These add to the long-running program at DoE. It is important to understand that the character of commercialization differs quite fundamentally between DoD, NASA, and the remaining nonprocurement agencies respectively (DoE is partly a procurement agency, but for our purposes here, it purchases such a small amount of SBIR outputs for internal consumption, that it is best grouped with the nonprocurement agencies). At DoD, where the agency provides a substantial market if companies can find a connection to the acquisitions programs, the critical focus of commer- cialization is on bridging the gaps to adequate Technology Readiness Levels and on finding ways to align companies with potential downstream acquisition programs.

OCR for page 170
 AN ASSESSMENT OF THE SBIR PROGRAM At NASA, where the market within NASA for technologies, though impor- tant, may not be large enough to sustain long term development and profitability, the focus is increasingly on the spinout of technologies into the private sector. At NIH, NSF, and DoE, commercialization means finding markets in the private sector. These differences mean that while in general all commercialization as- sistance programs provide help in formulating business plans, in developing strategic business objectives, and in tuning pitches for more funding, there are important differences. In particular, DoD, which accounts for about half of the entire SBIR program, has commercialization programs that are largely (though not exclusively) focused on markets internal to DoD and on the particularly complex process of finding a way into the acquisition stream. This requires dif- ferent training, different analysis, and different benchmarks than do other com- mercialization programs. All of these suggest that it is important to find appropriate benchmarks against which to measure success.