National Academies Press: OpenBook
« Previous: Panel IV: Best Practice for Agency Programs: Program Executive Offices and Program Offices
Suggested Citation:"Panel V: Lessons Learned." National Research Council. 2007. SBIR and the Phase III Challenge of Commercialization: Report of a Symposium. Washington, DC: The National Academies Press. doi: 10.17226/11851.
×

Panel V
Lessons Learned

Moderator:Jacques S. GanslerUniversity of Maryland

Dr. Gansler said that the last question of the day was both important and difficult: To improve the program, where is the best place to focus attention: on the program office? the firm itself? the prime contractor? on the policy or Congressional levels? He said that the last panel had representation for each of the four areas that had been discussed throughout the day: Congress, the prime contractors, the services, and the small business community. He asked representatives of each for key points they would take away from the conference.

Richard CarrollInnovative Defense Strategies

Mr. Carroll said he had been involved in the SBIR program since its inception, and offered a quick overview of his experience. He started a small business in 1982, and competed on a number of SBIR programs. The company built up an SBIR inventory of Phase I and Phase II awards, and developed a large number of Phase III projects. He had sold the company to General Dynamics about 19 months previously, when it had significant Phase III awards, which the new owner was using to compete in the marketplace.

He said he was also closely involved with the reauthorization of the SBIR program in 1992. It was during that reauthorization that many Phase III issues were debated and the importance of Phase III became apparent, so the Congress added significantly to that phase of the program. He worked with the Small Business Administration, the DoD, and the Office of Management and Budget in helping to frame new Phase III initiatives for the new SBIR directive issued in 2000. When the directive was issued, it became clear that Phase III was a viable

Suggested Citation:"Panel V: Lessons Learned." National Research Council. 2007. SBIR and the Phase III Challenge of Commercialization: Report of a Symposium. Washington, DC: The National Academies Press. doi: 10.17226/11851.
×

pathway for SBIR businesses, resulting in many Phase III activities. The bill also called for the current study to evaluate the program, compare it with other science and technology initiatives, and recommend improvements.

How to Fund Phase III

He said that he was still working to improve Phase III and involved in debates on how to fund Phase III activities, which “remains one of the most significant issues in the program.” There was discussion about the Senate and the House Armed Services bill in 2005 that dealt with the subject. He brought to the conference several charts to illustrate the process of producing defense contracts, including purchasing and acquisition pathways, as well as other science and technology activities throughout the agency. The charts were extraordinarily complex, with many dozens of acronyms. And yet, he said, in order to make Phase III successful, the SBIR program had to be able to incorporate or feed into that labyrinthine process “in a way that makes sense to program managers.” He said that he believed it could be done, however, in a meaningful way so as to avoid the Valley of Death.

The Advent of Spiral Development

Dr. Gansler added that a major change now taking place in the acquisition process is “spiral development,”39 a method that is designed to constantly increase capability at lower and lower cost. This process brings an opportunity to take full advantage of the SBIR program, he said, by introducing in each subsequent cycle the changes that have been made in the previous cycles. The next challenge that has been discussed in this conference is to apply appropriate testing and evaluation to reduce risk to an acceptable level. This technique, which is being used for the Joint Strike Fighter, is an excellent focus for the SBIR program—to demonstrate techniques that lower risk and introduce them in subsequent cycles.

39

Spiral development was introduced in the mid-1980s as a way to reduce risk on large software projects. A spiral, or cyclical, approach is one that allows customers to evaluate early results and in-house engineers to identify potential trouble spots at an early stage. On subsequent “turns” of the spiral, early changes are incorporated, additional evaluation is done and changes made. The Department of Defense has adapted the technique as part of its evolutionary acquisition strategy to move newer technologies into large platforms, such as assault vehicles and computer systems, much more quickly. <http://www.washingtontechnology.com/news/18_5/cover-stories/20881-1.html>.

Suggested Citation:"Panel V: Lessons Learned." National Research Council. 2007. SBIR and the Phase III Challenge of Commercialization: Report of a Symposium. Washington, DC: The National Academies Press. doi: 10.17226/11851.
×

John P. WaszczakRaytheon Company

Mr. Waszczak said that his summary comment would begin with the cultural barriers that must be overcome to be successful in Phase III. A critical element for doing this is technology roadmaps that are integrated by government, prime contractors, sub-prime contractors, and small businesses. Another is program manager and Program Executive Officer pull, as well as a plan to transition from Phase II to Phase III before beginning Phase II. This means that a funding pool and insertion plan is needed for each projects.

He also extolled the matchmaking effort that John Williams and his team from the Navy and others had developed to match prime contractors with small businesses. This is expedited by the Navy’s Technology Assistance Program and similar gatherings by the Defense Department, NASA, and others.

The Advantage of Beginning with “Phase Zero”

To overcome small businesses’ difficulty in selling to prime contractors, he said that the key element is to start working with small businesses in “Phase Zero,” before Phase I even begins, so that they can better prepare for insertion as they go through Phases I and II, building trust with the customer and the prime, and trying to mitigate the perceived risks of working with a small firm.

He underlined the importance of incentives, and said that what truly motivates prime contractors—and small businesses—is return on investment. Because prime contractors, like every other organization, are resource constrained, they need to make sure the Phase III process is efficient if they are going to invest in it. They must also be sure that the customer and the end user—the warfighter— receive the benefits.

Finally, he said, the prime contractors needed good metrics to evaluate the Phase III transition and the performances of customers, prime contractors, and small businesses. This conference, he said, had produced many good suggestions for metrics and had strengthened the network for sharing them across the different services, agencies, and prime contractors.

Dr. Gansler interjected that the three large prime contractors at the conference had all indicated that within the past year, they had noticed a shift toward more pull from the prime contractors, which is one of the key elements in making this program a success.

John WilliamsU.S. Navy

Mr. Williams said he had always been a strong proponent of Phase III transitions, and of keeping the focus on Phase III throughout the SBIR program. Phase

Suggested Citation:"Panel V: Lessons Learned." National Research Council. 2007. SBIR and the Phase III Challenge of Commercialization: Report of a Symposium. Washington, DC: The National Academies Press. doi: 10.17226/11851.
×

III transitions involve multiple partners who must all address the same issues together, as they had been doing at this conference.

Noting that the Navy has a technology-pull approach, while the other services tend toward a technology-push approach, he suggested that it was always more challenging to transition technologies from a research or laboratory environment into an acquisition program. The Navy SBIR program has evolved to a point where the people who control the money for SBIR awards—the Program Executive Offices—are the people who plan the Phase III transition; and, their acquisition program offices directly influence the flow of Phase III dollars.

He recommended that the most important metric should be Phase III funding to the SBIR firm, and that those SBIR-managing Navy offices that generate significantly more Phase III follow-on funds should be rewarded. The Navy program, he said, basically gives back the 2.5 percent SBIR assessment to all assessed Program Executive Offices, but he feels that providing more funds to those Program Executive Offices that generate more Phase III metrics makes better sense. Since 2002, the Navy has awarded more Phase III dollars (non-SBIR funds) than Phase I and II SBIR dollars combined. In fact, for every SBIR dollar, Navy SBIR projects raise almost two dollars in additional funds. He said that this was one result that Congress likes to see, and would like to see more.

Using the Company Commercialization Report

He also said that SBIR firms should be able to compete for the “other” 97.5 percent of extramural RDT&E dollars, in addition to the 2.5 percent reserved for SBIR. Accordingly, the Navy uses a Company Commercialization Report (CCR) through which SBIR and non-SBIR revenues for small firms are identified. Even for small firms winning many tens of SBIR awards, he said, SBIR funds represented about 25 percent of their revenues—never more than 50 percent. That is a good sign, he said; if a company continued to win only SBIR awards, but not other awards, it’s likely that its technologies are not being transitioned. He recommended that the Academies use the CCR for measuring Phase III results of Phase II awards. The CCR captures not only Navy SBIR firms, but also many small firms working with the other agencies.

One challenge in building the Company Commercialization Report is that those in government who are evaluating the data are typically engineers who lack experience in business. He recommended that the NRC committee consult with leading economists and other experts to develop insight about how to gather the most relevant data and how best to evaluate them to measure effectiveness in commercialization.

Suggested Citation:"Panel V: Lessons Learned." National Research Council. 2007. SBIR and the Phase III Challenge of Commercialization: Report of a Symposium. Washington, DC: The National Academies Press. doi: 10.17226/11851.
×
Blending a New Technology with Other Technologies

He mentioned a business technical assistance program first initiated for the Department of Energy, and to some extent for NIH and NSF, by Dawnbreaker, Inc., that educates firms about how to commercialize technologies. In the Navy version of this program, he said, the objective is to show small firms actual requirements and reports describing the specific needs of the customer. This would include the details of other technologies it would have to work with, whether it needed to be licensed, and other information.

A paradox for the Navy SBIR program is that it has been so successful with their Transition Assistance Program program that it is beginning to drain its administration budget, given that it costs the Navy about $15,000 per small business participant. In one sense, this does not appear to be a serious expense—within 20 months of the program, participating firms average $3 million in Phase III awards, so the return on investment is almost a hundred-to-one. Nevertheless, he noted that it is still difficult to find additional dollars to put on a SBIR project to encourage commercialization because of Title 301 in the 1992 legislation, which describes what funds could be used for technical assistance. Because of the way this rule is written, he said, it is impossible to conclude whether the same percentage of funds allowed to support technical assistance in Phase I can be used in Phase II. He requested clarification of the rule, since the ability to use such funds for Phase II would provide up to $12 thousand per small firm to SBIR agencies to help SBIR companies.

DISCUSSION

James Rudd of the NSF said that his agency was involved in commercialization, and asked the DoD participants whether they would declare their SBIR program a success if a certain percentage of their R&D funding came from the SBIR program. Mr. McNamara responded that for an organization such as his, which spent between $5 and 6 billion a year, much of it on submarine procurement and costly ship maintenance, the SBIR money was a small amount, but when used, it has a great return on investment. Nonetheless, in parts of his organization, especially in electronics, where R&D is central, SBIR played a larger and more important role.

The Cost-effectiveness of Small Business

Another questioner asked how the services would rate the impact of SBIR companies on their overall innovation and business activities.

John Williams replied that the impact was large. He added that small businesses, especially those coming through SBIR, were about 30 percent cheaper than large businesses, and also cheaper than laboratories and field activities. In

Suggested Citation:"Panel V: Lessons Learned." National Research Council. 2007. SBIR and the Phase III Challenge of Commercialization: Report of a Symposium. Washington, DC: The National Academies Press. doi: 10.17226/11851.
×

addition, he noted that some small firms had proven more reliable than large firms. More broadly, he said, the process benefited by “adding people to the landscape” and producing more options for the services. Moreover, a lot more innovation had occurred at the big firms, he said, because they knew they were being measured against organizations other than their peers.

Performance Improvements at Lower Cost

Dr. Gansler added that a lot of SBIR activity occurred not at the weapons system level, but at the sub-system, component, or even the materials levels. For example, SBIR contributions often brought performance improvements at lower cost. This, he said, is “the way the real world, the commercial world, actually operates,” and that government should try to follow the lead of the computer industry, which produces new models at lower cost every two or three years.

A New Metric to Gauge Phase III Success …

Another participant said that dollar amounts and percentages were less valuable as metrics than the results of the Phase III contracts as requested every year by the Small Business Administration (SBA). “That,” he said, “is the easiest and cleanest metric” to indicate success, and it would be inexpensive. The SBA metrics would not provide a complete picture, he said, but they could show trends. In the same way, the achievements of the prime contractors could be measured. Success stories were not helpful, he suggested; every large program should have many success stories; and there were no clear metrics by which to measure innovation. The program should focus simply on measuring how much additional money was going to SBIR firms. “That was the intent of Congress,” he concluded, “and an easy way to measure it.”

… and a Caution about this Metric

Larry James of the Department of Energy urged caution about using the simple metric of Phase III dollars, which might drive the program further toward the development side. The “Innovation” in SBIR would eventually be lost. The program will gravitate toward low risk, incremental development projects. The program’s major benefit to the nation is that Small Business Concerns are willing and able to take on high-risk, innovative research that provides disruptive technological advances in the marketplace.

Suggested Citation:"Panel V: Lessons Learned." National Research Council. 2007. SBIR and the Phase III Challenge of Commercialization: Report of a Symposium. Washington, DC: The National Academies Press. doi: 10.17226/11851.
×
Page 140
Suggested Citation:"Panel V: Lessons Learned." National Research Council. 2007. SBIR and the Phase III Challenge of Commercialization: Report of a Symposium. Washington, DC: The National Academies Press. doi: 10.17226/11851.
×
Page 141
Suggested Citation:"Panel V: Lessons Learned." National Research Council. 2007. SBIR and the Phase III Challenge of Commercialization: Report of a Symposium. Washington, DC: The National Academies Press. doi: 10.17226/11851.
×
Page 142
Suggested Citation:"Panel V: Lessons Learned." National Research Council. 2007. SBIR and the Phase III Challenge of Commercialization: Report of a Symposium. Washington, DC: The National Academies Press. doi: 10.17226/11851.
×
Page 143
Suggested Citation:"Panel V: Lessons Learned." National Research Council. 2007. SBIR and the Phase III Challenge of Commercialization: Report of a Symposium. Washington, DC: The National Academies Press. doi: 10.17226/11851.
×
Page 144
Suggested Citation:"Panel V: Lessons Learned." National Research Council. 2007. SBIR and the Phase III Challenge of Commercialization: Report of a Symposium. Washington, DC: The National Academies Press. doi: 10.17226/11851.
×
Page 145
Next: Concluding Remarks »
SBIR and the Phase III Challenge of Commercialization: Report of a Symposium Get This Book
×
Buy Paperback | $61.00 Buy Ebook | $48.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

In response to a Congressional mandate, the National Research Council conducted a review of the Small Business Innovation Research Program (SBIR) at the five federal agencies with SBIR programs with budgets in excess of $100 million (DOD, NIH, NASA, DOE, and NSF). The project was designed to answer questions of program operation and effectiveness, including the quality of the research projects being conducted under the SBIR program, the commercialization of the research, and the program's contribution to accomplishing agency missions. This report summarizes the presentations at a symposium exploring the effectiveness of Phase III of the SBIR program (the commercialization phase), during which innovations funded by Phase II awards move from the laboratory into the marketplace. No SBIR funds support Phase III; instead, to commercialize their products, small businesses are expected to garner additional funds from private investors, the capital markets, or from the agency that made the initial award.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!