Panel V
Lessons Learned

Moderator:Jacques S. GanslerUniversity of Maryland

Dr. Gansler said that the last question of the day was both important and difficult: To improve the program, where is the best place to focus attention: on the program office? the firm itself? the prime contractor? on the policy or Congressional levels? He said that the last panel had representation for each of the four areas that had been discussed throughout the day: Congress, the prime contractors, the services, and the small business community. He asked representatives of each for key points they would take away from the conference.

Richard CarrollInnovative Defense Strategies

Mr. Carroll said he had been involved in the SBIR program since its inception, and offered a quick overview of his experience. He started a small business in 1982, and competed on a number of SBIR programs. The company built up an SBIR inventory of Phase I and Phase II awards, and developed a large number of Phase III projects. He had sold the company to General Dynamics about 19 months previously, when it had significant Phase III awards, which the new owner was using to compete in the marketplace.

He said he was also closely involved with the reauthorization of the SBIR program in 1992. It was during that reauthorization that many Phase III issues were debated and the importance of Phase III became apparent, so the Congress added significantly to that phase of the program. He worked with the Small Business Administration, the DoD, and the Office of Management and Budget in helping to frame new Phase III initiatives for the new SBIR directive issued in 2000. When the directive was issued, it became clear that Phase III was a viable



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 140
SBIR and the Phase III Challenge of Commercialization: Report of a Symposium Panel V Lessons Learned Moderator:Jacques S. GanslerUniversity of Maryland Dr. Gansler said that the last question of the day was both important and difficult: To improve the program, where is the best place to focus attention: on the program office? the firm itself? the prime contractor? on the policy or Congressional levels? He said that the last panel had representation for each of the four areas that had been discussed throughout the day: Congress, the prime contractors, the services, and the small business community. He asked representatives of each for key points they would take away from the conference. Richard CarrollInnovative Defense Strategies Mr. Carroll said he had been involved in the SBIR program since its inception, and offered a quick overview of his experience. He started a small business in 1982, and competed on a number of SBIR programs. The company built up an SBIR inventory of Phase I and Phase II awards, and developed a large number of Phase III projects. He had sold the company to General Dynamics about 19 months previously, when it had significant Phase III awards, which the new owner was using to compete in the marketplace. He said he was also closely involved with the reauthorization of the SBIR program in 1992. It was during that reauthorization that many Phase III issues were debated and the importance of Phase III became apparent, so the Congress added significantly to that phase of the program. He worked with the Small Business Administration, the DoD, and the Office of Management and Budget in helping to frame new Phase III initiatives for the new SBIR directive issued in 2000. When the directive was issued, it became clear that Phase III was a viable

OCR for page 140
SBIR and the Phase III Challenge of Commercialization: Report of a Symposium pathway for SBIR businesses, resulting in many Phase III activities. The bill also called for the current study to evaluate the program, compare it with other science and technology initiatives, and recommend improvements. How to Fund Phase III He said that he was still working to improve Phase III and involved in debates on how to fund Phase III activities, which “remains one of the most significant issues in the program.” There was discussion about the Senate and the House Armed Services bill in 2005 that dealt with the subject. He brought to the conference several charts to illustrate the process of producing defense contracts, including purchasing and acquisition pathways, as well as other science and technology activities throughout the agency. The charts were extraordinarily complex, with many dozens of acronyms. And yet, he said, in order to make Phase III successful, the SBIR program had to be able to incorporate or feed into that labyrinthine process “in a way that makes sense to program managers.” He said that he believed it could be done, however, in a meaningful way so as to avoid the Valley of Death. The Advent of Spiral Development Dr. Gansler added that a major change now taking place in the acquisition process is “spiral development,”39 a method that is designed to constantly increase capability at lower and lower cost. This process brings an opportunity to take full advantage of the SBIR program, he said, by introducing in each subsequent cycle the changes that have been made in the previous cycles. The next challenge that has been discussed in this conference is to apply appropriate testing and evaluation to reduce risk to an acceptable level. This technique, which is being used for the Joint Strike Fighter, is an excellent focus for the SBIR program—to demonstrate techniques that lower risk and introduce them in subsequent cycles. 39 Spiral development was introduced in the mid-1980s as a way to reduce risk on large software projects. A spiral, or cyclical, approach is one that allows customers to evaluate early results and in-house engineers to identify potential trouble spots at an early stage. On subsequent “turns” of the spiral, early changes are incorporated, additional evaluation is done and changes made. The Department of Defense has adapted the technique as part of its evolutionary acquisition strategy to move newer technologies into large platforms, such as assault vehicles and computer systems, much more quickly. <http://www.washingtontechnology.com/news/18_5/cover-stories/20881-1.html>.

OCR for page 140
SBIR and the Phase III Challenge of Commercialization: Report of a Symposium John P. WaszczakRaytheon Company Mr. Waszczak said that his summary comment would begin with the cultural barriers that must be overcome to be successful in Phase III. A critical element for doing this is technology roadmaps that are integrated by government, prime contractors, sub-prime contractors, and small businesses. Another is program manager and Program Executive Officer pull, as well as a plan to transition from Phase II to Phase III before beginning Phase II. This means that a funding pool and insertion plan is needed for each projects. He also extolled the matchmaking effort that John Williams and his team from the Navy and others had developed to match prime contractors with small businesses. This is expedited by the Navy’s Technology Assistance Program and similar gatherings by the Defense Department, NASA, and others. The Advantage of Beginning with “Phase Zero” To overcome small businesses’ difficulty in selling to prime contractors, he said that the key element is to start working with small businesses in “Phase Zero,” before Phase I even begins, so that they can better prepare for insertion as they go through Phases I and II, building trust with the customer and the prime, and trying to mitigate the perceived risks of working with a small firm. He underlined the importance of incentives, and said that what truly motivates prime contractors—and small businesses—is return on investment. Because prime contractors, like every other organization, are resource constrained, they need to make sure the Phase III process is efficient if they are going to invest in it. They must also be sure that the customer and the end user—the warfighter— receive the benefits. Finally, he said, the prime contractors needed good metrics to evaluate the Phase III transition and the performances of customers, prime contractors, and small businesses. This conference, he said, had produced many good suggestions for metrics and had strengthened the network for sharing them across the different services, agencies, and prime contractors. Dr. Gansler interjected that the three large prime contractors at the conference had all indicated that within the past year, they had noticed a shift toward more pull from the prime contractors, which is one of the key elements in making this program a success. John WilliamsU.S. Navy Mr. Williams said he had always been a strong proponent of Phase III transitions, and of keeping the focus on Phase III throughout the SBIR program. Phase

OCR for page 140
SBIR and the Phase III Challenge of Commercialization: Report of a Symposium III transitions involve multiple partners who must all address the same issues together, as they had been doing at this conference. Noting that the Navy has a technology-pull approach, while the other services tend toward a technology-push approach, he suggested that it was always more challenging to transition technologies from a research or laboratory environment into an acquisition program. The Navy SBIR program has evolved to a point where the people who control the money for SBIR awards—the Program Executive Offices—are the people who plan the Phase III transition; and, their acquisition program offices directly influence the flow of Phase III dollars. He recommended that the most important metric should be Phase III funding to the SBIR firm, and that those SBIR-managing Navy offices that generate significantly more Phase III follow-on funds should be rewarded. The Navy program, he said, basically gives back the 2.5 percent SBIR assessment to all assessed Program Executive Offices, but he feels that providing more funds to those Program Executive Offices that generate more Phase III metrics makes better sense. Since 2002, the Navy has awarded more Phase III dollars (non-SBIR funds) than Phase I and II SBIR dollars combined. In fact, for every SBIR dollar, Navy SBIR projects raise almost two dollars in additional funds. He said that this was one result that Congress likes to see, and would like to see more. Using the Company Commercialization Report He also said that SBIR firms should be able to compete for the “other” 97.5 percent of extramural RDT&E dollars, in addition to the 2.5 percent reserved for SBIR. Accordingly, the Navy uses a Company Commercialization Report (CCR) through which SBIR and non-SBIR revenues for small firms are identified. Even for small firms winning many tens of SBIR awards, he said, SBIR funds represented about 25 percent of their revenues—never more than 50 percent. That is a good sign, he said; if a company continued to win only SBIR awards, but not other awards, it’s likely that its technologies are not being transitioned. He recommended that the Academies use the CCR for measuring Phase III results of Phase II awards. The CCR captures not only Navy SBIR firms, but also many small firms working with the other agencies. One challenge in building the Company Commercialization Report is that those in government who are evaluating the data are typically engineers who lack experience in business. He recommended that the NRC committee consult with leading economists and other experts to develop insight about how to gather the most relevant data and how best to evaluate them to measure effectiveness in commercialization.

OCR for page 140
SBIR and the Phase III Challenge of Commercialization: Report of a Symposium Blending a New Technology with Other Technologies He mentioned a business technical assistance program first initiated for the Department of Energy, and to some extent for NIH and NSF, by Dawnbreaker, Inc., that educates firms about how to commercialize technologies. In the Navy version of this program, he said, the objective is to show small firms actual requirements and reports describing the specific needs of the customer. This would include the details of other technologies it would have to work with, whether it needed to be licensed, and other information. A paradox for the Navy SBIR program is that it has been so successful with their Transition Assistance Program program that it is beginning to drain its administration budget, given that it costs the Navy about $15,000 per small business participant. In one sense, this does not appear to be a serious expense—within 20 months of the program, participating firms average $3 million in Phase III awards, so the return on investment is almost a hundred-to-one. Nevertheless, he noted that it is still difficult to find additional dollars to put on a SBIR project to encourage commercialization because of Title 301 in the 1992 legislation, which describes what funds could be used for technical assistance. Because of the way this rule is written, he said, it is impossible to conclude whether the same percentage of funds allowed to support technical assistance in Phase I can be used in Phase II. He requested clarification of the rule, since the ability to use such funds for Phase II would provide up to $12 thousand per small firm to SBIR agencies to help SBIR companies. DISCUSSION James Rudd of the NSF said that his agency was involved in commercialization, and asked the DoD participants whether they would declare their SBIR program a success if a certain percentage of their R&D funding came from the SBIR program. Mr. McNamara responded that for an organization such as his, which spent between $5 and 6 billion a year, much of it on submarine procurement and costly ship maintenance, the SBIR money was a small amount, but when used, it has a great return on investment. Nonetheless, in parts of his organization, especially in electronics, where R&D is central, SBIR played a larger and more important role. The Cost-effectiveness of Small Business Another questioner asked how the services would rate the impact of SBIR companies on their overall innovation and business activities. John Williams replied that the impact was large. He added that small businesses, especially those coming through SBIR, were about 30 percent cheaper than large businesses, and also cheaper than laboratories and field activities. In

OCR for page 140
SBIR and the Phase III Challenge of Commercialization: Report of a Symposium addition, he noted that some small firms had proven more reliable than large firms. More broadly, he said, the process benefited by “adding people to the landscape” and producing more options for the services. Moreover, a lot more innovation had occurred at the big firms, he said, because they knew they were being measured against organizations other than their peers. Performance Improvements at Lower Cost Dr. Gansler added that a lot of SBIR activity occurred not at the weapons system level, but at the sub-system, component, or even the materials levels. For example, SBIR contributions often brought performance improvements at lower cost. This, he said, is “the way the real world, the commercial world, actually operates,” and that government should try to follow the lead of the computer industry, which produces new models at lower cost every two or three years. A New Metric to Gauge Phase III Success … Another participant said that dollar amounts and percentages were less valuable as metrics than the results of the Phase III contracts as requested every year by the Small Business Administration (SBA). “That,” he said, “is the easiest and cleanest metric” to indicate success, and it would be inexpensive. The SBA metrics would not provide a complete picture, he said, but they could show trends. In the same way, the achievements of the prime contractors could be measured. Success stories were not helpful, he suggested; every large program should have many success stories; and there were no clear metrics by which to measure innovation. The program should focus simply on measuring how much additional money was going to SBIR firms. “That was the intent of Congress,” he concluded, “and an easy way to measure it.” … and a Caution about this Metric Larry James of the Department of Energy urged caution about using the simple metric of Phase III dollars, which might drive the program further toward the development side. The “Innovation” in SBIR would eventually be lost. The program will gravitate toward low risk, incremental development projects. The program’s major benefit to the nation is that Small Business Concerns are willing and able to take on high-risk, innovative research that provides disruptive technological advances in the marketplace.