Panel I
SBIR at the Department of Defense

Moderator:

William B. Bonvillian

Office of Senator Lieberman

Mr. Bonvillian introduced himself as legislative director and chief counsel for Senator Lieberman, who served on both the Armed Services Committee and the Small Business Committee. The senator, he said, “has been a close observer of this program, as have I, since the time he arrived in the Senate. He worked hard on the authorization in the early 1990s, as well as the more recent authorization.”

The Roots of SBIR

He began by outlining some of the history of the SBIR program. The program “prelude” took place at the National Science Foundation. This was stimulated by an observation made in 1976 by Senator Kennedy, who said that the portion of the NSF’s R&D budget going to small business should be raised to 10 percent. That initial effort became the basis for the SBIR program itself in 1982 and 1983, said Mr. Bonvillian, even though that was a period of big government, big corporations, and big research universities. This program and its underlying philosophy “didn’t fit with that pattern.” R&D and its agenda were largely controlled by dominant government agencies, such as the Department of Defense; the large, traditional, and powerful corporate labs; and the major research universities. At the same time, there was a growing belief “that something interesting was going on in small firms.”

Delivering a “Huge” Return

The SBIR program started as a very modest, $45 million “tax” on R&D. It was systematically attacked by the large R&D organizations on the basis that



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 45
SBIR Program Diversity and Assessment Challenges: Report of a Symposium Panel I SBIR at the Department of Defense Moderator: William B. Bonvillian Office of Senator Lieberman Mr. Bonvillian introduced himself as legislative director and chief counsel for Senator Lieberman, who served on both the Armed Services Committee and the Small Business Committee. The senator, he said, “has been a close observer of this program, as have I, since the time he arrived in the Senate. He worked hard on the authorization in the early 1990s, as well as the more recent authorization.” The Roots of SBIR He began by outlining some of the history of the SBIR program. The program “prelude” took place at the National Science Foundation. This was stimulated by an observation made in 1976 by Senator Kennedy, who said that the portion of the NSF’s R&D budget going to small business should be raised to 10 percent. That initial effort became the basis for the SBIR program itself in 1982 and 1983, said Mr. Bonvillian, even though that was a period of big government, big corporations, and big research universities. This program and its underlying philosophy “didn’t fit with that pattern.” R&D and its agenda were largely controlled by dominant government agencies, such as the Department of Defense; the large, traditional, and powerful corporate labs; and the major research universities. At the same time, there was a growing belief “that something interesting was going on in small firms.” Delivering a “Huge” Return The SBIR program started as a very modest, $45 million “tax” on R&D. It was systematically attacked by the large R&D organizations on the basis that

OCR for page 45
SBIR Program Diversity and Assessment Challenges: Report of a Symposium government ought not to be funding private-sector activities, and also on the premise that it constituted a “raid” on more “legitimate” R&D activities. But when Roland Tibbets and others at the NSF evaluated the program, they found that it had delivered a huge return of approximately 30 times the government’s initial investment—a total of $9.1 billion. The 1990s, he said, confirmed to many federal policy makers that federal economic policy should be “at heart an innovation policy,” and that innovation was going to be the key to productivity gains and economic growth. “It was not only a matter of changing the measures of capital and labor supply traditionally used by the economics profession,” he said, “but bringing an understanding that innovation can change the entire curve of growth and fundamentally affect societal well-being. The SBIR program fit with an emerging view of capitalism that I think we’ve begun to see clearly in the 1990s, a kind of creative destruction process.”3 SBIR as a “Stealth” Program The SBIR program has really been a “stealth” program, he said. “It’s the least-known of the handful of federal government programs that are aimed at bridging what’s known as ‘the valley of death’ between the R&D and the venture-capital funding stages.” Its low profile was at least partly due to its decentralized nature and dispersion among many agencies. While this allowed flexibility and adaptation to the needs of different agencies, it had also resulted in a low level of oversight and evaluation. Thus there is little known about the following questions: Commercialization: Is a sufficient degree of commercialization being achieved by the agencies supporting the program? Should the success of the program be measured by the ability of companies to commercialize their products, or is some degree of usefulness to government a sign of success? Cost-sharing: Should there be more focus on obtaining cost-sharing or company participation by the companies participating in SBIR? Multiple awards: Are there too many repeat awards to recipients that have not commercialized their products? Program structure: Have the delays between Phase I and Phase II awards created a breakdown in program continuity and therefore a disincentive for program participation? Is there a need for a real “Phase III,” with a continuing role for government? 3   The term, “creative destruction” originates with Schumpeter, See Joseph Schumpeter, Capitalism, Socialism and Democracy, New York: Harper, 1975, (orig. pub. 1942), pp. 82-85. See also Richard Nelson (ed.) National Innovation Systems, Oxford: Oxford University Press, 1993.

OCR for page 45
SBIR Program Diversity and Assessment Challenges: Report of a Symposium Networking: Most SBIR awards deal with what economists refer to as complex technology, which is generated by complex science. Both complex technology and complex science are supported by networks and collaboration—for purposes of knowledge diffusion, knowledge application, collective learning, product development, and product evolution. Should there be improved networking for SBIR, with better cross-fertilization across SBIR agencies and partnerships among program participants? The Need for Thorough Evaluation He said that after two decades of operation, Congress had realized the need for a thorough evaluation of the SBIR program. In the SBIR re-authorization of 2000 Congress required participating SBIR agencies to work with the National Research Council to develop program metrics and conduct a six-year assessment of program quality. Many of the SBIR agencies were not enthusiastic about this additional exercise in oversight, he said, and it took about a year to agree on an evaluation and another year to initiate the necessary funding. Nonetheless, this symposium signified that the process was now underway. He turned to the SBIR program at the Department of Defense (DoD), the subject of this panel. The DoD SBIR program, whose budget had grown from $460 million in 1990 to $733 million in 2002, represented more than 50 percent of the entire SBIR program. He said that the DoD’s SBIR leadership, through the work of symposium participants Jacques Gansler and Jon Baron, among others, championed the federal government’s effort to explore the need for a thoughtful programmatic evaluation. That leadership, he said, was an important contribution that had helped to bring about this program-wide evaluation process. Within DoD itself, the internal evaluation resulted in the addition of new program initiatives. One was the continuation of the Fast Track sub-program to provide funding continuity between Phase I and Phase II. And Congress had both authorized and appropriated $12 million for a “challenge program,” intended as a Phase III transition period that provides some federal assistance. There had also been difficulties, he said. One sub-agency (the Missile Defense Agency) had not been enthusiastic about continuing to participate in SBIR, but had recently agreed to do so. And the SBIR office had faced staffing shortages and a 40 percent cut in its program administration budget. The Importance of SBIR to DoD Despite these difficulties, Mr. Bonvillian affirmed that “SBIR is potentially very important to the Defense Department.” “The Defense Department acknowledges a revolution in military affairs; it acknowledges a need for transformation. There are dramatically changing military rule sets, based upon new sources of

OCR for page 45
SBIR Program Diversity and Assessment Challenges: Report of a Symposium power and how those new sources are applied.” He described SBIR proponent Admiral Art Cebrowski, then working for Secretary Rumsfeld, as “one of the leading military thinkers of our time,” and quoted a recent talk: “All the world’s economies are moving from the industrial age to the information age. The issue for the U.S. military is how to sustain a competitive advantage in this rapidly changing area. The nation is entering an era where advantages are conferred on the small, the fast, and the many. Size shrinks because information is substituted for tonnage.” Updating the Evaluation Model That quote, he said, applied well to the defense innovation base that DoD needed to utilize more extensively. DoD still relied on essentially the same innovation system that evolved out of World War II, he said. Aside from several coldwar updates, notably DARPA, it still overwhelmingly depended on large platforms and the large defense contractors that could support these platforms. But in using this model, he said, DOD was having “profound problems” with technology transition—with moving technologies from the R&D stage to the service acquisition stage. “SBIR is an interesting option for DOD,” concluded Mr. Bonvillian. “It’s a very open and surprisingly sizeable program now; it creates lots of opportunities to do innovation in new ways.” He asserted that SBIR brought to DoD a new opportunity for innovation in technology transition, one of the department’s “big problems.” Arguably, he said, this would require a major effort by DoD to analyze the opportunities that SBIR creates, but one well worth making. He closed with an “SBIR Haiku” that expressed what was to become a common sentiment at the symposium: “Without measurement metrics SBIR remains As unknowable as the surface of the sea” ACHIEVEMENTS, OPPORTUNITIES, AND CHALLENGES Charles J. Holland Department of Defense Charles Holland introduced himself as overseer of the Pentagon’s science and technology programs that were executed by the service and defense agencies. He noted that he works for Ron Sega, Director of Defense Research and Engineering. Dr. Sega, in turn, works with Mr. Pete Aldrich, who oversees all acquisition technology and logistics, a position previously held by Dr. Gansler. Mr. Aldrich reports directly to the Secretary of Defense.

OCR for page 45
SBIR Program Diversity and Assessment Challenges: Report of a Symposium The Mission of DoD’s Science and Technology Program The mission for the science and technology program, said Dr. Holland, was to provide superior and affordable technology to the people who execute defense missions in the Army, Navy, and Air Force. All the science and technology programs of the Department of Defense could be arranged on a spectrum of activities, beginning with basic research that is performed primarily by universities, small businesses, and the defense laboratories. Moving along the spectrum are progressively more applied new technologies and “spiral technologies,” which typically have the goal of incorporating scientific ideas and technologies into systems.4 He illustrated the science and technology programs at DoD by describing the relatively small ACTD5 program. This program examined mature technologies, such as the Global Hawk Predator, an unmanned spy aircraft. The approach of the program would be to buy several of these aircraft and try to ascertain both whether the technology truly worked and whether it was consistent with the doctrine and tactics of the Department of Defense. ACTD asked whether the technology was important as well as whether the DoD could make good use of it. It also examined major systems, such as the Joint Strike Fighter,6 of which the U.S. and its military allies planned to buy more than 3,000 over the next 20 years. The program did “upgrades and refreshes” to those systems. The ACTD program relied upon a considerable number of people who gave advice, including the Joint Staff and the war-fighters “who tell us what they really need and how they really want to operate in the future.” It also received useful advice from Congress about what the program should be, and from the department’s own requirements process. “So we get requirements from the top,” he said, “and we try to do technology opportunity from the bottom.” The objective was to ensure technology superiority across all phases of acquisition. 4   For budgeting purposes the Department of Defense divides its R&D spending into three categories: 6.1 (basic research), 6.2 (applied research), and 6.3 (advanced technology development). These categories are used more for convenience than to imply rigid divisions among research activities. Research is a complex process characterized by numerous overlaps and feedback loops. 5   The Advanced Technology Concept Demonstrations program is budgeted at about $200 million a year. According to Sue Payton, Deputy Undersecretary of Defense for Advanced Systems and Concepts, the primary role of her unit at the Pentagon “to rapidly transition technology from defense and commercial developers into the hands of the war fighter.” 6   The Joint Strike Fighter, according the a DoD announcement, will be the world’s “premier strike platform beginning in 2008, and lasting through 2040.” A production contract was announced in Fall 2001 for a multi-version, “all-aspect stealth platform” designed to “replace the aging fleet of Air Force A-10s and F-16s, the early model Navy F/A-18s, and the Marine Corps AV-8Bs.”

OCR for page 45
SBIR Program Diversity and Assessment Challenges: Report of a Symposium Upgrading Technical Systems He gave some additional examples of the systems that were studied by the ACTD program, including the F-22 fighter airplane, where “we try to quickly mature technology and do some operational demos and concept demonstrations.” They also did “spiral modernization,” in which they tried to upgrade technological systems. This was the motivation for the Joint Direct Attack Munitions (JDAM)7 program, in which a family of unsatisfactory “dumb” bombs was converted to highly effective “smart” bombs through the addition of a guidance system. With this modification, the JDAM had become “really the GPS guided weapon of choice these days inside the department,” he said. One of the challenges for DoD, he said, was to look beyond single platforms and see them as parts of more complex systems. The Army, for example, was emphasizing rapid deployment, which meant that individual units must be smaller and more mobile than they had been in the past. This, in turn, meant that units must be networked to optimize their operational capability. This was one of the transformations to which Dr. Holland was contributing as the Army developed its future combat systems. “Success Stories” He turned to some “examples of success stories” that allowed the U.S. military to be dominant across the spectrum of its activities. One was the emphasis on night vision and stealth aircraft that was initiated by Secretary of Defense William Perry in the late 1990s. The capability of phased radar, stealth aircraft, and other tools had allowed the military to “own the night” and gain a range of advantages through remote sensing, as well as to commercialize night-vision tools. These advances should increase in the future with the deployment of adaptive optics and lasers, both for weapons and for communication. The strategic environment had changed, he went on: no longer was the military faced with a single large threat from the Soviet Union, but with a range of “asymmetric” challenges from smaller states, stateless entities, and even individuals.8 The U.S. faced asymmetric challenges in having to defend itself against chemical-biological attacks as well as respond to them; in defending both military personnel and civilian populations; and in defending against “information 7   The Joint Direct Attack Munition (JDAM) is a low-cost guidance kit that converts existing unguided free-fall bombs into accurately guided “smart” weapons that can be launched approximately 15 miles from the target. The weapon is intended to have high-accuracy, all-weather, autonomous, conventional bombing capability. 8   “Asymmetric threat” describes the almost limitless range of challenges posed by groups or individuals unable or unwilling to oppose conventional military strengths directly (“symmetrically”). “Asymmetric warfare” describes strategies and tactics designed to thwart or counter these challenges, and is seen within the Pentagon as an urgent priority following the September 11, 2001 attacks.

OCR for page 45
SBIR Program Diversity and Assessment Challenges: Report of a Symposium warfare” attacks. Military personnel not only had to operate in war-time environments, but were also expected to perform a broader range of economic, political, and humanitarian activities. Finally, he said, the funding environment for military research and technology had changed significantly. The DoD had changed from being the nation’s dominant funder of R&D to one of many funders. A relatively larger portion of R&D was now funded by the private sector, so that the military was challenged to quickly locate and procure on the open market the technology it needed and to incorporate it into war-fighting systems. He then listed some new “war-fighting concepts” the military is using to change its battlefield strategies, including: Fuller dominance of space; “Network centric” warfare; Unmanned systems for land, air, sea, and underwater. He also listed several areas in which the Department of Defense is placing additional research funding in order to “capture what’s really going on in the basic research community”: Nanoscience and advanced materials; Advanced power; Human dimensions and psychological factors; Directed energy. Planning Research Investments He said that the way the department ensured an adequate, coherent investment in critical research areas was through joint planning across the military services and defense agency efforts. The Defense Technology Area Plan (DTAP) was one example—a document of about 400 pages that showed the focus, content, and principal objectives of the overall DoD science and technology efforts. The services jointly review the DTAP programs across the spectrum to monitor their own progress against goals. This allowed planners the ability to avoid duplication and terminate programs that are not making or contributing to progress. This plan also provided a basis for acquisition decisions and was structured to speed the transition of mature technology to the operational forces. Finally, the document was used to help optimize the contribution of the SBIR program. In this way, the SBIR program fit naturally into agency-wide planning and strategic efforts, said Dr. Holland. The DTAP helped translate the overall R&D strategy into war-fighting capabilities and provided the framework for selecting SBIR topics. In the wording of the department, “SBIR topics will fall within one or more of DoD’s key technology areas, as defined by DTAP. This will ensure that the SBIR projects are an integral part of the DoD R&D program.” So al-

OCR for page 45
SBIR Program Diversity and Assessment Challenges: Report of a Symposium though the SBIR program received only 2.5 percent of the R&D budget, by the rules of the program, the program was nonetheless “a very important component,” he said, and its effect was to better integrate the overall R&D investment. Individual activities are administered by appropriate units in the Army, Navy, Air Force, and DARPA. In practice, the Army, Navy, Air Force, and DARPA independently developed topics that were consistent with the DTAP and submitted them to Dr. Holland’s office. His office then reviewed the topics to eliminate duplication and to ensure that the proposals were clearly written and responsive to the criteria of the plan. A program overseer and a small technical staff were responsible for these functions. DoD is the Major Participant in SBIR He then showed a slide demonstrating that the DoD was a large participant in the overall federal SBIR /STTR program (see Figure 1). For SBIR, the department’s share was $773 million. Of the ten components, the five largest, in order, were the Air Force, Navy, Army, Missile Defense Agency, and DARPA. He said that his office controlled a small amount of SBIR money for programs such as the University Research Initiative Program and the DoD High-Performance Computing Modernization Program. The agency also had a smaller STTR program, with a budget of $42 million and five participating components. He discussed the results of the program, which he said showed a good rate of transition to commercialization (see Figure 2). Between 1994 and 1999, 57 percent of SBIR firms with recent Phase II projects had achieved sales and/or investments. While most had received sales and/or investment amounts of less than $1 million, several percent of the companies had sales and/or investment amounts of between $10 and $100 million. He listed a series of web sites for some of the “success stories” of the DoD SBIR program.9 Finally, he said that the SBIR program should be seen as spanning the whole spectrum of R&D, from early research to acquisition to procurement. “But the real purpose of this program,” he said, “is to harness the innovative talents of our nation’s small technology companies” for the benefit of the U.S. military and U.S. economy. The trick to maximizing that benefit, he said, was to bring in enough topics that we included the ones that “make the big breakthroughs.” He said in closing that the SBIR program was not the only way the department benefited from small businesses, which also compete throughout the science and technology program and become important suppliers to acquisition pro- 9   http://www.dodsbir.net/Materials/SuccessStories.htm, http://www.aro.army.mil/arowash/rt/sbir/stories.htm, http://www.navysbir.brtrc.com/success.htm, and http://www.afrl.af.mil/sbir/impact.htm.

OCR for page 45
SBIR Program Diversity and Assessment Challenges: Report of a Symposium FIGURE 1 DoD FY2002 SBIR/STTR programs.

OCR for page 45
SBIR Program Diversity and Assessment Challenges: Report of a Symposium FIGURE 2 SBIR Projects attract sales and investment.

OCR for page 45
SBIR Program Diversity and Assessment Challenges: Report of a Symposium grams. He wished the study panel well in its evaluation, and said that he looked forward to the advice they would bring to the agencies. As a final thought, he noted the relatively low levels of SBIR awards and recalled a condition in the law that would allow agencies to increase these levels with inflation. He asked the panel to look into the feasibility of this. A DEFENSE SUPPLIER PERSPECTIVE Richard Carroll Digital System Resources10 Richard Carroll said that he would speak on the SBIR program and “the impact of small, high-technology businesses and competition.” He said first that innovation and competition were concepts “basic to America itself.” Whereas the history of other countries is dominated by kings, queens, princes, barons, generals, and perhaps religious leaders, he said, American history is rich in inventors, entrepreneurs, innovators, scientists, engineers, and tinkerers of all kinds. Pioneer inventors include such familiar names as Benjamin Franklin, Eli Whitney, George Washington Carver, Thomas Edison, Samuel Morse, Alexander Graham Bell, Henry Ford, and the Wright Brothers. But equally significant are the names of entrepreneurs and inventors seen on products today, such as Singer, Otis, Westinghouse, Carrier, Eastman, Levi Strauss, Buick, Olds, Dodge, Colt, Browning, Sperry, and Birdseye. Most of the huge companies that now produce these products began in the form of small businesses whose founders had little more than a good idea. Innovation Drives the Economy Innovation and competition continued to be the driving force of the U.S. economy, he said, as new innovations continuously destroyed existing ways of doing things and powered the growth of small firms into giants of industry. He cited well-known examples of this growth pattern in the field of information technology, where over the last 25 years influential high-tech businesses such as Apple, Microsoft, Dell, Compaq, AOL, and Netscape had developed from tiny companies to dominant names that changed the business paradigm and culture of their respective business domains. He then mentioned the other side of innovation—the “surprise causalities.” Virtually no one living in 1975 suspected that IBM would lose its dominance of the computer hardware business by 1995, he said. Nor did anyone living in 1990 10   In 1982 Richard Carroll founded Digital System Resources, a computer hardware and software company that specializes in producing technology for national security. The company has grown to employ nearly 500 people, with revenues in 2001 of about $110 million.

OCR for page 45
SBIR Program Diversity and Assessment Challenges: Report of a Symposium place, I have no doubt that over the next 5 years it’s going to be a billion-dollar product.” An Unforeseen Success He added that he had not foreseen this product, or its success, when he and his colleagues began to grapple with the challenge from the SBIR program. Yet the accomplishment was sufficiently notable that the two inventors of the product had just received a lifetime-achievement award for “most significant environmental safety and health advance in the semiconductor industry during the decade of the 1990s.” Mr. Banucci emphasized that he was a very strong proponent of the SBIR program—and “an even stronger proponent of making it better.” He said that his own focus for ATMI was to develop a company that was performance-driven. He said that his company “benchmarks other companies, milestones where we are, look for best practices, and tries to execute them across the entirety of our organization.” He said that he talks about these points to his company, especially the last phase of execution. The SBIR gives many companies the opportunity to “drive the ball, get it on the fairway, and get it on the green.” But, he concluded, there is little value in those steps unless there is an incentive to “get the ball in the cup.” He said that he would help the study panel in any way he could so that more SBIR companies were able to bring their products to market and find commercial success. Jon Baron Coalition for Evidence-Based Policy Mr. Baron said he had left the SBIR program several years previously and that he was glad to be back as a member of the National Academies’ Study Committee. He said he would offer observations on the program based on his experience as a Congressional staff member who worked on the reauthorization of the SBIR in 1992 and the initiation of the STTR program. He also served as SBIR program manager for the Department of Defense. He said he had benefited by being “a really avid consumer” of evaluations of the program by the Government Accounting Office, starting in 1993; by the DoD’s Director, Defense Research and Engineering, in 1997; and by the National Research Council, which recently reported on the Fast Track portion of the DoD program.12 All of these surveys, he said, helped companies and observers to understand what had happened to their SBIR products and technologies. 12   National Research Council, The Small Business Innovation Research Program: An Assessment of the Department of Defense Fast Track Initiative, C. Wessner, ed., Washington, D.C.: National Academy Press, 2000.

OCR for page 45
SBIR Program Diversity and Assessment Challenges: Report of a Symposium Five Observations He offered five observations about the SBIR program. Without data on program outcomes, it is difficult for anyone, including program managers, to know how well the program is doing. As an example, he cited a DoD initiative led by Jacques Gansler to collect data whenever a company submitted a proposal for a new DoD SBIR proposal. The company would be asked to list all its previous Phase II awards, along with the sales, both commercial and DoD, that resulted from the awards and any additional investment the company had received. When that initiative was started, the database revealed that one company had reported over a billion dollars in sales. Officials were skeptical until they looked more closely and found that the company had in fact developed a new technology that increased the number of circuits on a computer chip by about 30 percent. The technology, which had been developed and licensed to another company to produce, was changing the state of the art in the industry and improving the computing power of virtually every commercial and defense system. It was an enormous success, and yet no one at the SBIR office had known about Science Research Laboratory, Inc. (SRL) of Somerville, Massachusetts, which licensed the technology to Cymer, Inc. As another example, he said that DoD officials had found that a number of companies had participated in the SBIR program and won a large number of Phase II awards; had very good reputations in the program; but “had never commercialized anything to speak of.” By contrast, there were a few companies that had a “terrible reputation” in the program but had a commercialization track record that was “mixed, or even a little bit above average.” Thus, without good data from the study panel, similar to what the GAO had produced in the past, it would be difficult to truly know what was happening in the program, beyond a history of anecdotes. Based on the data collected by DoD, many SBIR companies developed technologies that were not successfully commercialized in either government or commercial markets. This is partly because of the nature of research and development, where most new technologies fail to be adopted. Another cause, revealed in the 1993 GAO study, the DDR&E study, and in DoD’s commercialization data, was evidence of “systematic failure” in the program. Some companies repeatedly won SBIR awards but consistently failed to convert those awards into viable new products sold to commercial or government customers. Successes of the SBIR program have had an enormous impact on the economy and on defense systems.

OCR for page 45
SBIR Program Diversity and Assessment Challenges: Report of a Symposium He cited a technology produced by Digital System Resources that had improved the computing power of sonar equipment by a factor of 20, reduced the cost of that equipment by a factor of 2, and reduced the size of the equipment on submarines by a factor of 2. The technology had been installed on almost the entire fleet of Navy submarines. The program is characterized by a skewed distribution of successes. At the top were a few “gigantic” successes, such as the two that had been described by panel members. Below those were perhaps a hundred modest commercial or defense successes. Farther below those successes, he said, the program had produced few results. He said that the major successes had more than paid for the government’s entire investment in the SBIR program. Even so, however, it would be desirable to have a less skewed distribution of success. “I think the great challenge to the program,” he said, “is to improve the success rate. Can the incentives be changed to make this happen?” He said that the DoD leadership did implement a number of program reforms that successfully shifted the incentives in the program, and thereby substantially increased successful commercialization in both military and commercial markets. One positive change, he said, was the Fast Track initiative: a company that receives some matching funds from an outside investor toward the end of Phase I is awarded continuous Fast Track funding between Phase I and Phase II, along with the highest priority for a Phase II award. A National Academies study examining the Fast Track program found that the rate of preliminary commercialization was about seven times higher than the rate of a control group of regular SBIR participants. In addition, he said that a commercialization tracking system put in place by DoD seemed to be successful in identifying both companies that are “systematic successes” and companies that are not as likely to produce successful products. Kenneth Flamm University of Texas at Austin Dr. Flamm (who had worked with the SBIR program at the DoD and had also participated in the National Academies study of the DoD SBIR Fast Track program) opened his remarks with the observation that the comments he had heard at the symposium had fallen into two basic categories: these are the rationale for the SBIR program and the operation of the SBIR program. He said that the current Academies study seemed to be focused primarily on the operations of the program, rather than on the rationale. He suggested that the two topics are

OCR for page 45
SBIR Program Diversity and Assessment Challenges: Report of a Symposium organically related, and that a study of operations would have to consider the program’s rationale, whether or not it was an explicit theme of the study. He said he would address both of those topics, “perhaps in blunter terms” than he might have used when he worked in Washington. He noted that it would be naive to ignore the “significant political component in the SBIR program.” It was funded by Congress, and small business is very popular with the nation’s political leadership. He also said it would be ingenuous to pretend that initiation of the program had been preceded by a “grand, focused analysis” to specify exactly how the program should function. “The SBIR program is a politically popular program,” he said, “and it was put in place because our political leadership liked the concept of having R&D dollars spread a little more widely than they were being spread before. I think you have to start with that acknowledgment.” Addressing Capital Market Imperfections Having said that, however, he noted that the SBIR did seem to address a number of legitimate purposes. One purpose, stressed in the first National Academies report, was to address imperfections in the capital markets. For structural and institutional reasons, small businesses may have more trouble accessing the capital markets, and this could be a handicap in commercializing their innovations. At the same time, he said, it is not clear that the SBIR program is an optimal response to the imperfections of capital markets, where such imperfections are not well-documented. But one result of the National Academies studies was to point out that the SBIR program did play some role in rebalancing these kinds of capital market imperfections in at least some documented instances. Defraying “Contracting Overhead” A second issue was what he called “contracting overhead.” When dealing with the Department of Defense—he said that he had had less experience with other agencies of the government—companies must learn to deal with a complex and sometimes “arcane” contracting system characterized by many rules and procedures. The large defense contractors that show up on the “Top 100” list were organizations that had made the strategic decision to make large investments in systems and overhead to manage the complex details of the contracting process. Given the difficulties of coping with the contracting process, he said, one could regard the SBIR program as a kind of subsidy to small firms who lack the financial resources to invest in “contracting overhead.” The program had had the effect of defraying some of the fixed costs of accessing the government procurement system.

OCR for page 45
SBIR Program Diversity and Assessment Challenges: Report of a Symposium Overcoming the Small-firm Disadvantage in Lobbying A third rationale for the SBIR program, he suggested, was that in the DoD, as elsewhere, there is a strong political component in the contracting system. That is, all the major defense contractors have active Washington operations that are skilled at lobbying on Capitol Hill and at the Pentagon. In a delicate ballet, the services, the DOD, and the contractors engage in a continuous give and take that characterizes the federal budgeting process. An outsider, without the overhead to maintain a well-staffed Washington office and cultivate appropriate connections at the Pentagon on a regular basis, is likely to enter the contracting system at a disadvantage. In this sense, the SBIR program provides some rebalancing of market imperfections in a political sense. Because small firms cannot afford the expense of traditional lobbying, the SBIR affords a route for small firms to enter the procurement system directly. How to Characterize Successful Outcomes for SBIR? He cautioned, however, that there is some risk in characterizing the SBIR program solely as a way of overcoming imperfections in the commercialization process. One of these risks is that “a reasonable proportion” of programs the SBIR agencies invest in have nothing to do with commercialization. Some are specialized R&D services that are purchased by government agencies from small contractors or other specialists. These can be commercialized in the sense of selling them or their output to the DoD, but the services are too specialized to stimulate a large commercial market. There is essentially only one very large customer for weapons systems in the United States, so a certain proportion of agency missions have nothing to do with selling commercial products, and the R&D that supports those missions, by itself, is not related to selling commercial products. It follows that the SBIR program should not require that all agency missions be commercialized. A second issue, he said, is that “commercial” should not be confused with “small.” A small business that is successful in commercializing new technology almost certainly will not remain small. Successful small, high-tech businesses gradually become large, high-tech businesses. Perceived Problems in the Program Having made these observations, he turned to some of the problems in the SBIR program observed in 1993 and 1994, when the subject of reforms first arose. The SBIR program was, and still is, regarded by many R&D program managers within the Department of Defense as a kind of tax on R&D. These managers were required to set aside a portion of their budget to spend on this

OCR for page 45
SBIR Program Diversity and Assessment Challenges: Report of a Symposium program, whether or not it suited their perceived needs. Those who have reviewed the SBIR program within the DoD have concluded that the amount of money set aside for SBIR is, taken together, considerable. Some have suspected that the SBIR funds were being used as a supplementary pension plan for laboratory workers. That is, laboratory scientists, after their retirement, would succeed in receiving an SBIR grant from their former employer as a pleasant way of topping off a long career at a defense laboratory. Whether or not this was really a common custom, the managers of the DoD SBIR program decided to make the contracting process much more transparent, including an open process of developing topics and soliciting bids on those topics, so that lab managers did not have the choice of hand-picking projects to benefit certain people. The reformers were also concerned about “SBIR mills”—companies or individuals who repeatedly were able to win SBIR grants because of good contacts and bureaucratic skills. The SBIR management was concerned that a small group of players had learned how to repeatedly dip into the contract trough without producing large benefits for the Department of Defense. A Pathway to Commercial Innovation At the same time, these reformers saw the positive aspects of the SBIR program. They were aware that many of the revolutionary technologies they wanted for military systems were likely coming from the commercial sector, especially in critical fields of information and communications technology. They saw that the DoD could use the SBIR program as a way to support better adoption and use of commercial, dual-use technology within military systems. But they also recognized that not every technology needed by the military could or should be dual-use. There would always be a need for small, commercial firms that provide specialized services to the DoD. It did not seem useful to try to fit everything into a single mold if that would mean squeezing out firms that primarily performed specialized services for the DoD. The study panel, he suggested, should consider the continuing need for such firms when evaluating the SBIR program. A substantial number of small firms do specialized tasks for DoD, and broad requirements for commercialization should not preclude them from SBIR support. In conclusion, the SBIR program has a diverse set of objectives, and requires a diverse set of firms to meet those objectives. It is desirable to use SBIR as a way to develop commercial technology and speed its use by the Department of Defense. It is also desirable to use SBIR to overcome some imperfections in capital markets and possibly in political dimensions of the contracting system. But it is also important to recognize that it is “logical and justifiable” for the SBIR to support different types of firms with varied products and goals. “Not everybody has to be the next Microsoft operating system,” Dr. Flamm observed. “The small guy working on DSP propeller recognition for the Navy

OCR for page 45
SBIR Program Diversity and Assessment Challenges: Report of a Symposium may also be a valid and legitimate use of SBIR funds, even thought it may not result in the next commercial breakthrough. He ended by calling the SBIR program “a very complex beast, a program with many objectives, many fathers, and many different support bases. It seems to me that we have to recognize that diversity in our evaluation.” DISCUSSION Metrics and Appropriate Rates of Success Charles Wessner of the National Research Council said he had been “anxious to ask” what would be an appropriate rate of success for the SBIR program at DoD. He also asked how the panel should measure those successes, and how an agency could come to balance the research objective with the commercialization objective when making decisions to award grants to small companies. Charles Holland of the Department of Defense agreed that the first task was to decide on a definition of success for SBIR awardees. “To me,” he said, “success would be actually adding a sustained set of workers to the pool to support the mission of the Department of Defense and the overall economy.” Here he said that a success rate of around 10 percent would be acceptable—if 10 percent of these activities, following a couple of Phase II awards, still had a certain fraction of people employed. Another way to measure success would be to count the products being bought by the Department of Defense. This would be easy to track for large procurements, such as the Joint Strike Fighter (JSF), but more difficult for the products of second-tier suppliers, which would include most SBIR firms. A Tax on Agency R&D? He commented on the “tax issue,” which pertains to major programs, such as the JSF. These are taxed in the sense that some portion of the R&D funds that might be spent on JSF work are assigned to someone else, according to the SBIR formula. To program managers, this is tantamount to a tax on the JSF program itself. Jacques Gansler of the University of Maryland added that some programs, such as the Missile Defense Agency, actually keep SBIR dollars within the program to increase its flexibility. Vinny Shaper, SBIR program manager for the U.S. Navy, said that in the Navy SBIR program, when a program like JSF is “taxed,” typically that money goes back to the program where it originated. Metrics and Success II Jon Baron of the Coalition for Evidence-Based Policy, returning to the topic of measuring success, said that there are different ways of “succeeding” that meet

OCR for page 45
SBIR Program Diversity and Assessment Challenges: Report of a Symposium technical objectives, such as producing new knowledge and attaining commercialization. Beyond those, however, he emphasized the importance of successes in the program that “change the world”—a sonar system that fundamentally changes the way submarines can operate, say, or a software technology that allows the use of a credit card at the gas pump, both developed under SBIR. That software technology had improved productivity across a number of industries, including the defense industry. Such successes, he said, change capabilities in a fundamental way and are the great achievements of the SBIR program. Richard Carroll said he approved of searching for metrics that could be used to make historical comparisons within the SBIR program, but he thought that such a search would fail. One reason was that this kind of study is now being done for the other programs, and comparable data have not been found. Another reason is that the activity of the SBIR program is causing larger firms to become more innovative as well. This has been the case in the experience of his company, Digital Systems Resources, which produces sonar equipment for the submarine fleet. He said that Lockheed-Martin, a large company, was now competing against his small company for the same market. “They don’t behave at all like they used to behave when we weren’t around,” he said. In other words, the impact in DoD had been that a large firm was now presenting alternatives to the products of a small SBIR firm, and the resulting “creative transformation” benefited the DoD by bringing a choice of products. Mr. Carroll said he could think of no effective way to measure this effect, but it was a significant and useful one. Intellectual Property Steven Wallach of the law firm Penney and Edmonds asked Mr. Carroll about the intellectual property provisions that had been added to the SBIR program. As an IP lawyer, he said, this seemed like a good idea, but he also suggested that it might stifle innovation, particularly in software. Mr. Carroll agreed that the IP issue was “absolutely essential here.” From its inception, the SBIR program had conveyed to the government the royalty-free right to use the products developed by small businesses. But the program also contained a limitation with respect to disclosure of intellectual property developed under the program: the government is prohibited from giving the data away to the firm’s competitors for a period of four years after completion of the work. The small business, he said, is highly dependent on its product, market environment, competitive posture, and business strategy. In some cases, an SBIR company may be best served by retaining its own trade secrets, patents, copyrights, or some combination of these. In other cases, it may be best served by making intellectual property available, in part or in whole, to its customers or competitors. But it is critical, said Mr. Carroll, that such decisions are made by the company, not by the customers. This contrasts with the case of Linux, the

OCR for page 45
SBIR Program Diversity and Assessment Challenges: Report of a Symposium open-source software system, where the inventors decided from the outset to give the technology away for the purpose of continual improvement by users. He said that his own company was distinctive in that it had decided to give its software away—even though it was not required to because the IP was protected under the SBIR agreement. This approach was very different from that of larger companies, which tend to retain IP for themselves and even to use it against smaller firms. He suggested that the government may find superior benefits in intellectual property that is freely distributed to the R&D community, where creative synergies can advance the state-of-the-art for the common good. Again, he said, it should be the small business’ decision to make. Taking Risks while Reducing Systematic Failure Greg Millman, who said he was responsible for the SBIR program at the National Institute of Allergy and Infectious Diseases, returned to the topic of metrics and said that success alone could be a very dangerous metric to use for assessing the SBIR program. In many cases, he said, failure is actually desirable because it indicates that the program has taken risks. If there were no risk-taking, there would be no need for the SBIR program. The venture capital and commercial sectors would simply pay for the research needed to develop the products they expected. He agreed with Mr. Baron that a few outstanding successes justified the many failures, and that it was counter-productive to aim for a higher success rate by selecting less risky projects. Jon Baron said he agreed with Dr. Millman’s comments, and that the challenge for the program was not to decrease the failure rate per se; R&D is inherently risky, especially the kind that has the potential for the greatest payoff. The challenge for the program, he said, is to reduce “systematic failure,” such the phenomenon of the company that participates in the program, moves from one contract to the next, and produces only a report or a prototype that is never used in any meaningful way. This kind of output is never developed into a product or used to increase the knowledge of the Department of Defense. Definition of Commercialization and Competition William Bonvillian of Senator Lieberman’s office asked whether, given the loss of competitive factors in defense contracting, SBIR should be described more explicitly as a competitive factor in the defense contracting business. Jacques Gansler said he had always been “a big advocate” of competition. He agreed with the example of the sonar systems as an illustration of the importance of introducing alternative technologies on a continuing basis, not just at an initial auction. “The only way you can do that,” he said, “is through the continuous development of new products, and obviously much of that comes from the small business community.”

OCR for page 45
SBIR Program Diversity and Assessment Challenges: Report of a Symposium He did say he was concerned that he had heard from two different people at the symposium a concern that the total amount of R&D funding going to small businesses was being dominated by the SBIR program. In the past, a significant amount of R&D was generated through small business innovations outside the SBIR program. One way this happened was through unsolicited proposals from small firms that wanted to introduce new research ideas to compete with existing products. Dr. Gansler said he would be worried if that was happening, and asked the study panel to find out if it was. If so, it would indicate that the DoD was becoming overly dependent on SBIR for small business research, which was not a situation he would favor. He also commented on Dr. Flamm’s observation about “commercialization,” and offered an alternative definition; namely, a product is “commercialized” if it is used by the government or the commercial marketplace. If NASA or the Defense Department uses a product, he said, it becomes part of the economy, generates employment, finds application, and therefore adds value to the nation and the agency. He urged the study panel not to judge the success of a company solely on whether its product finds uses in the commercial sector. Some government applications are obviously limited, he said, and yet the applications are highly useful. Richard Carroll said first that he did believe the SBIR was displacing other small business R&D. But he asserted that overcapacity in the DoD was a serious problem: “A very fundamental principle in the Department of Defense is, you don’t kill incumbencies.” This is especially true at the level of large programs and bases where overcapacity may be caused by political pressures. A real problem for the department, he said, is that it tends simply to reallocate funding rather than take the more difficult route of canceling or closing unneeded programs. Tightening Definitions Kenneth Flamm offered two comments. The first was a caution again an overly loose definition of product or commercialization: If a company does analytical services for a DoD lab and writes a report on its analysis, is that a product? “If you’re broad enough to cover everything,” he said, “then your definition loses meaning.” His second comment concerned competition. He said that participants had already discussed interesting examples where SBIR grants did provide some competition. However, he cautioned that this may be only a minor degree of competition “at the margins.” He said it was unlikely that the Small Business Innovation Research program would truly provide competition against the large systems integrators and platform builders. “The bottom line,” he said, “is that there has been significant concentration and consolidation in the U.S. defense industry, and there’s no way around that.” One irony, he said, was that this consolidation had been inspired in the early 1990s by the projection that reducing the number of contractors and their high fixed overhead would better use resources in a DoD

OCR for page 45
SBIR Program Diversity and Assessment Challenges: Report of a Symposium procurement budget shrinking by 40 or 50 percent. In fact, the procurement budget had returned to the same high level of the early 1990s, with a significantly smaller number of contractors. It is unrealistic, he affirmed, to expect small businesses to compete with the remaining large systems integrators, except “for bits and pieces of inputs into the systems projects.” He concluded that the faults of the DoD procurement system would not be solved simply by the SBIR program, but would require further analysis. “The competition issue is a real one,” he said, “and ought to be considered. The size and configuration of the defense industrial base is an issue of national importance, and I don’t think that relying on SBIR to provide competition at the margins is going to fix it.” Richard Carroll said he did not disagree with Dr. Flamm’s general statement—that the SBIR could not be the only strategy for invigorating the procurement system—but he reaffirmed his view that SBIR companies can provide significant competitive pressures against large companies. He recalled the case of IBM, which misread the competitive pressure of Microsoft’s operating system, and reaffirmed the importance of creative transformation and the role of small businesses in bringing it about.