addition, he noted that some small firms had proven more reliable than large firms. More broadly, he said, the process benefited by “adding people to the landscape” and producing more options for the services. Moreover, a lot more innovation had occurred at the big firms, he said, because they knew they were being measured against organizations other than their peers.
Dr. Gansler added that a lot of SBIR activity occurred not at the weapons system level, but at the sub-system, component, or even the materials levels. For example, SBIR contributions often brought performance improvements at lower cost. This, he said, is “the way the real world, the commercial world, actually operates,” and that government should try to follow the lead of the computer industry, which produces new models at lower cost every two or three years.
Another participant said that dollar amounts and percentages were less valuable as metrics than the results of the Phase III contracts as requested every year by the Small Business Administration (SBA). “That,” he said, “is the easiest and cleanest metric” to indicate success, and it would be inexpensive. The SBA metrics would not provide a complete picture, he said, but they could show trends. In the same way, the achievements of the prime contractors could be measured. Success stories were not helpful, he suggested; every large program should have many success stories; and there were no clear metrics by which to measure innovation. The program should focus simply on measuring how much additional money was going to SBIR firms. “That was the intent of Congress,” he concluded, “and an easy way to measure it.”
Larry James of the Department of Energy urged caution about using the simple metric of Phase III dollars, which might drive the program further toward the development side. The “Innovation” in SBIR would eventually be lost. The program will gravitate toward low risk, incremental development projects. The program’s major benefit to the nation is that Small Business Concerns are willing and able to take on high-risk, innovative research that provides disruptive technological advances in the marketplace.