TABLE 1: OVERVIEW OF APPROACH TO SBIR PROGRAM ASSESSMENT

SBIR Assessment Parameters →

Quality of Research

Commercialization of SBIR Funded Research/Economic and non-Economic benefits

Small Business Innovation/Growth

Use of Small Businesses to Advance Agency Missions

Questions

How does the quality of SBIR funded research compare with that of other government funded R&D?

What is the overall economic impact of SBIR funded research? What fraction of that impact is attributable to SBIR funding?

How to broaden participation and replenish contractors? What is the link between SBIR and state/regional programs?

How to increase agency uptake while continuing to support high risk research

Measures

Peer review scores Publication counts Citation analysis

Sales; follow up funding; progress; IPO

Patent counts and other IP / employment growth, number of new technology firms

Agency procurement of products resulting from SBIR work

Tools°

Case Studies, Agency Program Studies, Study of Repeat Winners, Bibliometric Analysis

Phase II surveys, Program Manager Surveys, Case Studies, Study of Repeat Winners

Phase I and Phase II surveys, Case Studies, Study of Repeat Winners, Bibiometric Analysis

Program Manager Surveys, Case Studies, Agency Program Studies, Study of Repeat Winners

Key Research Challenges

Difficulty of measuring quality and of identifying proper reference group

Skew of returns; Significant interagency and inter-industry differences

Measures of actual success and failure at the project and firm level; Relationship of federal and state programs in this context

Major interagency differences in use of SBIR to meet agency missions

° Supplementary tools may be developed and used as needed.

Multiple Methodologies

Over the iterative development of the study’s methodology, it became clear that no single research methodology would suffice to assess a program as differentiated as SBIR—one with multiple objectives, distinctive agency missions, and varied participants (ranging from small start-ups to relatively large, well-established companies, with product cycles ranging from months to decades). Instead, a complement of methodological tools has been crafted to address different facets of the program’s operation.

These tools are firmly grounded in economics and, as noted, draw from the experience of successful approaches pioneered by previous NRC studies of SBIR. They will necessarily have to be implemented in a flexible manner, with additional approaches to be drafted as new research challenges emerge. This document is, in this sense, a working draft, reflecting the current state of the Research Team and Committee discussions. It represents the Committee’s considered understanding of the tasks at hand, and methodological tools that can be applied to address these tasks. The elaborated methodologies are, thus, not exclusive, precluding the adoption of other tools and approaches; nor are they definitive, representing a fixed and final statement. Instead, the document provides a summary of current thinking on the project, as it has evolved from the discussions of the Research Team and the Steering Committee, as well as other interested parties. Despite these necessary limitations, this document constitutes a clear statement of the research goals and the tools the Committee plans to use to address them.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement