Each of the survey instruments will have a stated purpose and each will be “mapable” to the objectives of the study to which they relate.62 All surveys will be pre-tested. These surveys are discussed in more detail below.
The program manager survey will focus on strategic management issues and on manager views of the program. It will be designed to capture senior agency views on the operations of the SBIR program focused on concerns such as funding amounts and flexibility, outreach, topic development, top-level agency support for SBIR, and evaluation strategies.
The survey may be administered through face-to-face interviews with senior managers, by telephone, by mail, via electronic questionnaire, or through some combination or these approaches. All senior program managers at the agency and all program managers at the sub-unit level (e.g., NIH institutes, DoD agencies) are to be covered. Altogether, there are approximately 45 program managers at this level in the five study agencies.
While program managers should have a strategic view of the SBIR program at their agency, the program is to a considerable extent operated by other managers. The responsibilities of these technical managers (or TMs) are focused on the development of appropriate topics, appointment of selection panels, process management (e.g., ensuring that reviews are received on time and that the selection and management process meets approved timelines), and contacts with the grant recipients themselves.
The Committee plans to conduct informal interviews with selected TMs. In addition, a survey instrument is currently being designed which will be sent to each TM in each agency. This instrument will address technical management issues, and will focus on the relationship between SBIR projects and non-SBIR components of each agency’s research and development program. TMs, for example, may play a pivotal role in the subsequent take-up of SBIR-funded research within DoD, and the survey is aimed at enhancing assessment of that possibility.
The survey will therefore be delivered to all TMs in the five agencies. Approximately 200-300 potential survey recipients are anticipated.
In order to identify characteristics of firms and projects that received SBIR Phase I awards only, the Committee anticipates the implementation of a survey of SBIR Phase I recipients. The objective of this survey is to enhance understanding about project outcomes, and to identify possible weaknesses in the SBIR Phase I—Phase II transition that may have excluded worthy projects from SBIR Phase II funding. (It should be understood that the Committee has no preconceptions on this issue—only that this is an important transition point and winnowing mechanism in SBIR, and should therefore be reviewed.)
As there have been more than 40,000 SBIR Phase I grants made, it is not feasible to cover all SBIR Phase I winners. Therefore, the Committee will developed an initial set of selection criteria, aimed at ensuring that outcomes are assessed for a range of potential independent variables. These will include:
Size of firm
Women and minority ownership
Multiple vs. single award winners
The SBIR Phase II recipient survey will be a central component of the research methodology. It will address commercial outcomes, process issues, and post-SBIR concerns about subsequent support for successful companies. Surveys must provide data that will allow the Committee to address the various questions defined in sections 3 and 4. Specifically, survey methodologies will need to differentiate between:
Funded and unfunded applications