is recognized that research quality is difficult to measure and that the value of knowledge created is difficult to predict, takes time to be realized, and is highly variable among projects.

Attention to Knowledge Dissemination and Spillover Effects

Technology development programs, in contrast to basic science programs, generally view knowledge gains as a means to a desired end, not the end point itself. Knowledge gains are seen as capacity building, as providing answers to questions impeding innovation, and in some cases as a means of broadening the scope of program benefits beyond those accruing directly to funding recipients and their customers. In the longer run, the social impacts of technology development programs come to reflect both the direct effects and the spillover effects of public R&D investment.1

Agencies that emphasize the value of research to accomplish their own mission-driven goals, such as the Department of Defense (DoD), tend to place less emphasis on measures of knowledge dissemination as an important program output. Programs that emphasize the generation of broad-based economic benefits, such as the Advanced Technology Program (ATP), emphasize both the value of their programs achieved through knowledge dissemination to others and the direct economic impact achieved by the innovator.

The NSF’s SBIR program would appear at first glance to be close to the ATP in emphasizing knowledge dissemination and spillovers, in that its second major merit review criterion is “broader impacts.” However, the details of the NSF’s broader-impacts criterion appear to put less emphasis on broadening economic benefits of the proposed activity to society than ATP’s. More important, in practice the NSF’s SBIR program has largely ignored potential spillover benefits from knowledge dissemination. “Broader impacts” appears to be defined by the program largely as commercial results.2 Patenting was the only knowledge-related measure found in NSF evaluation studies of its SBIR program,3 and it appears that patent data were collected to signal commercial activity.4 No effective pro-


In contrast, basic research programs view knowledge creation as their primary goal. For these programs, most of the agency’s output/outcome/impact measures will likely focus on knowledge creation and dissemination.


NRC Program Manager Survey and discussions with program managers.


It should be acknowledged that the required interim, final, and post-grant annual commercialization reports (see Appendix F) each include a request for the reporting, if applicable, of publications, including the reporting of “scientific articles or papers appearing in scientific, technical, or professional journals … and any publication that will be published in a proceedings of a scientific society, a conference, or the like.” However, no program reporting or use of this information on knowledge creation and dissemination has thus far been discovered in either program management or performance metrics. And the requirement for postgrant annual reports has recently been dropped.


Patents preserve ownership rights to innovations that may be critical to being able to exploit them commercially.

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement