The National Science Foundation (NSF) Science and Technology Centers (STC) program was established in 1987 to “fund important basic research and education activities and to encourage technology transfer and innovative approaches to interdisciplinary problems” (NSF 1992). Two competitions, in federal fiscal years 1989 and 1991, led to the establishment of 25 STCs. The centers span a wide range of science and engineering fields and vary substantially in how they have contributed to the goals of the STC program.
The long-term, stable funding that is a hallmark of the STC mode of research support should in principle encourage greater risk-taking and long-term thinking while providing flexibility in setting research directions, initiating projects and creating facilities, and engendering innovative educational ideas. The existence of stable clusters of expertise should also enable the nurturing of technology-transfer pathways from many of the centers. At this point in the evolution of the centers, NSF determined that it was appropriate and practical to evaluate how well the STC mode of support has been operating and to recommend adjustments.
CHARGE TO PANEL
As part of this effort, NSF requested that the Committee on Science, Engineering, and Public Policy (COSEPUP) of the NAS, the NAE, and the IOM form a panel to evaluate the accomplishments of the NSF STC program (not individual centers) against its goals in research, education, and knowledge transfer. This report is the result of the work of the panel charged with that effort. In a separate effort, NSF also contracted with Abt Associates to gather data about the STC program.
NSF charged the COSEPUP panel to
Review and interpret the data gathered by the outside contractor.
Reach conclusions regarding the progress of the STC program toward its goals.
Make recommendations concerning NSF's future use of the STC mode of support.
COSEPUP then provided additional guidance to this panel regarding each element of the charge.
With respect to the first charge, the panel not only could consider the data from the contractor, but could consider any information from any other source within the budgetary and time constraints of the study. The panel could also comment on the strengths and limits of the Abt data.
For the second charge, the panel could comment on the STC program 's goals, especially as seen in today's emerging changes in the R&D landscape (academic, industrial, and government). The panel could also comment on the appropriateness of the STC mechanism for achieving each of those program goals; this could necessitate comparing the mechanism's strengths and weaknesses with those of the individual investigator and other modes of funding (although a thorough comparison was not intended). And the panel could comment generally on the balance of criteria used in making STC awards (such as scientific merit, relation to national and societal needs, geographic distribution, demographic considerations, and distribution between large and small universities).
As to the third charge, the panel could consider recommendations about the STC program balance—mix of topics, mix of basic versus applied research, interdisciplinary nature of the centers, extent of industry orientation, degree of educational emphasis, and so on —and about management of the STC program without focusing on particular centers (for example, how does NSF structure its oversight?). It could also suggest changes in the mechanism and criteria by which STC proposals are evaluated by NSF, the duration of STC awards, and the mechanism for continuing STC evaluation. Finally, if the panel felt in general that increased or decreased emphasis should be put on the STC program, it could say so; but specific recommendations for program rebalancing could be justified only by a cost-benefit analysis across NSF's entire R&D portfolio.
SOURCES OF INFORMATION
In responding to its charge, the panel was able to use information and guidance from several reports and background documents. These items included the following:
Science and Technology Centers: Principles and Guidelines (NAS 1987)
This report (referred to as the Zare report) by the National Academy of Sciences was written by a committee chaired by Richard Zare. The study was com-
missioned by NSF as a result of President Reagan's 1987 State of the Union message, in which he called for several initiatives to enhance the nation's economic competitiveness, including the establishment of STCs by federal research agencies. The Zare committee examined the role of NSF in the president's program, the relationship between STCs and other modes of NSF support, essential and desirable features of STCs, mechanisms and criteria for soliciting and selecting proposals to encourage the most-promising ideas, principles and methods of governance (including relationship of STCs to their parent universities, NSF, and their scientific constituencies) and concerns raised within the scientific community by the proposed expansion of the center mode of research. The report provides guidance as to the goals, features, and solicitation and selection criteria for the program, as well as NSF's management of the program and a discussion of the risks of the program.
NSF Science and Technology Research Centers: Program Solicitations (NSF 1988, 1989)
The solicitations describe the features of the STCs, what institutions could submit proposals, requirements for a center director, and selection criteria.
Assessment of the National Science Foundation's Engineering Research Centers Program (National Academy of Engineering 1989)
This report was prepared for NSF to evaluate the mission of the engineering research center (ERC) program. NSF staff drew on existing ERC documents and expertise to develop the proposal solicitation, the review process, the cooperative agreement, etc. for the STC program.
The NAE report concluded that the ERC program was at least as important to engineering schools and industries as when it had first been proposed some 5 years earlier but expressed concern about the adequacy of the ERC program's funding and some aspects of its management.
University-Industry Research Centers (Wesley Cohen, Richard Florida, and W. Richard Goe, Carnegie-Mellon University 1994)
Centers, specialized institutes, and laboratories at academic institutions are not a new idea, and they are widespread. NSF alone has 11 center programs, and many universities have a profusion of centers. For example, this Carnegie-Mellon University report focuses on university-industry research centers (UIRCs) that have been funded by the federal and state government. The national study examined the characteristics and activities of UIRCs, their role in technologic innovation and technology transfer, and the effect of industrial funding on the academic research mission.
Alternative Models of Research Performance (Irwin Feller for U.S. Congress Office of Technology Assessment 1992).
This study evaluates the effects of alternative models of federal support for
academic research and of the organization of academic research on research productivity. It evaluates questions related to the effects of teamwork and block-grant funding on creativity, including relative performance (e.g., research productivity and relevance) of the investigator-initiated model and industrial modes of research support.
National Science Foundation's Science and Technology Centers: Building an Interdisciplinary Research Program (National Academy of Public Administration 1995)
This study—requested by the Senate Subcommittee on Housing and Urban Development, Veterans Administration, and Independent Agencies—reviewed the management of the STC program. In conducting its study, NAPA selected five STCs, which were then reviewed by an advisory panel. The study focused on the degree to which the centers exhibited interdisciplinary collaboration, a university base, knowledge transfer to industry, and educational outreach; the value of the center concept of management; the management approaches taken by each center, and NSF's approach to management of the STC program. The study found that STCs are valuable contributions to NSF's research portfolio and to national research goals and are managed in an appropriate manner; that the five STCs are university-based, interdisciplinary, basic-research efforts and good examples for encouraging synergism. It recommended continued funding of the program, continuance of the matrix method of managing the program via the Office of Science and Technology Infrastructure and the responsible directorates, and formation by the National Science Board of a subcommittee to act as a monitoring body and to advise NSF in establishing processes and criteria for management of the program.
STC Visiting Committee Site-Visit Reports
Per NSF requirements, each center was visited several times by an NSF visiting committee of academic and industrial researchers who evaluated its scientific quality and administrative management. These visits included annual site visits during years 1-3 and in-depth reviews in years 3 and 6. Excerpts from these reports (with identifying information removed) are provided in Appendix D.
Presentation by STC Directors
At its first meeting, the panel heard from several directors of STCs. The directors described the attributes of STCs and the strengths and challenges of the program as a whole and their individual centers.
An Evaluation of the NSF Science and Technology Center Program (Abt 1996)
This volume, prepared in parallel with the present report, provides a historical context for the STC program, analysis (bibliometry, patents, etc.), and responses to questionnaires sent to STC directors, university officials, researchers, and graduate students. The key findings from this report are shown in Box 1-1.The panel, however, has some concerns about the method of analysis discussed in the report; these concerns are discussed in Appendix A and the preface. For these reasons, only selected data from this report have been used.
Review of Abt Report Findings on STC Program (Abt 1996)
Principal Research Goals, Achievements and Impacts
Bibliometric Analysis of Research Performance
Educational Goals, Achievements and Impacts
Training Support and Job Performance of Graduates of the STCs
Knowledge Transfer Goals, Achievements and Impacts
The present panel's report focuses on three key questions in response to its charge:
Has the STC program accomplished its goals?
How well has the STC program been managed and evaluated?
How well has the design of the STC program worked?
In Appendix A , the panel provides its analysis of the Abt report and comments on the separate contractor/Academy panel approach as a model for future NSF program evaluations. Appendix B lists the STCs. Appendix C provides biographic information on the panel members and staff. Appendix D includes excerpts from the visiting committee reports on the STCs. These reports were a major basis of the panel's recommendations.