NSET should request funding for societal implications activities and then award that funding directly to agencies willing to do this kind of work and capable of doing it.

The societal implications theme has three components: educational, outreach, and social science. However, unless things change dramatically during FY 2002, the social implications theme will simply be a fancy title for a relatively straightforward educational initiative targeted at graduate and undergraduate students. While not every agency may want to address all three components, they should all be required to budget for, or at least report about, these areas separately.

Agencies willing to engage in outreach or social impact studies should allocate funds directly to the office or division that typically engages in or supports these kinds of activities. These divisions could then pursue focused intramural studies or develop solicitations targeted at the appropriate social science community. These NNI agencies should also be encouraged to consider focusing on the topics and funding strategies highlighted in NSF and NSET-sponsored workshops on societal implications.

Recommendation 10: The committee recommends that NSET develop performance metrics to assess the effectiveness of the NNI in meeting its objectives and goals.

The committee sees a need to measure the progress of the NNI as a whole, under the aegis of the OSTP, with measurable factors including quality, relevance, productivity, resources, and progress in moving research concepts toward applications. To date, NNI programs have been evaluated as part of the Government Performance and Results Act (GPRA) procedures of the individual participating agencies.

Despite a long history of efforts to define and improve evaluation criteria, the academic, industrial, and government sectors continue to struggle with the problem of measuring the effectiveness of research activities. The challenge of evaluation is compounded in the case of the NNI, since the program spans multiple agencies with varying missions. However, once the participating agencies have agreed upon program goals, evaluation and exit criteria can be developed to appropriately measure effectiveness or success in achieving the goals. These criteria should be developed jointly by an appropriate council and with the various agencies under NSET. One possibility for such council could be the suggested NNAB.

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement