The challenge raises issues that go well beyond the usual assessments of an individual agency or mission. Each agency already has in place processes for relating inputs to outcomes. There appear to be data that, although not routinely cast in this form, would permit each agency to evaluate the effectiveness and efficiency of its individual NNI investments. One example is the NSF website, detailing grants funded by the agency; another is the NIH Reporter, giving details of grants, albeit slightly different.2 The committee found that those computer-based assessment tools are not adequate for assessing the overall effectiveness of the NNI as a major national multiagency initiative. In particular, the kinds and formats of data collected by the participating agencies are neither mutually compatible nor readily shared among the agencies.
However, progress toward achieving the four NNI goals is currently reported by NNI in largely anecdotal form in the annual NNI supplements to the President’s budget. There, several agencies provide examples of successful projects; some provide numerical data, and some present short summaries without many details. Interagency activities are reported in the same manner. Clearly this makes it difficult to link what is reported to specific progress toward achieving each of the four goals.
The result is lost opportunities to evolve best practices; to measure the value added by interagency cooperation, planning, and collaboration; to identify and rectify programmatic gaps or redundancies; or to determine whether the levels of investment are adequate to meet the goals. The Nanoscale Science, Engineering, and Technology (NSET) Subcommittee of the National Science and Technology Council’s (NSTC’s) Committee on Technology and the National Nanotechnology Coordination Office (NNCO) could gather and aggregate such information from the agencies and bring the data and associated metrics to bear toward NNI goals in ways that are accessible to the various NNI stakeholders.
In this chapter, the committee identifies some of the shortcomings of the current processes and offers recommendations for improvement. It describes in general terms the role of data and metrics in assessment, identifies some aspects of particular relevance to the NNI, and then briefly reviews other studies of metrics for federal research and development (R&D) programs and suggestions for specific types of data and models appropriate for the NNI. It also discusses some new tools and methods that are becoming available owing to research in the field of metrics and assessment and concludes with a proposed implementation process.