As scientific research becomes more interdisciplinary and complex, scientists and engineers are challenged to describe the limits of their own intellectual activity. It seems likely that the definitions required by benchmarking exercises can help to illuminate criteria for defining fields and subfields. In the case of superconductivity mentioned above, American scientists who were already working in adjacent subspecialties were able to move quickly into superconductivity research, and this could offer a definition of the subfields within a field along functional lines; that is, a field might be defined as the array of related domains among which investigators can move without leaving the realm of their expertise.
COSEPUP evaluated the quality of panel results independently and via comments from the oversight groups. In addition, the feasibility and utility of benchmarking were assessed during meetings with disciplinary societies and at a full-day workshop attended by representatives of federal agencies, universities, Congress, and the executive branch. The summary of that workshop in appendix C provided valuable information for COSEPUP members.
Of particular importance were the contributions of reviewers, whose array of expertise was comparable with that of the panel. The reviewers, chosen to represent diverse industrial and academic backgrounds, provided invaluable commentary and criticism for use by the panels and a means of validating the panels' findings.
Among the topics proposed for further discussion were the use of benchmarking during the budgetary process and whether benchmarking would be useful in helping to set national science policy. In addition, federal-agency representatives were asked about their own procedures for evaluating research and whether benchmarking might have a role to play in those procedures. Finally, the relevance of benchmarking to GPRA was discussed.