has been made to university R&D since 1998 and was enabled by the addition of questions to the data collection form that break out how much of the R&D expenditures were passed through the institution to subrecipients, and how much were received by the institution as a subrecipient. These questions were added when NSF determined, as the result of an investigation, that at least $350 million of the $1.9 million difference in 1996 was due to double-counting (National Research Council, 2000). The effect of this correction was dramatic, although not necessarily consistent in direction (see Figure 7-1).

It is tempting to suggest that differences in definitions, time horizons, data collection methodology, and the effect of sampling and measurement errors simply invalidate a direct comparison of federal spending and performer expenditures. The estimates of federal funding for R&D in academia and industry, academic R&D expenditures, and industry R&D expenditures are independently derived from very different sources and cannot be expected to add up. However, it is essential to continue to peel away these potential sources of the discrepancy as a means of aiding in the interpretation of the NSF data, as well as a means of assisting in identifying sources of error in the estimates.

Thus, the panel’s recommendation is that a reconciliation of the estimates of federal outlays for R&D and performer expenditures be conducted by NSF on an annual basis (Recommendation 7.2). This reconciliation should be published and widely disseminated by NSF as an aid to data users and as a blueprint for modifying the structure and implementation of the data collections to improve their concordance over time. The recent decision on the part of NSF to attempt to reduce a source of noncomparatibility by collecting data on potential double-counting of university R&D subcontracting was in direct response to attention to the size of the discrepancy. This change in data collection indicates that shining the spotlight on these differences can lead to improvements in data collection that, in turn, will reduce the gap over time.