nonresponse and to direct the computation of response rates for each item, prior to sample weighting.
This large and important survey was the focus of much of the panel’s attention. We looked at design, data collection, estimation, and processing issues. To assist in controlling the possibility of editing error, the panel recommends that the editing system be redesigned so that the current problems of undocumented analyst judgment and other sources of potential error can be better understood.
We also considered the vexing problem of estimation of state R&D expenditures. A new composite estimator has recently been developed by NSF and the Census Bureau. The panel commends NSF and the Census Bureau for developing this composite estimator, which takes into account research on small-area estimation. However, we recommend that additional simulations be conducted to assess the bias, variance, and mean square error of these new state estimates.
The panel highlights the long-term issue of timeliness of the estimates for these surveys, which have high response rates and do not pose serious issues in terms of statistical methodology.
The academic and biomedical facility surveys rely heavily on a knowledgeable point of contact to collect information, which generally must be obtained by a wide data collection in these institutions. The panel urges NSF to contact a sample of respondents to check their records in order to improve understanding of the best means of gathering the data, the sophistication of reporting sources within an institution, and the interpretation of questions and definitions. Furthermore, concerned about the lack of knowledge about the response patterns, the panel recommends study of the cognitive aspects of collection instruments and reporting procedures.
In discussing recent promising development and testing of improvements in imputation procedures using a regression model, the panel is concerned that the tests were not sufficient to judge the soundness of the regression approach. The research should be redone utilizing a more standard procedure of withholding a set of independent data in order to test the model.
Nonresponse and imputation in the facilities survey were found by the panel to be troublesome. The panel recommends that NSF carefully review item nonresponse