patterns, conduct a response analysis survey to determine the base quality of these new and difficult items, and make a commitment to a sustained program of research and development on these conceptual matters.

The panel determined that imputation procedures in this survey vary by whether or not the institution previously reported. If the unit previously reported, prior responses were used in the imputation procedure; if not, other methods were employed. This procedure is of major concern to the panel. Imputation for unit nonresponse is highly unusual in surveys. In most surveys, unit nonresponse is handled by weighting, as it was in this survey in 1999. At a minimum, NSF should compare the results of imputation and weighting.

Although the focus of this interim report has been on issues of the reliability and accuracy of the statistical methodology, the panel recognizes that its study of these matters is incomplete without reference to several other sources of error and other shortcomings in the surveys. The issues of concept and definition, of timeliness, and of survey management have much to do with the overall quality of these surveys.

Likewise, the challenges posed by a changing environment for data collection, due to the growing prominence of “novel” forms of organizational arrangements for the conduct of research and development, need to be explored, as does the impact of the increasing globalization of R&D. Sectoral shifts in the focus of R&D and the influence of firm size also pose new challenges and opportunities for data collection. These issues and more will be addressed in the final report of the panel.