Measuring Research and Development Expenditures in the U.S. Economy Interim Report (2004) / Chapter Skim
Currently Skimming:
Interim Report
Interim Report
Pages 4-21
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 4... ...
the Survey of Federal Funds for Research and Development; the Survey of Federal Science and Engineering Support to Universities, Colleges, and Nonprofit Institutions; 3. the Survey of Research and Development Expenditures at Universities and Colleges; 4.
|
From page 5... ...
continue recent efforts to provide NSF staff with professional development opportunities to improve statistical skills; and (3) continue to develop and strengthen a program of methodological research, undertaking rigorous analysis of the data collected to assess the quality of the data relative to concepts they are supposed to measure.
|
From page 6... ...
Three of the research items were clesignect to address OMB clearance conditions: record-keeping and administrative records practices; effects of mandatory versus voluntary reporting on the state items; and web-basect survey operations. Other items on the list of nine key activities inclucle research on data collection below the company level; cognitive research on survey forms and instructions anal identification of appropriate respondents; survey sample clesign issues; survey processing Data editing, cleaning, and imputation)
|
From page 7... ...
in OMB budget documents. The Survey of Research and Development Expenditures at Universities and Colleges has coverage of the largest academic institutions in terms of defined R&D activity, and the Survey of Science and Engineering Research Facilities utilizes the same institutional list as the R&D expenditure survey and an additional list of biomedical institutions maintained by the National Institutes of Health (NIH)
|
From page 8... ...
Such a stucly needs to be clevelopecl and conducted in SRS. A comprehensive feasibility study for an effective web-based instrument requires both use of interactive contact with individual respondents, as recommencled in this report, and a carefully designed field pilot stiffly.
|
From page 9... ...
Holmberg's experimental study of preprinting values on a self-administered questionnaire for business establishments by Statistics Sweden found that questionnaires with preprinted historical values outperformed questionnaires without preprinted values on several measures of (lata quality. These data quality improvements included (not surprisingly)
|
From page 10... ...
Thus there is little unit or item nonresponse in this family of surveys, although, as mentioned earlier, some of the methods of achieving these response rates have a questionable overall impact on data quality, and their impact needs to be investigated in an evaluation study. The Survey of Inclustrial Research anc!
|
From page 11... ...
The basic sampling frame is the Business Register, previously known as the Standard Statistical Establishment List. This list, used since 1976, has problems in coverage and currency that will be examined in more detail in the panel' s final report.
|
From page 12... ...
The data collection procedures employed by the Census Bureau for the industry survey stand in stark contrast to the more technologically advanced procedures employed in the smaller federal and academic surveys, which, for the most part, are web-based and have more intensive education, response control, and follow-up schemes. The printed questionnaire is in dire need of review and substantial revision.
|
From page 13... ...
Instead, as mentioned earlier, the Census Bureau publishes imputation rates, which can be quite large and serve as a poor proxy for item nonresponse rates. NSF could make a significant contribution to unclerstancling the quality of the data by ensuring clear definitions and regular reporting of item nonresponse rates.
|
From page 14... ...
little error, but the editing process was replete with potential for error. That potential starts with the observation that there is no written description of the editing process, including the process in which an analyst supplies data codes.
|
From page 15... ...
The response rate for the census is an enviable 100 percent, and there is no item nonresponse since the agency respondents must answer all questions before the data can be submitted. Lack of timeliness is a continuing issue.
|
From page 16... ...
SURVEY OF FEDERAL SCIENCE AND ENGINEERING SUPPORT TO UNIVERSITIES, COLLEGES, AND NONPROFIT INSTITUTIONS The Survey of Federal Science and Engineering Support to Universities, Colleges, and Nonprofit Institutions is congressionally mandated ant! is the only source of comprehensive data on federal science and engineering support to individual academic anal nonprofit institutions.
|
From page 17... ...
As was the practice with the ORC Macro surveys, all missing data items, including those for nonrespondents, were imputed. No item nonresponse rates were reported, so it was not possible Report-14
|
From page 18... ...
not provide information at all, as well as for item nonresponse. The imputation factors were generated by class of institution and derived from responding institutions for three key variables: total R&D expenditures, federally financed R&D expenditures, and total research equipment expenditures.
|
From page 19... ...
The differences in response rates between public and private institutions is of concern, with smaller rates for the private institutions perhaps the result of traditions and maybe the cause of larger error in their estimates. Some of these issues may be resolved in the current collection of ciata for 2003, when NSF will publicize data by institution, with only a few sensitive data items suppressed.
|
From page 20... ...
Such issues as the willingness of respondents to respond to otherwise voluntary items and the quality of the responses should be examined by NSF through matching the 2002 survey responses with those of previous and subsequent years, as well as through concentrated response analysis surveys of the reporters for whom response rates appear to be affected by the mandatory requirements. Resurrection of a Response Analysis Survey Program The program of visits by NSF and the Census Bureau to key RD- I reporters, which was cancelled several years ago due to funding shortfalls, needs to be reinstated in a formal way.
|
From page 21... ...
Cognitive Research into Survey and Questionnaire Design Issues This research agenda pertains to all of the NSF surveys, particularly those collected in a web-based mocle. NSF would be well advised to continue to obtain expert advice to develop a multiyear research and implementation agenda for improving questionnaire content, layout, and processes.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.