Measuring the 21st Century Science and Engineering Workforce Population Evolving Needs (2018) / Chapter Skim
Currently Skimming:
5 Data Collection and Processing
5 Data Collection and Processing
Pages 91-108
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 91... ...
The GSS collects data from approximately 700 academic institutions annually. Respondents are institutional coordinators who are asked to provide aggregate data on graduate students enrolled in science, engineering, or selected health field organizational units (such as academic departments and degree-granting programs)
|
From page 92... ...
NCSES's data collection contractor follows up by e-mail, postal letter, postcards, and telephone with doctoral degree recipients who do not respond. Although respondents can participate in the SED by Web, mail, or telephone, NCSES has successfully transitioned the survey from a primarily paper-based data collection (whereby students returned completed mail questionnaires to the institutional coordinators, which were then forwarded to the data collection contractor)
|
From page 93... ...
. In 2015, the overall NSCG response rate was about 70 percent, slightly lower than the 74 percent response rate in 2013.
|
From page 94... ...
Over the years, NCSES has continued to refine the incentive strategy for the SDR. The survey currently uses a prepaid incentive of $30, strategically targeted to reduce nonresponse bias depending on the sampling group and whether an incentive was needed for the sample member in previous rounds of data collection.
|
From page 95... ...
The remainder of this chapter focuses on aspects of the data collection and processing that would particularly benefit from changes or enhancements. DATA COLLECTION MODES NCSES has been actively researching the optimal use of different data collection modes for its science and engineering workforce surveys.
|
From page 96... ...
First, there is an intricate interdependence among the different modes of contact and different response options. Even if Web responses are the goal or the only response mode offered, contact through another mode, such as mail or telephone, often is necessary, and combining modes in this way can substantially boost response rates.
|
From page 97... ...
The use of multiple contact modes improves response rates because the survey reaches a larger proportion of the sample members and because using a variety of modes to communicate with sample members offers multiple opportunities to deliver persuasive messages. Although NSCG sample members are contacted initially by mail, they are asked to complete the survey by Web, and a mail questionnaire generally is not provided until later stages in the follow-up process.
|
From page 98... ...
granting research-based master's or doctoral degrees in science, engineering, and health fields (information collected from institutional coordinators) Survey of Earned Doctorates Recipients of research doctorate degrees in any field, (SED)
|
From page 99... ...
The four experimental conditions were as follows: (1) mail questionnaires in weeks 8 and 18 (control group)
|
From page 100... ...
However, there was a small difference in when people responded: the use of a paper questionnaire in week 8 rather than a Web invitation resulted in an incremental response rate increase of approximately 2 percentage points. While this research provides valuable insight into the potential consequences of removing the mail response option, further research is needed on the role of the paper questionnaire.
|
From page 101... ...
Some questions could simply use further refinement because they may be too technical or ask about circumstances with which many respondents may not be familiar. For example, the SED question on tuition remission asks respondents whether they received a "full or partial tuition remission (waiver)
|
From page 102... ...
Employer Name Department/Division City/Town State/Territory ZIP Code FIGURE 5-4 Principal employer question on the National Survey of College Graduates and Survey of Doctorate Recipients. SOURCE: National Survey of College Graduates Questionnaire.
|
From page 103... ...
Changes should be implemented with careful consideration of their impact on trend data. Topic Modules As discussed in Chapters 3 and 4, increased reliance on topic modules is a potential solution that could reconcile the need to provide fine-grained data on core questions with the need to gather detailed data periodically for an in-depth understanding of certain topics.
|
From page 104... ...
An additional consideration for the use of modules is that if the questionnaires became highly customized, resulting context effects could have an impact on the trend data available for the other items, and potentially the core items, on the surveys. Government surveys exist that use topic modules in a variety of different ways, and the experiences of other agencies could provide valuable insights to NCSES.
|
From page 105... ...
TABLE 5-2 National Survey of College Graduates Web Survey Response by Device, 2013 and 2015 Number of Mean Median 2013 NSCG Web Respondents Completion Time Completion Time Device Smartphone 766 35.8 32.1 Tablet 5,595 30.0 26.2 Desktop/Laptop 56,021 27.8 24.2 Number of Mean Median 2015 NSCG Web Respondents Completion Time Completion Time Device Smartphone 2,123 31.0 27.7 Tablet 4,644 28.3 25.4 Desktop/Laptop 52,676 25.8 22.5 SOURCE: Fiegener and Finamore (2016)
|
From page 106... ...
MAINTAINING PANEL PARTICIPATION AND RESPONSE RATES NCSES has a number of completed and ongoing research projects focused on maintaining participation and response rates. In recent years, the agency also has begun implementing adaptive design, which enables researchers to tailor data collection decisions in ways that optimize costs and data quality.
|
From page 107... ...
RECOMMENDATION 5-5: The National Center for Science and Engineering Statistics should continue research aimed at expanding the use of adaptive design in the National Survey of College Graduates and Survey of Doctorate Recipients. Experiments involving contact schedules, incentives, and modes should focus on the goals of increas ing participation, reducing nonresponse bias, and controlling costs.
|
From page 108... ...
Given the increase in Web responses and continuous improvements in data cleaning and data entry procedures in general, it may be worthwhile to evaluate the data processing procedures for the science and engineering workforce surveys to assess (1) whether all types of errors that are found and corrected are sufficient in number to warrant the time and expense, and (2)
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.