4
Science and Engineering Human Resources

Graduate Education of Scientists and Engineers

Science and engineering graduate education in the United States is widely regarded as the leading system for advanced training in the world (Bowen and Rudenstine 1992, NRC 1995, AAU 1995). Several factors contribute to its quality and effectiveness. The decentralized nature of graduate education in the United States is one of its defining characteristics. More than 600 institutions provide graduate education at the masters or doctorate-level in these fields, and within these institutions a range of departments and their faculty oversees the training of graduate students. The integration of graduate training with scientific and technological research is another hallmark of graduate education in the United States, especially at the doctoral level. Doctoral candidates in science and engineering typically work on advanced research projects as research assistants and also complete a research-based dissertation. Federal support of graduate students, particularly through their participation in research activities, is a third prominent feature of advanced science and engineering education. "A major objective of the federal/university partnership in research and education historically," the National Science Board (NSB) has written, "has been to attract high-ability youth into science and engineering careers by providing significant multi-year financial support that is competitively allocated and based on the student's past achievement and future promise" (NSB 1997). Much of the federal funding for graduate students is a by-product of federal support for academic science and engineering research, but it nevertheless remains substantial and has served as the vehicle for integrating graduate education and research in science and engineering. As seen in Table 4-1, the federal government was the primary source of support for 20 percent of graduate students in 1997. The federal government provided the primary source of support for almost half of the graduate students with research assistantships and for more than half of those with traineeships. Other important sources of support were institutional support for teaching assistantships and self-support (own, spouse, or family resources) (NSF 1999e, NSF 1999l).



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 59
Measuring the Science and Engineering Enterprise: Priorities for the Division of Science Resources Studies 4 Science and Engineering Human Resources Graduate Education of Scientists and Engineers Science and engineering graduate education in the United States is widely regarded as the leading system for advanced training in the world (Bowen and Rudenstine 1992, NRC 1995, AAU 1995). Several factors contribute to its quality and effectiveness. The decentralized nature of graduate education in the United States is one of its defining characteristics. More than 600 institutions provide graduate education at the masters or doctorate-level in these fields, and within these institutions a range of departments and their faculty oversees the training of graduate students. The integration of graduate training with scientific and technological research is another hallmark of graduate education in the United States, especially at the doctoral level. Doctoral candidates in science and engineering typically work on advanced research projects as research assistants and also complete a research-based dissertation. Federal support of graduate students, particularly through their participation in research activities, is a third prominent feature of advanced science and engineering education. "A major objective of the federal/university partnership in research and education historically," the National Science Board (NSB) has written, "has been to attract high-ability youth into science and engineering careers by providing significant multi-year financial support that is competitively allocated and based on the student's past achievement and future promise" (NSB 1997). Much of the federal funding for graduate students is a by-product of federal support for academic science and engineering research, but it nevertheless remains substantial and has served as the vehicle for integrating graduate education and research in science and engineering. As seen in Table 4-1, the federal government was the primary source of support for 20 percent of graduate students in 1997. The federal government provided the primary source of support for almost half of the graduate students with research assistantships and for more than half of those with traineeships. Other important sources of support were institutional support for teaching assistantships and self-support (own, spouse, or family resources) (NSF 1999e, NSF 1999l).

OCR for page 59
Measuring the Science and Engineering Enterprise: Priorities for the Division of Science Resources Studies Table 4-1 Full-Time Graduate Students in science and Engineering Fields, by Mechanism and Source of Financial Support, 1997 Source of Support Fellowships Traineeships Research Assistant-ships Teaching Assistant-ships Other All Federal 6,255 5,155 41,370 776 3,133 56,689 Institutional 15,233 3,877 29,311 60,693 10,163 119,277 Other 4,968 635 12,526 583 4,480 23,192 Self-Support 0 0 0 0 81,454 81,454 All 26,456 9,667 83,207 62,052 99,230 280,612   Source: NSF/SRS, Survey of Graduate Students and Postdoctorates in Science and Engineering (1999l). Continuing excellence in graduate education and the preparation of our nation's scientists and engineers for research remains an important objective for universities and the federal government, but the context for graduate education and the nature of science and engineering careers have changed in recent years. A series of reports has raised questions about the role of the federal government in providing financial support for graduate students, the progress of students through graduate school, the content of graduate education, and the preparation of graduate students for science and engineering careers. These reports have also raised questions about the ability of existing data to address these questions. Reports on Graduate Education In 1995, the National Academies' Committee on Science, Engineering, and Public Policy (COSEPUP) published Reshaping the Graduate Education of Scientists and Engineers. Citing changes in the job market for Ph.D.s, COSEPUP called for changes in graduate education and the way graduate students are supported to better position graduates for the job market and their future careers (NAS 1995). In the wake of the COSEPUP report, the National Science Board also examined graduate and postdoctoral education in the context of the federal/university research partnership. The NSB recommended that mission agencies recognize "the intimate connection between research and graduate education in universities" and adopt practices to ensure that their funding "reaps the dual benefits of simultaneously advancing both research and graduate education." The Board also recommended that federal agencies "recognize and reward institutions that, in addition to the core Ph.D. education, provide a range of educational and training options to graduate students . . .tailored to the career interests of the individual Ph.D. candidate" (NSB 1997). In 1998, the Association of American Universities (AAU) released a report on graduate education that noted the common criticisms of graduate education in the 1990s: overproduction of Ph.D.s; narrow training; emphasis on research over teaching; use of students to meet institutional needs at the expense of sound education; and insufficient mentoring, career advising, and job placement assistance. However, this report reached different conclusions about the nature and extent of these problems, and in the end, different recommendations (AAU 1998).

OCR for page 59
Measuring the Science and Engineering Enterprise: Priorities for the Division of Science Resources Studies While they differed in diagnosis and recommendations, each of the NSB, COSEPUP, and AAU reports recommended collecting additional data on, and conducting additional analyses of, graduate education and the careers of scientists and engineers. The NSB report asked for "improved policy data to assess the effectiveness of current Federal support for graduate education including attention to attrition and time-to-degree." The NSB also asked for improved policy data "to identify current and emerging national needs for the science and engineering workforce" (NSB 1997). COSEPUP asked the National Science Foundation to collect improved data on "time to employment" (including unemployment and underemployment) of recent Ph.D.s and on the academic and nonacademic careers of Ph.D.s (NAS 1995). The AAU recommended that universities improve tracking of their graduates for up to five years in order to generate better information on Ph.D. placement and employment (AAU 1998). Obtaining Improved Data on Graduate Education SRS is responsible for collecting and acquiring key data relating to the policy issues in graduate education and the labor market for scientists and engineers that the NSB, COSEPUP, and the AAU have identified. To improve the data it provides policymakers and others on graduate education, SRS needs to explore ways to collect additional data in a cost-effective manner to fill several specific gaps in information that policymakers need. SRS may collect data on graduate students, their educational experiences, and their financial support at three points in time: during graduate school, at the completion of a graduate degree, and retrospectively through personnel surveys. Currently, SRS administers a Survey of Graduate Students and Postdoctorates in Science and Engineering (GSPSE) that collects data on current graduate students and postdoctorates through their institutions. SRS collects data on the universe of new Ph.D.s in science and engineering through the Survey of Earned Doctorates at the time the Ph.D. is received. SRS has also recently collected limited retrospective data on the graduate school experience through a onetime module added to the 1997 Survey of Doctorate Recipients. To expand the range of data collected on graduate education, SRS is exploring the potential for fielding a longitudinal survey of beginning graduate students. This survey would be designed to collect data on a panel of students from matriculation through degree attainment or attrition in order to fill gaps in the data on graduate education such as those identified above. Questions that solicit data on the following could potentially be considered for such a survey: Financial support--longitudinal tracking of types and level of support by year Graduate school experience—decision to enter graduate school and career expectations; reasons for attrition (if applicable), and relationship with mentor and department Time to degree--data on educational milestones Graduate school training—responses to the depth and breadth of knowledge gained The overall goal of a survey of graduate students would be to inform decisions made by the NSB and others concerned with graduate education, and in particular, how types of financial support affect outcomes. It would also provide data to graduate students and mentors on the graduate school experience. SRS should carefully examine the potential for developing and administering such a longitudinal survey of beginning graduate students, yet it should defer

OCR for page 59
Measuring the Science and Engineering Enterprise: Priorities for the Division of Science Resources Studies implementation of such a survey until additional information on graduate education is gathered and analyzed. Based on what we currently understand, we question whether such a survey would be able to obtain useful data in a cost-effective manner. Financial support packages that graduate students receive during their course of study may be difficult to define because students do not always know the source of their financial support and administrators rarely have a complete picture of student financial support. Even if respondents were able to provide valid responses to questions on this subject, it is not clear that the data would be useful unless also linked to data on both the academic standing of students at matriculation, e.g., grade-point averages and Graduate Record Examination (GRE) Scores and later career outcomes. Having all of these data would greatly improve our understanding of graduate education, but we question whether the benefit of collecting this information longitudinally for a large enough sample would be worth the very large expense it would require. We recommend that in exploring whether to develop a survey in this area SRS should assess whether quality data could be obtained and at what cost by taking these steps: thoroughly review the literature on the research undertaken on the graduate school experience, particularly on financing graduate education, and determine which issues have not been resolved address why these issues have not been resolved delineate the data that are necessary to address these issues establish whether these data can be obtained and, if so, from whom determine the costs associated with obtaining the data that could be collected. If SRS determines that a survey of graduate students could be successful and cost-effective, it should be longitudinal in nature, but only if NSF intends to support and utilize it as such. Longitudinal surveys are very rich data sources. Also, questions asked over time in a longitudinal format will have only minimal response errors associated with memory. For example, the same person may respond to the query "Why did you go to grad school" very differently at time of matriculation as opposed to retrospectively at time of exit. However, SRS has done very little to exploit the longitudinal nature of the personnel surveys they already administer. Financial Support for Graduate Students The National Science Board asked in 1997 for an exploration of "improved policy data to assess the effectiveness of current Federal support for graduate education" (NSB 1997). The Survey of Graduate Students and Postdoctorates in Science and Engineering (GSPSE) and the Survey of Earned Doctorates (SED) provide data on the financial support of graduate students, but neither provides data that present a sufficiently complete picture of how students are supported over the course of graduate school. GSPSE provides institutional counts of students by financial support category, but does not track individual students. The SED asks respondents for type of support (fellowships, traineeship, teaching or research assistantship, etc.), but its data do not provide reliable information about the source of support (university, federal agency, foundation, etc.). Students have trouble identifying the original source of their funding, especially when federal funds pass through their institution (NRC 1994). The SED also asks for primary and secondary type of support, but the terms "primary" and "secondary" are insufficiently described to be clear as to whether respondents are thinking "largest" or ''most important" when they answer this question. Moreover, SED data do not provide a picture of financial support for an individual by year (or smaller unit of time) over the course of graduate school. It only asks respondents to indicate what kinds of

OCR for page 59
Measuring the Science and Engineering Enterprise: Priorities for the Division of Science Resources Studies financial support they had during graduate school in the aggregate. Finally, SED data are available only for those individuals who complete their course of study. The ability to relate different types of support—especially "packages" of support over the time spent in graduate school—to the graduate school experience, graduate training, and short- and long-term career outcomes, would provide policymakers with means for evaluating effects of types of support (research assistantships, fellowships, and traineeships) that federal agencies, such as NSF, offer to students. To best analyze the efficacy of certain types of support provided to students, policymakers could utilize data that show how students were supported at different points as they proceeded through graduate school and how these patterns of funding affected students' education, progress, degree completion, and job outcomes. They would also need to have an indicator of student potential in graduate school, such as undergraduate grade point average (GPA), selectivity of bachelor's institution, or GRE scores, and indicators of career outcomes so that student potential, graduate financial support, and career outcomes could be analyzed as a package. If it were possible and cost-effective to obtain data on student potential and graduate school through a longitudinal survey of graduate students and link it to career data, this would provide the most complete picture of how financial support is packaged and how it relates to graduate school and career outcomes. SRS should investigate whether it is possible and cost-effective to collect and link such data in that way. SRS should also consider how it might augment data on the sources of graduate student support by linking SED data to data available on federal financial support of graduate students. This would include federally-funded fellowships and traineeships, and potentially, research assistantships. If principal investigators reported data on graduate students supported by federal grants, SED data could be linked to these data, too. Completion of Graduate School Policymakers concerned about graduate education and the efficacy of certain types of funding have been seeking to know more about the factors affecting completion of graduate school. An internal NSF task force examining data availability for the analysis of graduate education argued "the proportion of graduate students who complete the curriculum they undertake is a very important intermediate outcome of federal support for graduate education" (NSF 1996). A recent NRC report on graduate school completion and attrition, The Path to the Ph.D., also argued that while some attrition from graduate school is expected, proper interventions could encourage more students who do not complete to do so, and make contributions to our universities, science and medicine, industrial development, and society (NRC 1996a). Both the internal NSF report and the NRC report on attrition conclude that no national data exist that measure the completion rates of graduate students in all disciplines from the variety of institutions awarding doctoral degrees. More complete data on the graduate school experience would allow better understanding of attrition and completion and of the interventions by government agencies or others that might best prepare Ph.D. students for their careers. Such data would include the decision to go to graduate school, career expectations, paths through graduate school to degree completion, financial support, reasons for attrition (if applicable), and the role of mentor, department, and career advising in completion/attrition and job placement. Data on these decisions, expectations, and relationships could potentially illuminate the graduate school experience for those grappling with policy issues in this area.

OCR for page 59
Measuring the Science and Engineering Enterprise: Priorities for the Division of Science Resources Studies Completion and attrition data for racial and ethnic minorities would be particularly useful. A substantial amount of demographic data on gender, race/ethnicity, and disability status already exist and are used by SRS in a number of publications; other kinds of data— such as those on graduate completion or attrition—could provide insights on how to improve graduate school and career outcomes for minority groups. Data on minority attrition could inform policies and programs designed to retain students from minority groups in academic programs. A survey that longitudinally tracks students through graduate school could potentially collect much of this data on completion and attrition at the national level though it would be expensive, as discussed above. It will also take additional time before data may be used in a longitudinal manner. Questions posed in such a survey could be fielded to students at the beginning of graduate studies, at formal entrance into a doctoral program, at the completion of all requirements but the dissertation, and at degree receipt. Retrospective questions about career expectations and overall satisfaction with the respondent's Ph.D. program were added to a one-time module for recent Ph.D.s in the 1997 SDR. Analysis of these data may shed light on the kinds of additional data required in this area, and whether such data might be collected through a survey of graduate students or the SED. A related issue in the progress of Ph.D. students through graduate school across disciplines is time to degree, which increased steadily over the last three decades before stabilizing in recent years (NRC 1996b). This issue is important both to graduate students, because of the opportunity costs associated with time spent in graduate school, and to graduate institutions, because of the costs required for supporting students over longer periods of time. Observers of graduate education have argued that lengthening time to degree has a variety of causes including the need for longer training in the face of growing knowledge across fields. Some also argue that time to degree has been driven up by students remaining in graduate school to avoid a tough job market or by departments that benefit from the teaching and research duties carried out by advanced graduate students at low cost. Some have shown that the increase can be attributed to a "cohort effect" that occurs when entering classes are smaller than previous classes. This phenomenon logically results in more graduates coming from earlier cohorts. More data on the progress of students through graduate school might better characterize this phenomenon. Again, such data could be obtained through a longitudinal survey of beginning graduates students if that were cost-effective. Data on progress toward the doctoral degree could also be readily collected through the Survey of Earned Doctorates. For example, SRS could ask SED respondents for the date when they completed all requirements for the doctoral degree except for the dissertation. This could be used to disaggregate time to degree into predissertation and dissertation phases. Graduate Education and Career Skills COSEPUP and the NSB have both argued that in a changing job market for Ph.D. scientists and engineers, where jobs and workplaces are more diverse than ever, graduate education needs to be reshaped to better prepare students for their future careers. COSEPUP noted that "more than half of new graduates with Ph.D.s—and much more than half in some fields, such as chemistry and engineering—now find work in nonacademic settings." It argued, therefore, that the narrow research focus of some graduate programs provided intellectual depth, but that it needed to be balanced by other skills required in the workplace (NAS 1995). COSEPUP recommended that the graduate education enterprise, particularly departments,

OCR for page 59
Measuring the Science and Engineering Enterprise: Priorities for the Division of Science Resources Studies implement reforms in the education of students in science and engineering who will work in either academic or nonacademic sectors. The NSB subsequently also argued that federal agencies should reward institutions that provide a range of educational options to Ph.D. students, including opportunities for interdisciplinary research and for learning teamwork, business management skills, and information technologies (NSB 1997). SRS took steps to provide data on skills obtained in graduate school by adding a question to a one-time module on the 1997 Survey of Doctorate Recipients about the adequacy of training that respondents were provided by their doctoral programs in a range of knowledge and skill areas. Included were subject matter, general problem solving, oral communication, writing, quantitative skills, computers, teaching, teamwork, research integrity/ethics, networking, and management. We recommend that SRS utilize the data it has collected through the 1997 SDR to examine this issue to the extent possible. We do not, however, recommend that SRS commit additional resources to collecting data in this area. The Ph.D. is and should remain a research degree. In our experience, graduate students are best served by developing subject matter expertise and having a meaningful research experience. There are also negative consequences for adding activities to promote "skills," such as lengthening time to degree. Also, many graduate students already improve communication skills through teaching assistantships and learn teamwork through research assistantships. We are skeptical that meaningful information can be collected on skills obtained in graduate school, or that data collected on skills obtained are likely to suggest how graduate programs might be reformed. While we do not recommend that SRS commit additional resources to collecting data on "skills," SRS might consider obtaining data on other factors or indicators related to graduate school and career outcomes. The possible collection of data on the decision to go to graduate school, career expectations, paths to degree completion, financial support, reasons for attrition (if applicable), and the role of mentor, department, and career advising in completion/attrition, and job placement have already been mentioned. Researchers would also like to be able to link student scores on the GRE to data from the Survey of Earned Doctorates and the Survey of Doctorate Recipients to examine further the predictive power of the GRE with regard to career outcomes. Issues in the Science and Engineering Labor Market Even before the difficulties in the labor market in the 1990s raised questions about the transition to employment of new and recent Ph.D.s, the 1989 NRC report Surveying the Nation's Scientists and Engineers argued that NSF needed to improve the data it provides on career paths and work of scientists and engineers. For example, the report recommended the collection of additional data describing key career transitions of scientists and engineers, such as entry into the labor force and mobility across fields and sectors. The report also recommended that NSF pursue the development of estimates for immigration to and emigration from the United States of scientists and engineers and that NSF include these estimates in its personnel data system (NRC 1989). Despite the substantial technical work SRS conducted in developing a new personnel data system for the 1990s, calls for additional data on careers have continued. For example, in Reshaping the Graduate Education of Scientists and Engineers, the National Academies' Committee on Science, Engineering, and Public Policy (COSEPUP) concluded that "more information is needed on

OCR for page 59
Measuring the Science and Engineering Enterprise: Priorities for the Division of Science Resources Studies the career tracks followed by scientists and engineers both inside and outside universities" (NAS 1995). In its statement on The Federal Role in Science and Engineering Graduate and Postdoctoral Education the National Science Board recommended an exploration of "improved policy data . . . to identify current and emerging national needs for the science and engineering workforce" (NSB 1997). To meet ongoing and evolving needs of policymakers and others for current information on the science and engineering labor market, SRS should improve the data it collects on the transition to employment, career paths of Ph.D.s, and the international flows of scientists and engineers. Obtaining Improved Data on Science and Engineering Careers The most comprehensive source of data on the science and engineering workforce is SRS's human resources surveys. Its three personnel survey—the National Survey of College Graduates, the National Survey of Recent College Graduates, and the Survey of Doctorate Recipients—plus the integrated Scientists and Engineers Statistical Data System (SESTAT) that draws on them provides substantial data on scientists and engineers educated at the bachelor's degree level and higher in the United States. The personnel surveys in the SESTAT system include data on the following individuals: Those with bachelor's or higher degrees in science and engineering who lived in the United States at the last decennial census (1990) Those with bachelor's or higher degrees in a non-science and engineering field who lived in the United States and worked in a science or engineering occupation in 1990 Those with bachelor's or higher degrees in science and engineering from U.S. institutions since 1990 They do not include those who graduated with bachelor's or higher degrees in a non-science and engineering field since 1990 who now work in science and engineering. They also do not include those who received bachelor's or higher degrees in science and engineering who live in the United States, but received their degrees since 1990 from a non-U.S. institution, unless they are already included under one of the rules listed above. The personnel surveys and the SESTAT system provides data that can be used to describe: Employment status and unemployment rates Characteristics of principal job, such as employment sector, occupation, work activities, salary, employment changes over the last two years, government support status, and relationship of principal job to degree Other career characteristics, such as membership in professional societies, work-related training, second jobs, (for Ph.D.s) postdoctoral positions held, and (for academically employed Ph.D.s) academic rank and tenure status Educational background, such as high school diploma, associate degree(s), first bachelor's, two most recent degrees, and degree fields Demographic characteristics, including age, gender, race/ethnicity, citizenship status, country of birth, disability, marital status, spouse's employment status, dependents, and parental education The purpose of the SESTAT system, as seen in these data elements, has been to estimate the population of scientists and engineers in the United States and to characterize their employment and demographic patterns (NSF 1999k).

OCR for page 59
Measuring the Science and Engineering Enterprise: Priorities for the Division of Science Resources Studies The personnel surveys also collect information through one-time modules on important current issues in the science and engineering labor market. For example, all three surveys asked in 1993 for data on respondents' labor force status in 1988. The 1995 SDR asked questions on work history and postdoctoral experiences of Ph.D. scientists and engineers. The 1995 NSCG and SDR added questions about professional output (articles, papers, and patents). All three surveys asked questions on alternative or temporary work experiences (i.e., consulting, contracting), the reasons for such work arrangements, and whether benefits were provided (NSF 1999k). The 1997 SDR also included retrospective questions on career expectations, satisfaction with the respondent's doctoral program, and characteristics of the job search posed to recent Ph.D.s (those earning a doctorate between 1990 and 1995). There are other vehicles for collecting labor market information that SRS could tap further. For example, the Survey of Earned Doctorates, completed by new Ph.D.s at the time the doctorate is earned, poses a series of questions on postgraduation plans. These job market questions provide important trend data on type of postgraduation position (i.e, employment, further training), employment sector, and anticipated work activities. Other questions could be added to this section of the SED to make the data more robust. For example, as evidenced by the percentage of Ph.D.s who already have definite postgraduation plans at the time of degree receipt (62 percent), most Ph.D. scientists and engineers begin their search for employment or further training prior to graduation (NSF 1999i). Thus, questions could be posed to them about the job market search they engaged in prior to degree receipt. As another example, obtaining data on starting salary for those who do have definite commitments for employment would provide another indicator of the current status of the job market by field for new Ph.D.s and how this compares to the job market for more experienced Ph.D. scientists and engineers. As will be discussed below, SRS may also draw on other sources for data on scientists and engineers such as professional societies and universities. Creating and Refining the Science and Engineers Statistical Data System (SESTAT) The SRS personnel surveys are the primary sources of data in the United States on the pool of scientists and engineers, and SRS has already implemented a series of changes to these surveys in the 1990s. In 1986 SRS asked the NRC's Committee on National Statistics (CNSTAT) to make recommendations and provide design specifications for a science and engineering personnel data system in the 1990s. CNSTAT assembled a study panel that issued a report in 1989 entitled Surveying the Nation's Scientists and Engineers: A Data System for the 1990s. Appendix C provides an overview of recommendations from this report that urged SRS to restructure, expand, and better integrate its three personnel surveys—the National Survey of College Graduates, the National Survey of Recent College Graduates, and the Survey of Doctorate Recipients—to meet the information needs of policymakers, planners, and researchers on the population of scientists and engineers in the United States. SRS has won praise for the technical work it has performed on these surveys and for the development of the Scientists and Engineers Statistical Data System (SESTAT) following the guidelines of the 1989 NRC report. Individuals interviewed for this study believe that the design of the SESTAT system has effectively integrated survey results across the three surveys. Effective data collection by the Census Bureau, Westat, and formerly the NRC in administering these surveys during the 1990s, moreover, has led to low survey and item non-response rates for each of the surveys in the past. Questionnaire redesign

OCR for page 59
Measuring the Science and Engineering Enterprise: Priorities for the Division of Science Resources Studies has led to consistent responses across items and an expanded range of data collected on scientists and engineers and their careers. The praise is well deserved, yet issues remain that need to be addressed in the labor market for scientists and engineers and the design of the SRS personnel surveys and the SESTAT system. We have not been charged with carrying out the "thorough, zero-based evaluation of the design and operation of [the SRS] personnel data system" at the end of the 1990s that the 1989 report recommended SRS conduct. Thus our observations do not provide an exhaustive review of the personnel data system and its design. SRS should carry out such a review in conjunction with the Special Emphasis Panel (i.e., advisory committee) of the Doctorate Data Project and other experts who may provide insight on the content and design of all three personnel surveys. However, we would like to provide general observations on the personnel surveys and other SRS human resources surveys that re-emphasize, modify, or augment the recommendations of the 1989 report in light of circumstances a decade later. We would like to re-emphasize the 1989 recommendation that "NSF should increase the research utility of the science and engineering personnel data base by enriching the content of its surveys" by exploring three content areas that required monitoring then and still do today. First, SRS needs to modify or add content to provide greater understanding of "the career paths that scientists follow and the factors that influence key transitions, including initial entry into the labor force, mobility across fields and sectors, and retirement." Likewise, SRS should revise its personnel surveys to improve data on "the kinds of work that scientists do and how their work is changing in response to changes in technology, organizational structure, and other factors." Further, we recommend that SRS increase the research utility of the personnel data system by developing better estimates of the international flows of scientists and engineers—the estimates of immigration and emigration that the 1989 panel urged SRS to pursue. Transition to Employment for New Ph.D.s In the early 1990s, new Ph.D.s in some fields encountered increasing problems in the job market. For example, the American Institute of Physics found that the percentage of new physics Ph.D.s still unemployed during the winter following degree receipt rose to 6 percent for Ph.D.s who received their degrees during the 1993–1994 academic year. It declined to 4 percent for those receiving their Ph.D.s in the 1994–1995 and 1995–1996 academic years, yet along with the decline in unemployment has come an increase in the percentage of physics Ph.D.s working outside the field of physics (Mulvey 1998). Similarly, the American Mathematical Association found that new Ph.D.s who were still unemployed in the fall following degree receipt reached an all-time high of 14.7 percent in 1995. This unemployment figure has declined since to 7.2 percent for 1997–1998 Ph.D.s, but this still remains higher than it was in the late 1980s when, for example, it was 5.7 percent for 1989–90 (Davis, Maxwell, and Remick 1998 and 1999). Uncertain at the time whether this job market situation was a short-term problem or a sign of long-term structural change, the Association of Graduate Schools (AGS) associated with the Association of American Universities (AAU) identified the immediate sources of these job market difficulties: It is a fact that the 1992–93 recession, the downsizing of industrial basic research labs, the tapering off of federal R&D funding as a consequence of the end of the Cold War, the influx of experienced scientists from the former Soviet Union, and the substantial budget reductions in many colleges and universities all had a

OCR for page 59
Measuring the Science and Engineering Enterprise: Priorities for the Division of Science Resources Studies deleterious effect on the employment of Ph.D.s, and they all hit at once (AAU 1995). The future job market, the AGS argued, depended on the performance of the economy, future trends in federal R&D support, whether or not colleges and universities hired more faculty in light of a growing college age population, and whether the cut in industrial basic research was, itself, a short-term adjustment or a long-term structural change. In light of job market problems such as these, COSEPUP undertook its study of graduate education. COSEPUP, the NSB, AAU, and others have raised the following kinds of questions about the transition of recent Ph.D.s to science and engineering employment and have called for additional data to address these questions: What are the labor market needs and opportunities for Ph.D.s, by sector, industry, and field? What are the job market expectations of new graduates? What positions are Ph.D.s obtaining in their early careers and what do recent patterns of postdoctoral positions and non-tenure track positions mean for careers of Ph.D.s? What is the unemployment rate for new Ph.D.s during the first year following graduation? During the first five years? What are appropriate measures of underemployment for this segment of the science and engineering doctorate population? What proportion of new Ph.D.s are working involuntarily out of field and how are increases in this proportion to be interpreted? What are the starting salaries of graduates, by type of post-graduation position? What are the characteristics of the search for a first job and how can the search be improved? As AGS urged in 1995, "before we conclude that there is a long-term job crisis, we need to undertake the kind of information gathering that COSEPUP suggests and monitor development in the coming year" (AAU 1995). While SRS is attuned to these issues for Ph.D.s generally, obtaining data on the job market for recent Ph.D.s is notoriously difficult. A serious gap in SRS data on science and engineering Ph.D.s has been the job market experience of Ph.D.s in the twelve months before and after receipt of the degree. While the SED captures postgraduate plans and status at the time the doctorate is awarded, it does not adequately capture information on the job search of Ph.D.s in the months before and after receipt of the doctorate. Meanwhile, the SDR does not pick up new Ph.D.s until at least nine months after they receive their Ph.D. Thus, it misses the period when Ph.D.s face the most uncertainty and greatest difficulty if they have not already obtained a definite commitment for work or further study at the time they receive their degrees. SRS took steps to address this gap by fielding twenty survey questions on the job search in a one-time module in the 1997 SDR to be answered by those who received their doctorate degrees between July 1990 and June 1996. This cohort was asked to answer questions about: Career expectations (kind of work, employment sector) Ph.D.s had at the beginning of their doctoral programs The state of the job market at the time they completed their doctorates The time that elapsed between receipt of the Ph.D. and the time the doctorate recipient took a first career path job Constraints they encountered and resources they used in the job market Whether they found a job that met their career expectations and the relationship between the field of their degree and field of work

OCR for page 59
Measuring the Science and Engineering Enterprise: Priorities for the Division of Science Resources Studies The effect of completing the doctoral degree on salary, level of job responsibility, job security, degree of interest in position, degree of technical demand in work, management activities expected of them, and other aspects as specified by the respondent Overall satisfaction with the doctoral programs they completed SRS is commended for taking this step to obtain data that address a critical policy issue. Though the data are retrospective and subject to some post hoc bias, the graduate education community eagerly awaits analysis of the data from the 1997 SDR on these issues. The gap in data on the transition to employment, however, needs to be more fully addressed by SRS. While the new questions on the SDR noted above will hopefully yield valuable and interesting information, many of the questions are retrospective and past experiences may be described by respondents through the lens of the present rather than as they were experienced at the time. Because of this, we recommend that SRS take additional steps. SRS should add questions to the SED about the experiences of new Ph.D.s in the job market that they have had by the time of degree receipt. We believe that even with the addition of questions to the SDR, the SED should obtain at least a limited set of data on the job market expectations of new doctorate recipients as a counterbalance to the retrospective data that will be obtained at a later time. Also, adding a question to the SED on salary for those Ph.D.s who have a definite commitment would add important information about the status of employed Ph.D.s. To supplement its own data SRS should continue its productive work with others to obtain data on the job market experience of new Ph.D.s in the immediate months following graduation. SRS has interacted extensively and productively with professional societies and the Commission on Professionals in Science and Technology (CPST) in obtaining data on the job market experiences of new Ph.Ds. This interaction should be continued, and strengthened. SRS should also explore how it might work closely with colleges and universities to assist them with the development of standardized data sets on the placement of their recent graduates. In its recent report on graduate education, the AAU urged colleges and universities to maintain comprehensive data on completion rates, time-to-degree, and job placement for each of their graduate programs. The report specifically recommends that institutions track their graduates at least until first professional employment beyond postdoctoral appointments. The AAU suggests that institutions should provide student applicants with this information (AAU 1998). As research universities implement this recommendation, SRS should help them standardize their data collection efforts so that locally collected data could be aggregated in a meaningful way at the national level. Career Paths of Scientists and Engineers The NRC report Trends in the Early Careers of Life Scientists released last year has added further to the enumeration of problems facing recent Ph.D.s as they negotiate a changing labor market. The following passage from the report provides an overview of the problems facing recent Ph.D.s in the life sciences in particular: The training and career prospects of a graduate student or postdoctoral fellow in the life sciences in 1998 are very different from what they were in the 1960s and 1970s. Today's life scientists will start graduate school when slightly older and take more than 2 years longer to obtain the Ph.D. degree. Today's life-science Ph.D. recipient will be an average of 32 years old. Furthermore, the new Ph.D. today is twice as likely as in earlier years to take a postdoctoral fellowship and thus join an ever-growing pool of postdoctoral

OCR for page 59
Measuring the Science and Engineering Enterprise: Priorities for the Division of Science Resources Studies fellows—now estimated to number about 20,000—who engage in research while obtaining further training and waiting to obtain permanent positions. It is not unusual for a trainee to spend 5 years—some more than 5 years—as a postdoctoral fellow. As a consequence of that long preparation, the average life scientist is likely to be 35–40 years old before obtaining his or her first permanent job (NRC 1998c). Adding to the issues associated with a lengthier training period, the report identified problems with the job market: The 42% increase in Ph.D. production between 1987 and 1996 [in the life sciences] was not accompanied by a parallel increase in employment opportunities, and recent graduates have increasingly found themselves in a "holding pattern" reflected in the increase in the fraction of young life scientists who after extensive postdoctoral apprenticeships still have not obtained permanent full-time positions in the life sciences (NRC 1998c). This particular report drew heavily on SRS data to analyze the job market situation of life sciences Ph.D.s, and thus, provides an example of how useful SRS data may be. There still are questions about career patterns like this one that SRS data do not yet fully address. In his dissent to the report, for example. Henry W. Riecken cited the "totally inadequate evidential basis" for the report's recommendation that more federal resources should be funneled into training grants, because they would presumably enhance the career outcomes of the graduate students who receive them (Riecken in NRC 1998c). The job market situation for recent life sciences Ph.D.s, therefore, is an example of the kind of career issue that Ph.D.s, employers, and policymakers confront and for which they require additional data. The range of appropriate data needed to carry out a valid comparative evaluation of the alternatives available to the federal government for supporting graduate students and the career outcomes of students with different types of support is incomplete. The need for improved data on financial support for students throughout their time in graduate school, one component of this analysis, was discussed earlier in this chapter. To provide its users with data that allow broader and deeper analysis of career paths SRS should take a number of steps. First, SRS should strive to fully support longitudinal and time series analyses with its personnel data—especially data from the SDR. The current difficulties with using SDR data in these ways stem from several sources. In response to the 1989 NRC report, Surveying the Nation's Scientists and Engineers, SRS revised the SDR questionnaire in 1993 to make it more comparable with the other two personnel surveys and to expand the range of questions posed to respondents (Cox, Mitchell, and Moonesinghe 1998b). We endorse this questionnaire revision, but caution that while additional revisions have continued to be required to meet the needs of data users, SRS should avoid ongoing minor changes to questions that potentially disrupt time series. Second, longitudinal and time series analyses of the SDR have been compromised over the last decade because of "maintenance cuts" (i.e., changes in sample size due to fluctuations in survey budget) and new methods of survey follow-up (Cox, Mitchell, and Moonesinghe 1998a and 1999b). There is no doubt that improved survey follow-up techniques (e.g., computer-assisted telephone interviewing) have improved response in such areas as race, have compensated for earlier non-response bias on such variables, and have thus contributed to survey quality. SRS, however, should avoid changes in survey samples from year-to-year that compromise the longitudinality of surveys like the SDR, obtained at great expense and with a

OCR for page 59
Measuring the Science and Engineering Enterprise: Priorities for the Division of Science Resources Studies respondent burden that is hard to justify if the data cannot be used longitudinally. Because SRS has adopted high statistical standards in the 1990s, it has decided to eschew longitudinal and time-series analyses of SDR data for the reasons stated above. While we applaud its focus on data quality and statistical standards with respect to data collected from this point forward, we urge SRS not to err on the side of caution with respect to the data already collected. Analysis of these unique longitudinal and time series data from the 1970s and 1980s must not be abandoned simply because they are less than perfect. SRS should support analysts in their use of these data. Second, evidence from the interviews conducted for this study suggest that SRS could provide better career path data by making it available at a more detailed level as well. Often important trends in the labor market are field or sub-field specific. Thus, in order to properly analyze a labor market issue and reach conclusions as to what, if any, policy adjustments need to be made, analyses need to be carried out at this sub-field level. SRS should, along with its data users, consider options for addressing this problem. One option might be to increase the sample size for the SDR. Because the SDR samples only 8 percent of the doctorate-level scientist and engineer population, cell sizes are too small to be used in analysis. SRS should consider increasing the sample size to facilitate this level of analysis. We recognize that this could be an expensive fix for the problem; however, we suggest that SRS consider this and other means of addressing this analytic issue in a cost-effective manner. Third, NSF should work with the National Endowment for the Humanities (NEH) and private foundations to revive the humanities component of the Survey of Doctorate Recipients. This component, administered through the Survey of Humanities Doctorates, provided data on the careers of Ph.D.s in humanities fields (history, art history, philosophy, English and American language and literature, modern language and literature, classics, music, and other humanities fields). This survey was fielded biennially by the National Research Council with funding from NEH, through NSF, as a component of the Longitudinal Doctorates Project from 1977 to 1995. The loss of humanities data from the SDR represents one of the biggest gaps in the data on the academic sector of which academic science and engineering is a part. The data are important to humanists concerned about career and labor market trends in their fields. The data are also important to analysts who seek to look comprehensively at research universities and the career paths of academically employed doctorate recipients. This is so for two reasons. First, to fully understand the health of academic science and engineering it is important to understand the health of the entire academic enterprise. Second, analysis of trends in academic careers of scientists and engineers is richer when trends in careers of academic humanists are also available for analysis. From the perspective of those interested in scientists and engineers, humanists provide a comparison group that allows analysts to determine whether certain academic career trends are specific to science and engineering fields or occur across all fields. For example, the growing number of postdoctoral fellows is seen in science fields, particularly the life sciences, and far less in the humanities. Also, when those who work part-time are asked why they do so, 43 percent of humanists working part-time in 1995 responded that there was ''no suitable job available" compared to just 22 percent of scientists and engineers working part-time (NRC 1997b; NRC 1998a). These kinds of differences provide important comparative information that illuminates trends across science, engineering, and the humanities. Again, we recognize the substantial costs associated with reviving the humanities SDR.

OCR for page 59
Measuring the Science and Engineering Enterprise: Priorities for the Division of Science Resources Studies We also recognize that the NEH budget was substantially reduced in 1995 and has remained flat since that time. However, NSF could approach private foundations as well as NEH for funding. Also, while we prefer this component to be restored on a biennial basis, we recognize that having humanities data on a quadrennial basis beginning with the 2001 SDR is better than having none at all, and would support this less-costly option for reviving these data. Fourth, NSF should investigate the cost-effectiveness of linking its personnel data (especially the SDR) to data on grants, publications, and patents to facilitate deeper investigation of career outcomes. For example, federal agencies that provide research grants keep electronic records of their grant recipients, and research award data could be linked to SDR records. In 1997, for example, the SDR oversampled individuals who received NSF Graduate Research Fellowships (NSFGRF). Data on these individuals could be linked to data from the NSFGRF in an ongoing evaluation of that program, but we would also like to see SDR data linked to these kinds of data on federal awards supporting education or research for use as part of the SDR data file. For some agencies, these award data have been incomplete, e.g., only the principal investigator (PI) is listed for research grants and not all associated faculty. Still, such linking could enrich the SDR as a data set. Linking personnel data with publication and patent data is more difficult, but SRS should seek ways to make such linkages easier. SRS asked for data on the number of articles, publications, patents applications, and patent awards in a special module for the 1995 NSCG and 1995 SDR. If these data points were collected on a continuous basis they would provide benchmarks against which researchers could measure their ability to link SRS with publication and patent databases. Nonacademic Careers To better understand the career paths of scientists and engineers and the career options of new Ph.D.s, SRS must place a high priority on revising the SDR to obtain data that better describe nonacademic careers of Ph.D. scientists and engineers. The SDR questionnaire is oriented too much toward surveying doctoral scientists and engineers who hold faculty positions in colleges or universities and not enough toward those who work in government, business, or nonprofits. The majority of doctoral scientists and engineers do not work for educational institutions, but for private businesses, government agencies, or nonprofit organizations. In 1995, 48.5 percent of doctoral scientists and engineers worked in educational institutions. Of these, 42.6 percent worked in 2- or 4-year colleges or universities, 4.7 percent worked in university-affiliated research institutions, and 1.1 percent held teaching positions in elementary or secondary schools. Of those who were faculty and teachers, moreover, only about three-quarters were in tenured positions or tenure-track jobs. Just one-third of all employed, doctoral-level scientists and engineers fit the stereotype of tenure-track or tenured faculty (NRC 1998a). This has not always been the case. While the percentage of Ph.D. scientists and engineers working in educational institutions has been declining since the SDR was first fielded in 1973, a majority nonetheless did so until the 1990s (NSF 1991). As shown in Table 4-2, this labor market indicator first fell below 50 percent in the 1990s. Since it did so in the midst of job market difficulties for new Ph.D.s, this trend was finally given the level of attention it had deserved for some time. The COSEPUP report, for example, made much of

OCR for page 59
Measuring the Science and Engineering Enterprise: Priorities for the Division of Science Resources Studies Table 4-2 Employed Doctoral Scientists and Engineers by Sector of Employment, 1987–1995 Sector 1987 1989 1991 1993 1995 Educational institutions 51.7 51.3 46.9 47.9 48.5 Industry 32.0 32.5 36.2 36.6 36.2 Government 9.2 9.0 9.1 10.1 9.9 Nonprofit 6.6 6.6 6.8 5.1 4.9 Other 0.4 0.4 0.4 0.3 0.6 No report 0.2 0.2 0.5 * * * Beginning with 1993 missing data are imputed. Source: Cox, Mitchell, and Moonesinghe 1998b, p. IV-8. the fact that the majority of Ph.D. scientists and engineers work outside of academia, and thus called for more information on the career paths of these Ph.D.s both inside and outside of colleges and universities. SRS could better describe the kinds of careers pursued by Ph.D.s outside of academia by adding to the SDR questions that solicit the following kinds of detail to flesh out these careers: Data on compensation of scientists and engineers, particularly in the private sector. The SDR currently asks for salary, but this does not adequately describe compensation in the private sector where it may include stock options, bonuses, and additional benefits. Data on the productivity of Ph.D.s in the private sector where they may focus as much on innovation as knowledge generation. In its 1995 cycle, the SDR asked for information about articles, papers, and patents, and these are valuable data points, but these may provide an incomplete measure of scientific productivity for a researcher in a private firm that guards its scientific advances and innovations as proprietary information. In this instance, an expanded description of compensation may be the best proxy for productivity in the private sector although SRS should consider the range of options for capturing productivity in the private sector. Data on new occupations and not easily identifiable research jobs for Ph.D.s in industry (especially in emerging technology-based fields such as biotechnology and information technology) in such areas as sales, regulation, and patenting. This was a subject of much discussion at the September 1998 workshop. There was little consensus on whether it is necessary to train people to the Ph.D. level for these positions. However, participants noted that the number of individuals in such positions is growing and that this phenomenon requires tracking and better understanding of tasks performed. This is not an exhaustive, but rather an illustrative list of the kinds of data that could be collected. For each of these areas, new data would shed light on developments that weigh on the thinking of policymakers and program administrators who seek to assure that new degree recipients may have successful outcomes in industrial and other nonacademic careers, as well as academic careers.

OCR for page 59
Measuring the Science and Engineering Enterprise: Priorities for the Division of Science Resources Studies Work Arrangements, Field, and Occupation Surveying the Nation's Scientists and Engineers recommended that "NSF should assign priority to new or modified content items that will provide greater understanding of the kinds of work that scientists and engineers do and how their work is changing" (NRC 1989). This recommendation holds as true today as it did a decade ago, not because SRS did not address this in the redesign of its personnel survey instruments in 1993, but because the science and engineering enterprise continues to change. Specific kinds of data about the work of scientists and engineers that analysts would like to have today include: The kinds of positions they hold (e.g., permanent/temporary, full-time/part-time, contract/consultancy, job sharing; etc.) The nature of other jobs or positions held simultaneously, particularly when the additional position is in a different organization as in the case of a university faculty member also working in a start-up company (e.g., administrative positions, supplementary employment, consultancies, start-up companies, joint appointments at other institutions) The organization of the work they engage in (e.g., traditional organization, ad hoc teams, virtual teams, inter-organization partnerships, consulting/outsourcing) Job and career flexibility (e.g., flex-time, telecommuting, portable benefits) SRS has already begun to collect data on these kinds of new or alternative work arrangements. In the 1997 survey cycle, the personnel surveys include a special module on alternative or temporary work arrangements, such as contracting or consultancies. SRS, in concert with its data users, should continue to examine how all three of its personnel surveys answer these questions today and revise or add content items as needed to better describe the current work arrangements of scientists and engineers educated at the bachelor's degree level and above. Another issue that should be better addressed in the SRS personnel surveys concerns whether appropriate data are being collected on field and occupation. Surveying the Nation's Scientists and Engineers recommended that "key questionnaire items in the SDR be made comparable with those in the NSF Panel Survey. Specifically, it is imperative to include a question on occupation in both surveys that does not bias respondents toward reporting their degree fields and that conforms with the [Standard Occupation Classification]" (NRC 1989). In response, SRS dropped field of science and engineering from the SDR and added questions to obtain occupation codes utilized by the Bureau of Labor Statistics that were designed to better capture the kinds of positions held by scientists and engineers. This change standardized SRS occupational data with the other personnel surveys. To further refine the data collected in this area, SRS should consider adding current field of science and engineering back into the SDR questionnaire while also retaining the question added on occupation. The occupational categories added to the personnel surveys allow for better integration of these data with other workforce surveys. However, the work of scientists and engineers cannot be adequately described and their careers tracked without data on the field in which they currently work. Analysts need field data to more fully characterize the work of scientists and engineers. For example, respondents who may categorize themselves as "managers" may yet define themselves as working in a specific field, such as "chemistry." Analysts also need to know if a respondent has switched fields or is working in more than one field. Given the inter-and multi-disciplinary nature of research, it would also be advisable to allow respondents to the SDR in particular to select more than one field in which they are working.

OCR for page 59
Measuring the Science and Engineering Enterprise: Priorities for the Division of Science Resources Studies International Flows of Scientists and Engineers Surveying the Nation's Scientists and Engineers also argued that "NSF should pursue its planned research program to develop estimates of immigration and emigration of scientists and engineers and to develop ways of incorporating such estimates into the personnel data system" (NRC 1989). The immigration of scientists and engineers has long been a feature of the U.S. science and engineering enterprise. Data on immigration and emigration suggest, however, that flows of scientists and engineers into and out of the United States have only intensified in the last ten years, so the recommendation to develop and improve estimates of immigration and emigration is, if anything, of greater import today than in 1989. Some indicators of increased international flow in the science and engineering labor market include the following: In 1993, foreign-born individuals accounted for 16 percent of all scientists and engineers and 29 percent of doctorate-level scientists and engineers engaged in R&D in the United States (NSF 1998c). From 1986 to 1996, non-U.S. citizens grew from 22 to 33 percent of individuals receiving doctorates from U.S. colleges and universities. The number and percentage of non-U.S. citizens, however, varied by field and is higher in the natural sciences and engineering: in 1996, non-U.S. citizens made up 58 percent of new engineering Ph.D.s, and 47 percent of new Ph.D.s in each of the life sciences and physical sciences (NRC 1998b). Non-U.S. citizens accounted for two-thirds of the growth in the number of Ph.D.s from 1986 to 1996. The number of non-U.S. citizens receiving life sciences Ph.D.s increased 184 percent during that ten-year period (NRC 1998b). Citizens of China, India, Taiwan, and Korea made up 55 percent of the non-U.S. citizens who received Ph.D.s in the U.S. in 1995. Citizens of China have increased the rate at which they stay in the United States after graduation so that, in 1995, 96 percent planned to remain in this country. Similarly about 90 percent of Indian citizens have planned to stay in the United States. By contrast, increasing percentages of Ph.D.s from Taiwan and Korea plan to return to their country of origin after degree receipt (NRC 1996b). The science and engineering enterprise has become global in its dimensions in the 1990s and so has the science and engineering labor market. While the flow of funding for research and development is as important to the science and engineering enterprise as the flow of people, comparability of data across nations is weaker in the human resources area than for R&D funding. Since the movement of personnel is one of the principal ways to both facilitate the diffusion of knowledge and to meet demand for skills in the labor market, obtaining better data in this area should be a high priority for SRS and the first step would be to develop a strategy for improving its collection and acquisition over time. Substantial work on how to produce and acquire data both on comparisons of national populations of scientists and engineers and on international "flows" or "circulation" of scientists and engineers is needed. We recommend that NSF provide SRS the resources to continue and expand significantly its data collection and analysis in this area. Analytical issues on the international flow of scientists and engineers that could be informed by the additional data include: The number of U.S. citizens who study abroad and why they do so

OCR for page 59
Measuring the Science and Engineering Enterprise: Priorities for the Division of Science Resources Studies The number of foreign-educated scientists and engineers in the United States, by citizenship status, field and occupation, and whether they plan to stay The contribution that non-U.S. citizens who earned Ph.D.s in the United States have made to their home countries in teaching, research, and technology development The patterns of circulation that are of benefit to the United States, home countries, and to the global diffusion of scientific knowledge It is important to an analysis of science and engineering resources, their distribution, and use, that these issues be understood. While obtaining these data is not an easy matter, they can help illuminate science resources issues. SRS should develop a strategy for improving its data on the international flow of scientists and engineers and we urge NSF to fund additional data collection in this area. SRS should begin by focusing particular attention on improving estimates of immigrant and non-immigrant foreign-trained scientists and engineers, foreign students, and postdoctoral fellows in the United States. Data Users and External Researchers The Panel that wrote Surveying the Nation's Scientists and Engineers argued that the personnel data system in the 1990s should "provide a research base for improved analysis of relevant labor markets and of flows into, out of, and within the science and engineering labor force that can pinpoint trouble spots and provide early warnings of future problems, and . . .support basic innovative research on scientists and engineers and the science and engineering pipeline." SRS has performed well in creating a science and engineering personnel data system for the 1990s. Today, the SESTAT system and the three personnel surveys from which it draws data provide solid information for describing the stock of scientists and engineers who were educated and work in the United States. To create an improved personnel system for 2000 and beyond, SRS will also have to provide data on the flows of individuals through career paths and transitions, across fields and sectors, and across national boundaries. To accomplish this, SRS needs to develop a comprehensive plan for the SESTAT data system for the decade beginning in 2000 that takes these flows into account. The plan needs to specify analysis goals that can be used to guide both survey and sample designs. SRS should begin its work on "SESTAT 2000" with a research statement for each of the three personnel surveys that contribute to SESTAT. These research statements should detail specific policy questions the data collected from each survey are designed to address. These statements may be amended as new issues warrant. In the meantime, they will strengthen the surveys by focusing them on important issues. To accommodate new data needs SRS should re-examine survey content for the three personnel surveys and make tradeoffs on specific questions to be asked. The questionnaires for the surveys were expanded beginning in 1993 to capture additional information, and are now already long and expensive. Additional questions need to be added at this time, but increasing survey length further will increase respondent burden and cost. Some current questions may be retired to make room on the questionnaire for new questions, though SRS may also field questions on some new issues through modules that change from cycle to cycle, thus allowing additional questions to be asked on a one-time basis. In designing SESTAT 2000, SRS should also consider the frame from which it draws its

OCR for page 59
Measuring the Science and Engineering Enterprise: Priorities for the Division of Science Resources Studies samples. Currently, the three personnel surveys do not include, and therefore do not provide data on, scientists and engineers in the United States who did not receive degrees from U.S. institutions except those in the United States at the time of the 1990 census. They therefore omit a potentially substantial segment of the science and engineering workforce. A discussion of additional alternative sampling frames for the personnel surveys is included in Appendix D. We would like, finally, to re-emphasize two more of the recommendations made by the 1989 panel. First, this panel argued that "NSF should actively solicit feedback from its users on the design, content, and quality of the data system, and on the content and format of data products. NSF should consider for this purpose establishing a user panel to provide input on a regular basis." This remains an important recommendation today. SRS has recently organized a Special Emphasis Panel for the Doctorate Data Project that includes the SED and SDR. We recommend that this panel also assist with the design and content of all three personnel surveys in SESTAT for 2000 and beyond. Second, the 1989 panel also argued that "NSF should actively encourage and provide support to researchers for innovative studies of science and engineering personnel using survey microdata. NSF should consider for this purpose establishing a grants program to fund projects that use the personnel data." This sentiment was also echoed in the 1995 COSEPUP report. Reshaping the Graduate Education of Scientists and Engineers. Again, this recommendation still needs to be more fully implemented. Indeed, SRS data are often underutilized and nowhere is this truer than for the data obtained through the three personnel surveys that provide data to the SESTAT System. The SDR is a prime example. From January to October 1997, the National Research Council, then administrator of the surveys, received just seven requests for custom tabulations from the SDR compared to 27 such requests made for data from the SED.1 SRS should better publicize SESTAT data and also allow SESTAT data to be more available to and accessible by external researchers. As noted in Chapter 3, SRS and its data would benefit from a program that provides grants to external researchers to utilize SRS data in their analyses. Within this program external researchers who utilize SESTAT data should be given special consideration, so that this underutilized database receive additional use, scrutiny, and exposure. These researchers should also be involved in writing the research statements that will guide SESTAT as SRS revises this survey for the next decade and specifies the kinds of analyses that will be performed utilizing data from the personnel surveys. Finally, SRS should monitor and summarize research conducted by others using data from the NSCG, NSRCG, and the SDR. This could be tasked to a contractor responsible for research. Other federal statistical agencies provide similar summaries of research based on data of use to social scientists. If SRS were to do this also, such summaries would be an aid to researchers as well as a source of information for SRS in its role of advising policymakers. 1   These are requests for custom tabulations and do not include requests that were handled with existing data or tabulations via phone call, e-mail, or fax.