1

Introduction and Framework for the Future

The National Science Foundation’s (NSF) National Center for Science and Engineering Statistics (NCSES), one of the nation’s principal statistical agencies, is charged¹ to “collect, acquire, analyze, report, and disseminate statistical data related to the science and engineering enterprise in the United States and other nations that is relevant and useful to practitioners, researchers, policy makers, and to the public, including statistical data on”:

• research and development (R&D) trends;
• the science and engineering workforce (the focus of the present study);
• U.S. competitiveness in science, engineering, technology, and R&D; and
• the condition and progress of U.S. science, technology, engineering, and mathematics (STEM) education.

Like its predecessors, NCSES organizes the data it collects and acquires into reports and provides the data (with appropriate ethical and confidentiality protections) to researchers and policy makers for their analysis. Thus, NCSES data have become the major evidence base for American science and

___________________

¹ In 2010, Section 505 of the reauthorization of the America COMPETES Act established a National Center for Science and Engineering Statistics as 1 of the 13 federal statistical agencies, located within NSF and absorbing the prior functions of the Division of Science Resources Statistics (DSRS).

technology policy, and by all accounts the agency is well respected globally for these data.

NCSES’s past and current high standing has been maintained in part through a continuing process of evaluation and improvement, in which reports produced by panels of the National Academies of Sciences, Engineering, and Medicine at the request of NCSES have played a critical role (see Chapter 2). The most recent of these panels was asked by NCSES “to assess and provide recommendations regarding the need for revised, refocused, and newly developed indicators designed to better reflect fundamental and rapid changes that are shaping global science, technology, and innovation systems” (National Research Council, 2014, p. 6). One chapter of that panel’s report focuses on “measuring human capital,” which is the exclusive focus of the present report. Given the centrality of the measurement of the science and engineering workforce to NCSES’s mission, activities, staff, and budget, NCSES and NSF requested that the National Academies’ Committee on National Statistics “convene an ad hoc panel to review, assess, and provide guidance on NCSES’s [current] approach to measuring the science and engineering (S&E) workforce population in the United States,” . . . [especially] “to develop a framework for measuring the S&E workforce [into the future] that provides flexibility to examine emerging issues related to this unique population while at the same time allowing for stability in the estimation of key trends.” The full charge to the panel is given in Box 1-1.

A suite of surveys constitutes the core of NCSES’s efforts to characterize the constantly changing nature of the science and engineering workforce. These surveys have evolved and improved over half a century. The panel’s overall approach to addressing its statement of task was to consider the data collection, analysis, and reporting systems to which NCSES should be aspiring and moving in the future while also providing guidance on how to make the current system more efficient and how to transition toward a framework that can be implemented in the longer term.

HISTORICAL CONTEXT

During and after World War II, the role and value of the nation’s scientists and engineers and their research were recognized as important to securing victory in the war. The need to support and monitor the science and engineering enterprise came into increased focus with the advent of the Korean War. The deficit of trained scientists and engineers resulting from World War II was a primary concern addressed by two major reports of the time: Science: The Endless Frontier by Vannevar Bush (1945) (reprinted by the National Science Foundation in 1990) and the Steelman report (Steelman, 1949).

In recognition of the foundational importance of science and engineering to the fundamental goals of the U.S. government, President Truman signed into law the National Science Foundation Act of 1950. The law gave NSF the mandate “to promote the progress of science; to advance the national health, prosperity, and welfare; to secure the national defense; and for other purposes” (Public Law 81-507). As one adjunct of the central goal “to develop and encourage the pursuit of a national policy for the promotion of basic research and education in the sciences” (including engineering, medicine, and other more applied sciences), NSF was to “maintain a register of scientific and technical personnel and in other ways to provide a central clearinghouse for information covering all scientific and technical personnel in the United States.”

In September 1950, William T. Golden was charged by the Bureau of the Budget to prepare a report for President Truman on how the nation might best mobilize its scientific resources for a possible protracted military emergency. He interviewed the country’s top scientists and military

officers before recommending that the president appoint a full-time science advisor. President Truman subsequently adopted the essence of this recommendation by establishing the Scientific Advisory Committee to the White House Office of Defense Mobilization, which later evolved into a full-scale presidential scientific advisory system. Golden also was instrumental in supporting the establishment of NSF. He provided a memorandum outlining a possible program for the agency that included four recommendations, two of which related to science and technology personnel:

• Golden’s second recommendation called for a “survey detailing the existing support of graduate and undergraduate education in the sciences by the many public and private agencies so engaged,” and noted that “a clear understanding of the existing situation is a condition precedent to an effective fellowship and scholarship program by the National Science Foundation.”
• Golden’s third recommendation called on NSF to “study the scientific manpower resources of the United States: (a) as specifically called for in the Act, by taking over, completing, and keeping current the detailed National Scientific Register; and (b) by preparing quantitative analytical studies of available and prospective scientific and technical manpower. The categories would include scientific and technical specialties as well as degrees of proficiency, years of experience, age brackets and the like. These latter studies would be based in part on judgment and statistical analyses of available data rather than just on head counts. Dependable data of this sort is badly needed in mobilization planning and would influence Selective Service policies and decisions as to educational emphasis and support by Government and private agencies.” (Blanpied, 1995)

NSF’s third annual report, covering the period July 1, 1952, to June 30, 1953, included the first survey results on human resources for science and engineering. The report was well received by President Eisenhower, who issued an Executive Order (10521) requiring NSF to continue to produce such comprehensive reports. Thus, the Division of Science Resources Statistics (DSRS) was established within NSF early in its history.

The Cold War and the launching of Sputnik I by the Soviet Union highlighted the continuing need for science and engineering resources and information about those resources. In 1972, the National Science Board concluded that it could provide a significant service to the nation by issuing a series of reports on science indicators based on quantitative data, including data on science and technology personnel. In 1982 and again in 1987, Congress mandated biennial Science and Engineering Indicators

(SEI) reports (see National Science Board [2000] for a detailed history of the importance of quantitative data, including data and analyses on science and engineering human resources, during this period).

The SEI reports are produced by the National Science Board. These are highly regarded flagship reports on all quantitative data regarding the U.S. science and engineering enterprise in a global context. They are authoritative, policy neutral, and extensively reviewed, and over the years have reported on a wide variety of data on science and engineering personnel. The reports take a broad look at the science and engineering landscape, ranging from elementary and secondary mathematics and science education to labor market conditions and beyond, relying on data from a variety of sources. The 2016 SEI report contains two chapters focused on higher education in science and engineering and the science and engineering labor force. The chapters on academia and industry also contain salient information, for instance, on the doctoral and postdoctoral science and engineering workforce. These chapters rely primarily on the various NSF human resource surveys detailed below but also include data from other external sources. Appendix A summarizes the SEI figures and tables that rely on data from the NCSES science and engineering surveys that are the focus of this report. Because the SEI data and analyses are extensive, the reports have not been published in hard copy since 2016, but are fully available electronically (see National Science Board, 2016).

In 1980, Congress enacted the National Science Foundation Authorization and Science and Technology Equal Opportunities Act (Public Law 96-516). The act established within NSF a Committee on Equal Opportunities in Science and Technology to advise the agency concerning the act’s implementation. The act also required the NSF director to prepare and transmit to Congress a biennial report concerning the participation and status of women and minorities in science and technology, including an accounting and comparison by sex, race, and ethnic group and by discipline of the participation of women and men in scientific and technical positions. This legislation was enacted in recognition of the importance not only of providing equal opportunities but also of expanding the diversity of people entering STEM fields. It reflects awareness that population shifts in the United States necessitate training and employing a greater percentage of women and underrepresented minorities in STEM fields if the nation is to recruit and employ an adequate science and engineering workforce and maintain its global competitive advantage.

According to the 2017 Women, Minorities, and Persons with Disabilities in Science and Engineering report, the representation of certain groups of people in science and engineering education and employment still differs from their representation in the U.S. population (National Center for Science and Engineering Statistics, 2017g). Women, persons with disabilities,

and three racial/ethnic groups—blacks, Hispanics, and American Indians/Alaska Natives—are underrepresented in science and engineering. Women have reached parity with men among recipients of science and engineering degrees overall, but their representation differs by field, and they constitute disproportionately smaller percentages of employed scientists and engineers relative to their percentage of the U.S. population. Blacks, Hispanics, and American Indians/Alaska Natives have gradually increased their share of science and engineering degrees, although the extent of this increase varies across fields, while they have remained underrepresented overall in educational attainment and the science and engineering workforce (National Center for Science and Engineering Statistics, 2017g). It should be noted that sample size limitations in the Survey of Doctorate Recipients (SDR) have historically made it particularly difficult to conduct in-depth analysis on subgroups.

The need for and role of science and engineering human resources continue to represent an issue of national economic competitiveness and security. A report of the National Research Council on research universities and the future of the United States highlights the importance of these universities to the nation’s prosperity and national security (National Research Council, 2012b). Among that report’s recommendations is the need to focus on building the talent of all Americans, including women and underrepresented minorities, in STEM fields; reforming graduate education; and ensuring that the United States will continue to benefit from the participation of international students and scholars. Likewise, Pressman and colleagues (2017) document that from 1996 to 2015, academic patents and the subsequent licensing to industry increased the gross output of U.S. industry by up to $1.33 trillion and U.S. gross domestic product (GDP) by up to $591 billion, and supported up to 4.3 million person-years of employment (see also Roessner et al., 2017).

The remainder of this chapter provides a brief overview of the current NCSES surveys, which are described more fully in Chapter 2. It then describes the key elements of a framework for further improving NCSES’s measurement of the science and engineering workforce, which are addressed more fully in subsequent chapters:

• evolving data needs (Chapter 3);
• sample design and estimation procedures (Chapter 4);
• data collection and processing (Chapter 5); and
• data analysis, dissemination, and outreach (Chapter 6).

NCSES’S CURRENT APPROACH TO MEASURING THE SCIENCE AND ENGINEERING WORKFORCE

At the core of NCSES’s current approach to measuring the U.S. science and engineering workforce are two census surveys and two probability surveys, supplemented for selected purposes by data from other sources, including NCSES’s own Business Research and Development and Innovations Survey (BRDIS), the Bureau of Labor Statistics’ Occupation Employment Statistics (OES) Program, and the Office of Personnel Management’s (OPM) database of the U.S. science and engineering workforce. The principal components of the NCSES data system, as shown in Table 1-1, are the (complete census) Survey of Graduate Students and Postdoctorates in Science and Engineering (GSS), the (complete census) Survey of Earned Doctorates (SED), the National Survey of College Graduates (NSCG), and the SDR. In addition, every 2–3 years NCSES also conducts the Survey of Postdocs at Federally Funded Research and Development Centers. A new Early Career Doctorates Survey (ECDS) also is currently being launched. All of the existing surveys are widely used as a source for NCSES reports, policy analysis, and basic research (National Center for Science and Engineering Statistics, 2017b, 2017c, 2017d, 2017e; National Academy of Sciences, National Academy of Engineering, and Institute of Medicine, 2010; National Research Council, 2012a).

The GSS, conducted since 1972, is an annual census of all U.S. academic institutions granting research-based master’s or doctoral degrees in science, engineering, and health fields. From each institution, the GSS collects the total number of graduate students, postdoctoral appointees, and doctoral-level nonfaculty researchers classified by gender and other characteristics, such as source of financial support. Aggregate data are reported in the fall of each year by institutional coordinators from all U.S. institutions granting research-based master’s degrees or doctorates in science, engineering, and selected health fields.

The SED, an annual census of all persons receiving a research doctorate in any discipline from an accredited U.S. academic institution in a given academic year, is the oldest and best-established of the NCSES core surveys. It has been conducted since 1958, but data are available on individuals who received doctorates from U.S. institutions beginning in 1920.

The NSCG, which began in 1973, is the primary means by which NCSES collects information on the science and engineering workforce. The survey samples individuals who have at least a bachelor’s degree in any discipline and has undergone a number of design revisions over the years, most recently when the American Community Survey (ACS) became the sampling frame for college graduates. The NSCG uses a rotating panel design that includes a new sample of graduates every biennial cycle. Sample members are

TABLE 1-1 Science and Engineering Education and Workforce Surveys of the National Center for Science and Engineering Statistics

^a For a discussion of science and engineering degree fields, as well as scientists and engineers, see NCSES’s Scientists and Engineers Statistical Data System Frequently Asked Questions at https://www.nsf.gov/statistics/sestat/#metadata&sestat-faq [October 23, 2017].

recontacted for three additional cycles (through 6 years postgraduation). This is NCSES’s only source of detailed information on the education and occupations of the science and engineering workforce below the doctoral level (i.e., bachelor’s and master’s degree holders).

The SDR also began in 1973, when NCSES began to follow up a sample of respondents from the SED on a biennial longitudinal basis. A sample of new recipients of a U.S. research doctoral degree in a science, engineering, or health field is added at each cycle, and other new samples are added occasionally as a result of special requests or funding (as in 2015). Typically, sample members are followed until the age of 76. The number and balance of longitudinal cases versus new (cross-sectional) entries are determined at the time of each cycle (see Chapters 2 and 4 for additional detail on the sample designs). Since 2005, NCSES also has periodically conducted a Survey of Postdocs at Federally Funded Research and Development Centers (see Chapter 2).

Although the nature, strengths, and weaknesses of each of the surveys listed in Table 1-1 are discussed in more detail in Chapter 2, key aspects of these surveys are noted here. All are fixed-interval, largely cross-sectional surveys (or censuses), with some (again fixed-interval) longitudinal components. Aside from the derivation of the SDR sample from the SED, there are no data linkages over the separate surveys or with other major data sources. Finally, the content is focused on characterizing relatively detailed (or “fine”) fields of degree and occupation, along with other demographics (e.g., gender, race/ethnicity, age) and on occupationally descriptive data (e.g., type of employment and employer, income/salary/wages).

The surveys have innovated and improved continually over the years in response to new substantive concerns and improvements in survey and other data collection methods and technologies. Yet they also still maintain some of their original design characteristics because of the need for caution in preserving trend data. They also reflect some of the important characterizations of and assumptions about the science and engineering workforce in the mid- to late 20th century: composed largely of native-born males who were sole or primary breadwinners, with doctoral-level scientists and engineers being a major driving force for scientific innovation and productivity, and with education and training occurring mainly in early adulthood and leading to fairly stable careers or jobs that reflected and utilized that education and training (National Academies of Sciences, Engineering, and Medicine, 2016a; National Science Board, 2015, 2017; Weisberg and Galinsky, 2014). The survey content also reflects much of the social, political, and economic context of the Cold War era in the United States and globally, as well as the organization of economic and technological innovation and development in universities, government, and large private organizations.

Increasingly since the 1980s, however, dramatic changes have occurred, and continue to do so, in all of the conditions and assumptions underlying the nature and functioning of the science and engineering workforce; the social, political, economic, and geographic context in which that workforce is situated and functions; and the sources and methods for collecting, organizing, analyzing, and reporting/disseminating data on the workforce. Thus, while respecting the need to maintain comparability in key data over time, NCSES and this panel are interested in a new framework for the science and engineering workforce surveys that reflects the realities of the nature and functioning of this workforce in the 21st century, as well as changes and opportunities in both (1) the nature and sources of data needed and available for describing and understanding the dynamically changing workforce, and (2) the methods for collecting, analyzing, reporting, and disseminating such data. The possibilities for and broad outline of a potential new framework are increasingly clear. Maintaining and enhancing the strengths of the current system while evolving it toward a new framework will be challenging, and that challenge is the focus of the succeeding chapters of this report. The remainder of this chapter describes the initial outline of this aspirational framework and ways of moving toward it.

AN ASPIRATIONAL FRAMEWORK FOR THE FUTURE

Evolving Data Needs (Chapter 3)

Several changes in the past decades in the nature of work and employment over the life course, especially for the science and engineering workforce, have yet to fully be reflected in the current suite of NCSES surveys. Perhaps most important is the increasing variability and fluidity of occupational and educational careers as a function of increasingly dynamic changes in the social and economic organization of society. Over just three decades, for example, information technology has become one of the dominant economic sectors and social forces in the United States and other societies (Auriol et al., 2013; National Research Council, 2012a). Individuals in the mid-20th century generally moved fairly linearly and steadily through educational levels and into jobs for which their education had prepared them, and then often had quite stable long-term careers within a given occupation and even work organization or employer. By contrast, current generations have much more irregular patterns of educational attainment, moving in, out of, and across higher education institutions and levels, as well as similarly variable patterns of employment that entail increasing rates of change across occupations and employers and employment sectors (e.g., academic, government, and private employers and, increasingly, self-employment/entrepreneurship) and even academic fields (National Science Board, 2015, 2017). Approaches to caring for family members (such as

children and aging parents) while pursuing careers in science and engineering fields also have grown more diverse (Nerad, 2009).

To understand the sources, nature, and future directions of the science and engineering workforce at any given point in time, then, requires much better understanding of the nature, causes, and consequences of career pathways through which individuals move over their life course. To this end, there is a growing need for longitudinal data from the science and engineering workforce surveys. Although the NSCG and SDR follow sample members longitudinally, NCSES’s focus in the past has been on producing cross-sectional estimates, and the data released have not been configured in ways that would facilitate longitudinal analysis (see Chapter 4). The panel believes that going forward, greater emphasis is needed on enabling and supporting use of the data for longitudinal analyses, which would expand the usefulness of these surveys in important ways. Longitudinal data that follow individuals through the pathways of their careers in higher education and the workforce can help researchers understand their reasons for changing those pathways and the precursors and antecedents of such changes. Educational and occupational pathways no longer occur primarily within nation states, but internationally and globally. Thus, NCSES data need especially to capture the patterns of immigration into American science and engineering educational and work organizations by foreign nationals, as well as the subsequent emigration to home countries or other nations of some of these immigrants and the emigration of native scientist and engineers. Similarly, it is important to understand whether and why members of the American science and engineering workforce work for American, foreign, or multinational organizations, and the consequences for both the workforce and scientific and technological development.

A major factor shaping the career paths of scientists and engineers, as well as their productivity and personal satisfaction, is the skills and training acquired in the course of education and work experience and their match with and utilization in current jobs. The increased fluidity of the occupational structure and of individual careers creates an increased need to assess and reassess both the skills and training individuals do and do not have and the skills and training required and used in their work settings (National Academy of Sciences, National Academy of Engineering, and Institute of Medicine, 2010).

The science and engineering workforce also is becoming increasingly diverse, not only in terms of nativity but also, within both native and foreign-born populations, in terms of gender, race/ethnicity, and other characteristics, although this increased diversity has occurred only gradually and modestly over the past few decades and has varied greatly by field and group (National Academy of Sciences, National Academy of Engineering, and Institute of Medicine, 2010; National Science Board, 2016). In particular, a wide gap in educational attainment and workforce participation still

exits for women and underrepresented minorities in science and engineering fields such as engineering and computer science. The future recruitment, growth, and development of the nation’s scientists and engineers will depend on greater understanding not only of the diverse composition of the science and engineering workforce but also of the factors that facilitate or impede the entry, retention, and advancement of underrepresented groups in the workforce. These factors range from marital and family status, to work–life balance, to conditions of work, to patterns of discrimination and harassment. NCSES needs to incorporate increased measurement of these and other emerging issues in all of its surveys.

In the future, it will potentially become more feasible to obtain a growing array of important data that can be used to understand the nature, causes, and consequences of various facets of the science and engineering workforce from administrative records or other forms of nonsurvey data. Such data include characteristics of organizations and geographic contexts in which scientists and engineers work, data on the products of their work (e.g., publications and patents), employment and earnings histories, and so on. NCSES has already begun to research the use of such administrative records and other nonsurvey data, and these sources will likely have an increasing role as part of the human resources data system in the future.

The U.S. workforce is becoming increasingly more diverse as well in terms of educational background. The SED and SDR have been the jewels in NCSES’s crown for many decades, but capturing the science and engineering workforce across all postsecondary levels is becoming more important. Whole science and technology enterprises, most notably the information technology sector, are developed and led by people with only a bachelor’s or master’s degree (or even no postsecondary degree, a segment of the workforce not captured by the current surveys, in part because of sample design constraints). Greater emphasis on such individuals, and even those with specialty training and degrees from community colleges, is therefore imperative (see National Academies of Sciences, Engineering, and Medicine, 2016a).

Sample Design and Estimation Procedures (Chapter 4) and Data Collection and Processing (Chapter 5)

The content needs described above, together with developments in survey and data methods, suggest a potential future framework for the design of the science and engineering workforce survey system with several important characteristics.

First, as suggested above, a new framework would make greater use of nonsurvey data resources, including administrative data, although surveys would remain an essential aspect of the data system. These nonsurvey data

sources could be useful in developing sampling frames, validating survey data, improving nonresponse weighting and imputation, substituting for some survey data to reduce respondent burden, and adding contextual or supplementary data to individual-level survey data.

Second, an integrated and longitudinal system of surveys and ancillary data would make the data collected more useful. Currently the SED and SDR have no relation to or overlap with the ECDS or NSCG. The ECDS could potentially be incorporated into the SDR, providing comparable national data on the same or similar topics now included only in the ECDS and allowing follow-up of ECDS samples over their life course. Following NSCG sample members longitudinally for longer periods throughout their life course would allow understanding of career pathways from the baccalaureate through doctoral levels and moves across those educational levels, as well as the career pathways of members of the science and engineering workforce with bachelor’s and master’s degrees. Although there are major practical limitations to developing a sampling frame based on institutional or administrative records for the NSCG (similar to the SED in the case of the SDR), the tradeoffs of different methods are constantly evolving and need to be researched continually. Such a shift may ultimately be necessary should the current ACS sampling frame cease to exist or become inadequate or inaccessible for NCSES’s needs.

Finally, technological advances in the digital age, combined with the growth of alternative data sources, open up new and innovative methods for the collection of survey data. In ongoing research, it will be important to maintain participation rates and data quality as the circumstances of sample members’ lives and their interactions with technology change. The NCSES surveys are increasingly moving to multimode methods (e.g., combinations of mail, Web, telephone, and face-to-face data collection), but much more use of such multimode methods will be possible in the future. In particular, in the future, administrative data sources may be able to substitute for and improve upon self-reports for the collection of certain kinds of data. One of the most promising data sources may be unemployment insurance earnings records, which also can be linked to the Quarterly Census of Employment and Wages. These data could in turn cue brief and specific follow-up survey modules with members of the NCSES longitudinal panels regarding, for example, the nature of and reason for unemployment, a job change, or a major change in income. These modules could reduce the frequency and intensity of fixed-interval longitudinal follow-up surveys and create a record of career pathways in real time. This change could also open up new possibilities for the use of modules tailored to and stimulated by survey responses or other data (e.g., job loss or change, marital and family change, reports of discrimination or harassment).

Data Analysis, Dissemination, and Outreach (Chapter 6)

A key problem for NCSES, as for any federal statistical agency, is how to get information and data to various users in a timely, well-documented, and targeted way. Such processes are always resource-constrained and must be prioritized, but advances in technology offer new options and possibilities. Were NCSES to achieve a more integrated, longitudinal, and timely mode of data collection (i.e., one in which it would gather and update a large amount of data from surveys and other sources on a continuing real-time basis), it could construct an integrated longitudinal dataset on the sample of science and engineering workforce members that could be queried for specific information at any time, and made similarly available to prospective analysts within and outside of NCSES. This would be a complex and challenging undertaking, but it would address many current problems entailed in getting descriptive data as of a given time to users in a timely fashion, while also facilitating analytical use of the data, particularly in full and longitudinal form.

CONCLUSION

In summary, there is an increasing need for NCSES’s data collection and products to reflect nonlinear education and work paths, as well as a science and engineering workforce that is diverse in gender, race/ethnicity, and citizenship and is highly geographically mobile beyond the United States. One can imagine a future framework for NCSES’s science and engineering workforce surveys that combines survey data and administrative and other nonsurvey data into a large and representative sample (or samples) of the workforce that is followed longitudinally in real time, using more specifically targeted survey modules to address a wider range of content and reducing the need for frequent fixed-interval follow-up surveys. In the future, it may be possible to draw samples increasingly and perhaps completely from institutional or other administrative records, including those capturing immigrants and emigrants. Ideally, science and engineering workforce data from all higher education levels could be linked, providing both descriptive and analytic users with fuller, timelier, and more accessible data on the entire spectrum of the workforce and the trajectories and pathways of its members over time and the life course.

Actualizing such an aspirational framework for NCSES would be challenging and require time and resources, but would represent movement toward an ultimately more integrated and efficient science and engineering workforce data system. Succeeding chapters further describe the current system and approach (Chapter 2) and present the panel’s recommendations for both near-term improvements and the transition to a new framework in the medium or longer term.