Click for next page ( 110


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 109
6 Measuring Human Capital The National Center for Science and Engineering Statistics (NCSES) produces a rich set of human capital indicators, ranging from elementary school education; to postdoctoral training; to employment in science, technology, engineering, and mathematics (STEM) occupations. These measures convey the magnitude, composition, and quality of human capital; funding of education; deployment of human capital in industry, government, and academe; and human capital creation within industry (see Hall and Jaffe, 2012). NCSES’s academic surveys provide information on academic funding for science and engineering (S&E) research, federal spending among fields of study, and spending on academic infrastructure. 1 The education surveys provide the data needed to measure the pipeline and pathways into higher education in STEM fields. Measured by online downloads (unadjusted for length of views), the most widely viewed statistics in the National Science Board’s Science and Engineering Indicators (SEI) relate to education and the workforce, making these statistics one of NCSES’s most important products and making NCSES an international leader in S&E education statistics. This chapter summarizes the human capital issues that current and potential users of NCSES statistics say are of high value to them and then details NCSES’s statistical resources for developing data that meet user needs. The discussion includes opportunities for obtaining data from currently untapped sources to produce statistics that accord more closely with user needs and the panel’s recommendations for new directions in human capital indicators. WHAT USERS WANT The America Creating Opportunities to Meaningfully Promote Excellence in Technology, Education, and Science (America COMPETES) Reauthorization Act of 2010 calls for NCSES to provide information on STEM education, reflecting the desire of Congress for such information. NCSES has many elements in its datasets with which to satisfy this requirement. To gain a better sense of what other current or potential users of NCSES human capital indicators would like to have, the panel interviewed users of science, technology, and innovation (STI) indicators for this study. During these interviews, panel members asked one key question about the policy relevance of indicators: What are the most pressing policy issues that your agency/organization/institution encounters for which it finds STI indicators useful to have? This question yielded several fruitful responses that appeared to revolve around the central theme of this report—using STI indicators to capture change (see Box 6-1). 1 The National Science Foundation (NSF) includes social sciences and psychology in its definition of S&E (see Regets, 2010). PREPUBLICATION COPY: UNCORRECTED PROOFS 6-1

OCR for page 109
6-2 CAPTURING CHANGE IN SCIENCE, TECHNOLOGY, AND INNOVATION: IMPROVING INDICATORS TO INFORM POLICY BOX 6-1 Policy Questions Related to Education and the Workforce The Changing Economy and the Impact on Supply and Demand for Education and the Workforce Is the United States producing the skills and competences needed to meet current and anticipated demand for science, technology, engineering, and mathematics (STEM) workers? How does the workforce respond to changes in the demand for skills? How mobile are science and engineering workers between employers, between public- and private-sector jobs, and between academic and nonacademic jobs? How many people, possessing what kinds of skills, are needed to achieve a robust science, technology, and innovation (STI) system? What fields other than science, technology, engineering, and mathematics are important for advancing STI? Changes in the Demography of the U.S. Population and in the Structure and Technology of Education Is the population of science and engineering researchers aging, and if so, at what rate? How many science and engineering doctorate holders take nontraditional pathways into the STEM workforce? Does this vary by race/ethnicity, gender, or the existence of a disability? How important are community colleges in developing human resources for STEM talent? How will the rapid growth of STEM courses on the Internet, sponsored by major universities, contribute to the size and competence of the nation's STEM workforce? Changes in the Stocks and Flows of Foreign Students in the United States and around the World What are the career paths of foreign-born STEM-educated or foreign-trained individuals? What is the mobility of STEM labor between countries? How much do foreign students benefit from federal funding of graduate training? What are the stay rates for foreign students? If they stay, what field or occupation do students enter? How long does it take a STEM student to acquire a study or work visa? Which degrees are students most commonly sponsored to acquire? International Comparisons of Scientific Talent Stocks and Flows Where does the United States rank among nations on elements of advancement of scientific knowledge? In which fields is the United States a net exporter of knowledge? SOURCE: Panel’s own work. PREPUBLICATION COPY: UNCORRECTED PROOFS

OCR for page 109
MEASURING HUMAN CAPITAL 6-3 NCSES’S EXISTING HUMAN CAPITAL INDICATORS NCSES produces a broad range of data on human capital, many of which are included in the SEI. These include data on enrollments and degrees by demographic classification, including citizenship and place of birth, as well as postdoctoral fellowships. The SEI report contains information on students by type of financial support in graduate school, including support from the federal government, by field of study. These data include stay rates and intent to stay in the United States. Data also are available in the SEI on tertiary degrees conferred in other countries. NCSES’s Scientists and Engineers Statistical Data System (SESTAT) is a comprehensive database on education, employment, work activities, and demographic characteristics. SESTAT collects information from three biennial sample surveys of individuals: the National Survey of College Graduates (NSCG), the National Survey of Recent College Graduates, and the Survey of Doctorate Recipients (SDR). The SDR has galvanized an international large-scale data collection initiative—the Careers of Doctorate Holders survey—by OECD; the United Nations Educational, Scientific and Cultural Organization’s (UNESCO) Institute for Statistics; and Eurostat (see Auriol, 2010). This survey is expected to provide an opportunity for international comparisons of doctorate recipients. The Survey of Earned Doctorates (SED) annually surveys individuals who have newly received a Ph.D. The SED is used in the latest SEI report to produce: Table 2-1—fraction of doctorate holders who earned a credit from a community college, broken out by ethnicity; Tables 2-4 and 2-5 and Figures 2-4 and 2-5—sources of support for graduate students, broken out in various ways (similar information is available in the National Science Foundation [NSF]/National Institutes of Health [NIH] Survey of Graduate Students and Postdoctorates in Science and Engineering); Figures 2-19 and 2-20—total number of Ph.D.’s earned, broken out by field and demographics (similar data are available from the Integrated Postsecondary Education Data System [IPEDS]); 2 Table 2-10—median time to degree; and Tables 2-12 and 2-13 and Figure 2-25—number of U.S. Ph.D.’s, by country of origin. It appears that many of the most interesting data on doctorates can be obtained from other sources, so, as discussed in greater detail later in this chapter, the SED appears to be a good candidate for less frequent administration, assuming its use as a frame for the SDR can be sorted out. 3 In 2009, following recommendations by the National Research Council (2008), the U.S. Census Bureau added a “field of bachelor’s degree” question to the American Community Survey (ACS). 4 The Census Bureau codes the open-ended ACS responses into degree field 2 IPEDS is an institutional database housed at the National Center for Education Statistics. It contains information on higher education institutions, including community colleges. 3 Further discussion of the potential rationalization of surveys is presented later in this chapter. 4 The National Research Council’s Panel to Assess the Benefits of the American Community Survey for the NSF Science Resources Statistics Division made the following recommendation: “The National Science Foundation should use current data from the American Community Survey to evaluate the degree to which the American Community Survey with the field-of-degree question would allow for the production of mandated indicator reports in the future” (National Research Council, 2008, p. 7). PREPUBLICATION COPY: UNCORRECTED PROOFS

OCR for page 109
6-4 CAPTURING CHANGE IN SCIENCE, TECHNOLOGY, AND INNOVATION: IMPROVING INDICATORS TO INFORM POLICY categories that are based on the Classification of Instructional Program (CIP) codes. For the 2010 and 2013 NSCG, the degree field information from the ACS was used to create a degree field stratification variable, differentiating S&E from non-S&E degree fields. This variable was combined with degree level, occupation, and key demographic stratification variables to select the NSCG sample. NCSES and the Census Bureau are currently evaluating the ACS degree field responses to determine the consistency between that information and the degree field information on the NSCG. The findings from this study will help determine whether NCSES changes the way it uses the ACS degree field responses for NSCG stratification purposes. NCSES draws heavily on major sources for elementary, secondary, high school, and some postsecondary statistics for its human capital indicators. These sources include the U.S. Department of Education, National Center for Education Statistics (including IPEDS); the U.S. Department of Commerce, Census Bureau; OECD; the UNESCO Institute for Statistics; the Programme for International Student Assessment; the Center for the Study of Education Policy, Illinois State University; the Higher Education Research Institute, University of California, Los Angeles; U.S. Citizenship and Immigration Services; the U.S. Department of Homeland Security; and the American Association of Engineering Societies. Statistics from statistical bureaus of foreign countries (e.g., China, Japan, South Korea, United Kingdom, Germany) also are reported. Statistical results from peer-reviewed articles can illuminate contemporary issues, such as trends in international higher education (Becker, 2010) and international migration of high-skilled workers (Davis and Hart, 2010; Defoort, 2008; Shachar, 2006). Major data sources for workforce statistics are the Department of Labor, Bureau of Labor Statistics (Occupational Employment Statistics), and the Department of Commerce, Census Bureau (ACS and Current Population Survey). There is fervent interest in the United States and abroad regarding the contributions of foreign-born scientists and engineers to knowledge creation, entrepreneurship, innovation, and economic growth. Analyzing data from NSF’s NSCG, Stephan and Levin (2001, p. 59) conclude that “immigrants have been a source of strength and vitality for U.S. science” and that they made “exceptional contributions to the physical sciences.” Hunt and Gauthier-Loiselle (2010), also using NSF’s NSCG, as well as state-level data, found that college-graduate immigrants have a higher incidence of patenting compared with their native cohort, primarily because the former immigrants have a higher propensity to obtain S&E degrees. Borjas (2005, p. 56), using NSF’s SED and SDR data to examine labor-market outcomes related to immigration of high-skilled immigrant workers, found that “an immigrant-induced 10-percent increase in the supply of doctorates in a particular field at a particular time reduces the earnings of that cohort of doctoral recipients by 3 percent.” 5 Franzoni and colleagues (2012) conducted a survey to study the mobility of scientists in four different fields, for 16 countries. 6 India was found to have the highest share of scientists working outside of the country, while the United States was the first or 5 Card (2009, p. 18), using Current Population Survey data, found evidence that “comparing high immigration cities like Miami and Los Angeles to low immigration cities like Philadelphia or Detroit, the relative wages of workers in the lowest skill group are about 3-4% lower, while relative wages for those in the highest skill group are 3-4% higher.” 6 Franzoni and colleagues (2012) note that when they conducted their survey, NSF had only recently begun to use SDR data to publish statistics on scientists and engineers who were educated in the United States and then migrated to another country. The GlobSci database includes the following fields: biology, chemistry, materials and Earth, and environmental sciences. The countries included in the study were Australia, Belgium, Brazil, Canada, Denmark, France, Germany, India, Italy, Japan, the Netherlands, Spain, Sweden, Switzerland, the United Kingdom, and the United States. Notably, data on China were not available. PREPUBLICATION COPY: UNCORRECTED PROOFS

OCR for page 109
MEASURING HUMAN CAPITAL 6-5 second most likely destination for scientists working outside their home country. Furthermore, using NSF’s data on stay rates, Kerr (2008, p. 536) found that “frontier expatriates do play an important role in technology transfer”; however “ties between U.S. ethnic research and entrepreneurial communities and their home countries” are important for the transfer of tacit knowledge that is critical for innovation. Using the Public Use Microdata Sample (PUMS) of the 1990 U.S. Census, Dun & Bradstreet data on high-technology firms, and in-depth interviews to study the impact of immigration on entrepreneurship, Saxenian (2002) found a high concentration of S&E workers from India and Taiwan in Silicon Valley. Indian and Taiwanese engineers facilitated codevelopment of technologies and technology sharing between firms in Silicon Valley and in their respective native regions. Some researchers are beginning to explore the concept of a “reverse brain-drain” or “brain sharing,” whereby migration of foreign-born, U.S.-educated talent back to their home countries either is beneficial for collaborative research and demand for U.S. high-technology products or is viewed as creating competitive high-technology research platforms abroad. Mobility of students and workers is an indicator of knowledge flows and knowledge networks, and it shows where there exist new sources of science and technology (S&T) talent, high potential for creative ideas and collaboration, new markets for high-technology goods and services, and potentially future competition for the development of high-technology products. Having data by field of degree, by occupation, for specific metropolitan areas, and for specific countries makes it possible to conduct a rich analysis of migration flows of high-skilled students and workers (see Hunter, 2013; Wadhwa, 2009). The NSF/NIH Survey of Graduate Students and Postdoctorates in Science and Engineering is the only survey used by SESTAT that provides coverage of recipients of foreign- earned degrees, but is limited to those at the postdoctorate level. The Department of Homeland Security’s database of Labor Conditions Applications (LCA) (for H-1B visa employers) provides a benchmark for the location of newly hired foreign doctorate recipients. The NSCG could use the ACS to identify doctorates granted outside the United States to workers in the United States. The ACS information (age, degree level, and year of immigration to the United States) could guide the NSCG sampling strata, allowing unique sampling rates for those likely to have earned doctoral degrees abroad. There is also an international component of the SDR—the International Survey of Doctorate Recipients—that captures U.S. doctorate recipients outside the United States. NCSES currently publishes a range of statistics on published papers, including countries, countries’ shares of cited papers, and international collaborations. NCSES also uses its Business Research and Development and Innovation Survey (BRDIS) for employment statistics. It publishes InfoBrief’s on research and development (R&D) employment intensity, domestic and foreign R&D employment, and company-performed R&D expenditures per R&D employee (see, e.g., Moris and Kannankutty, 2010). However, these data do not account for the entire STEM workforce. Headcounts and related statistics are available for the United States and worldwide for employment, R&D employment, R&D employment by occupation and gender, and highest degree earned. For the United States, there are also counts of H-1B and L-1 visa holders. 7 Full-time-equivalent (FTE) counts are available for S&E workers, 7 The H-1B and L-1 visas are for foreign workers in specialty occupations in fields that require highly specialized knowledge and intracountry transferees, respectively. For the fields covered, see U.S. Department of Homeland Security (2011a). PREPUBLICATION COPY: UNCORRECTED PROOFS

OCR for page 109
6-6 CAPTURING CHANGE IN SCIENCE, TECHNOLOGY, AND INNOVATION: IMPROVING INDICATORS TO INFORM POLICY as is the number of these FTEs that are funded by the federal government in the United States (see Burrelli, 2010). 8 NCSES could publish statistics other than those released in the 2010 InfoBrief on employment statistics. In particular, NCSES could use the BRDIS to count a defined group of non-U.S. citizens holding H-1B or L-1 visas and employed in the United States as R&D scientists and engineers for businesses. This statistic would not cover all H-1B or L-1 visa holders working for these firms, since some may be working in non-R&D activities. Since about one-third of H1-B visa holders are in computer-related occupations, 9 and only a modest proportion are included as scientists on the Immigration and Naturalization Service’s occupational list (see U.S. Department of Homeland Security, 2011b), the BRDIS number depends on how many computer-related workers a firm classifies as R&D scientists and engineers. Still, this would be a valuable new measure as it would count the stock of H-1B or L-1 visa holders working in a well-defined activity, while most of the standard data relate to flows based on less precise measures of what workers do. It would also be useful to have more up-to- date measures, since the demand for H-1B or L-1 visas varies from year to year with the economic outlook in high-tech sectors. THE SURVEY PROBLEM As noted in earlier chapters, the survey approach to data collection is growing increasingly expensive, while response rates are dropping (a trend that applies to all surveys). As budgets become tighter, the higher costs will make it increasingly difficult to continue maintaining the current set of surveys. A short-term approach for dealing with these challenges is to implement small, incremental changes that respond to immediate budgetary shortfalls. An alternative approach, however, is to reconsider the whole system of data collection over the longer term to identify opportunities for cost savings and quality-enhancing improvements. The SESTAT surveys provide a prime example. In lieu of four independent surveys, the sampling frames are interlinked, as illustrated in Figure 6-1. Each sampling frame ties back to a more complete census (or census-like) source. For the NSCG, this used to be the long-form sample of the U.S. census, now replaced by the ACS. The NSCG samples about 0.25 percent of its population. The National Survey of Recent College Graduates is based on a two-stage sample design that selects institutions, then samples students from those institutions. It samples 0.4-0.8 percent of its target population. The SDR ties back to the SED, which is actually a census of all doctoral graduates in a given period. The SDR sampled about 4 percent of its target population in 2008. 8 All of the statistics reported in this brief are special tabulations from the Student and Exchange Visitor Information System database, maintained by Immigration and Customs Enforcement of the U.S. Department of Homeland Security. 9 See National Science Board (2012a, p. 3-50) for indicators of foreign-born workers in the United States. The U.S. Department of Homeland Security (2012, p. 13) shows that approximately 51 percent of H1-B visa petitions approved were in computer-related occupations in fiscal year 2011. The U.S. Department of Labor reports positions certified for the H-1B program. STEM-related occupations accounted for approximately 49 percent of positions certified for H-1B visas for fiscal year 2011 (see U.S. Department of Labor, 2012b, p. 60). The computer-related occupations included in these statistics are computer science; computer science, engineering; and computer science, mathematics. Those three occupations made up 42.5 percent of the positions certified for H-1B visas in fiscal year 2011. PREPUBLICATION COPY: UNCORRECTED PROOFS

OCR for page 109
MEASURING HUMAN CAPITAL 6-7 2001 No SESTAT SESTAT SESTAT SESTAT SESTAT Integrated SESTAT SESTAT SESTAT SESTAT 1993 1995 1997 1999 File/ 2003 2006 2008 2010 National Estimates NSRCG NSRCG NSRCG NSRCG NSRCG Decennial NSRCG NSRCG NSRCG ACS NSRCG (new (new (new (new (new 2000 (new (new (new 2009 (new S&E S&E S&E S&E S&E SEH SEH SEH SEH bachelor's bachelor's bachelor's bachelor's bachelor's bachelor's bachelor's bachelor's bachelor's and masters and masters and masters and masters and masters and masters and masters and masters and masters 1990-1992) 1993-1994) 1995-1996) 1997-1998) 1999-2000) 2001-2002) 2003-2005) 2006-2007) 2008-2009) Decennial 1990 NSCG not NSCG NSCG NSCG NSCG NSCG NSCG NSCG NSCG conducted SED (1962-1992) SDR SDR SDR SDR SDR SDR SDR SDR SDR SED SED SED SED SED SED SED SED (1993-1994) (1995-1996) (1995-1996) (1999-2000) (2001-2002) (2003-2005) (2006-2007) (2008-2009) FIGURE 6-1 SESTAT Surveys in the 1990s, 2000s, and 2010s. NOTE: ACS = American Community Survey; NSCG = National Survey of College Graduates; NSRCG = National Survey of Recent College Graduates; SDR = Survey of Doctorate Recipients; SED = Survey of Earned Doctorates; SESTAT = Scientists and Engineers Statistical Data System. SOURCE: NCSES SESTAT Surveys in the 1990s, 2000s, and 2010s. These sampling frames were not designed around the requirements of consistent geographic coverage or maintenance of a panel over time. If the sampling rate is set below 1 percent in a state with a low number of institutions and graduates, making any reasonable set of measurements will be difficult without suppression to protect the confidentiality of respondents. This is particularly true if the sampling of institutions is not designed spatially. A design that allowed for consistent geographic coverage would make the sampling frame another level deep—geographic distribution, institutions in those regions, and students who attend those institutions. This is not the case for these SESTAT surveys. The timing of these surveys has been affected by external events. For example, the NSCG used to be administered in odd years but has switched to even years, maintaining a 2-year spacing, although for some surveys with a 5-year gap. A close tie to the decennial census is not crucial since the ACS provides data annually. In 2008, a Committee on National Statistics panel of the National Research Council recommended 10 that NCSES reconsider the design of SESTAT following the implementation of the ACS. NCSES made the decision to terminate the National Survey of Recent College Graduates 11 since the frequency of the ACS makes it unnecessary to refresh the data from the 10 “Recommendation 7-5: The National Science Foundation should use the opportunity afforded by the introduction of the American Community Survey as a sampling frame to reconsider the design of the Scientists and Engineers Statistical Data System (SESTAT) Program and the content of its component surveys” (National Research Council, 2008, p. 75). 11 NCSES, “Reconsidering the SESTAT Design,” internal document, January 2012. PREPUBLICATION COPY: UNCORRECTED PROOFS

OCR for page 109
6-8 CAPTURING CHANGE IN SCIENCE, TECHNOLOGY, AND INNOVATION: IMPROVING INDICATORS TO INFORM POLICY decennial census with data on recent college graduates. Starting in 2012, NCSES switched to a new sample frame that provides coverage similar to that of the National Survey of Recent College Graduates (with the nuance of young rather than recent college graduates) at a greatly reduced cost. The switch from the National Survey of Recent College Graduates to the ACS has the added benefit of providing better estimates of foreign degree holders working in the United States (see the section on mobility later in this chapter). 12 The decision to terminate the National Survey of Recent College Graduates is a reasonable first step toward rationalizing the SESTAT surveys, but it may not be enough to remedy the rising costs and declining response rates for all the surveys. Box 6-2 gives other suggestions for gaining efficiencies in obtaining data on human capital that complement existing NCSES survey strategies. BOX 6-2 Data Options to Consider Based on Existing Surveys Make up-front investments in improvements to survey processes and other infrastructure, with the goal of saving money down the road: — Shift to web/automated phone surveys. — Explore techniques that narrow response windows. — Explore piloting nonresponse methods being developed by the U.S. Census Bureau. Invest in increasing the value of existing datasets through annotation and processing: — Unify taxonomies from past iterations of the Survey of Doctorate Recipients, Survey of Earned Doctorates, and General Social Survey using crosswalks. — Reduce dimensionality using cluster analysis or other tools. Lay the groundwork for data/text-mining techniques going forward: — Support outside work on semantic annotation of data on research and researchers (e.g., ORCID) — Improve semantic annotation of existing National Science Foundation (NSF) data: - Standardized citation format for NSF support (similar to SciENCV at the National Institutes of Health [NIH]) - Department of the Interior for SESTAT datasets — Explore the use of administrative records on research funding from other agencies, human resource data from universities. — Conduct a modest ongoing prize competition for new metrics to develop ideas, experience, future directions, multimodal acquisition (see Chapter 7 for more on this option). — Pilot test real-time estimation during data collection (see Chapter 7 for more on the Census Bureau’s development of this idea). SOURCE: Panel’s own work. 12 Therefore, the last year of data collection using the NSRCG was 2010. PREPUBLICATION COPY: UNCORRECTED PROOFS

OCR for page 109
MEASURING HUMAN CAPITAL 6-9 POTENTIAL FOR NEW DATA SOURCES Currently, NCSES publications do not include statistics on how many U.S. employees in S&E occupations were trained in specific foreign countries. Continued collaboration between NCSES and the Department of Homeland Security is expected to yield better indicators of STEM education and the STEM workforce. Underutilization of existing data on STEM workers is a persistent problem. Although worker mobility is undermeasured in traditional STI employment statistics, some longitudinal studies capture data on the movement of workers with STEM degrees within and outside of traditional S&E jobs. During this study, staff from a range of agencies emphasized to the panel that they have a great deal of underutilized data, particularly regarding human capital. At the panel’s July 2010 workshop, for example, Erika McEntarfer of the Census Bureau described a potential project on which her division is working that entails using data from the Longitudinal Employer-Household Dynamics Program to link workers longitudinally across jobs. Integrating these data (along with similar data on firm dynamics) into the statistics offered by NCSES would create a useful set of indicators. Trends in how macroeconomic fluctuations affect workers and knowledge flows in S&E occupations are just one potential output from these data. More descriptive data on innovators also would broaden understanding of the skill sets that lead to STI advances. Obtaining this information would require case studies, which could enhance understanding of statistics based on counting stocks and flows of individuals and knowledge capital. At the July 2010 workshop, some presenters suggested that measures of STI talent should include the pool of students initially trained in community colleges as a pathway to bachelor’s and higher degrees. 13 Since many highly skilled jobs in certain engineering fields require a master’s degree and not a doctorate, NCSES has an opportunity to expand its human capital indicators by providing more information on master’s-level STI talent. More detailed data on immigrants, women, minorities, and people with disabilities in the S&E workforce would assist in answering such questions about talent. Without good counts of these individuals, the nation’s full STI workforce capacity cannot be known. Also important is to consider what fields other than physical and biological sciences, technology, engineering, and mathematics are important for advances in STI. A careful assessment of network sciences, as well as behavioral and social sciences, would be valuable because these fields are partly responsible for the acceleration of innovation. For example, social networks are used in “collaboratories” to further scientific inquiry without the need for bricks-and-mortar facilities. These contributions to the scientific enterprise come from the behavioral and social sciences. For advances in basic science, it is arguably true that the physical and biological sciences are the primary reservoir of talent. The broader scope of innovation, however, including managerial and organizational elements, includes the social, behavioral, and managerial sciences as critical contributors to outputs and outcomes. Hill (2007) argues that the skill set for advancing innovation is changing: In the post-scientific society, the creation of wealth and jobs based on innovation and new ideas will tend to draw less on the natural sciences and engineering and more on the 13 In July 2011, NCSES published an InfoBrief on this subject, “Community Colleges: Playing an Important Role in the Education of Science, Engineering, and Health Graduates” (Mooney and Foley, 2011). The data for that report were taken from NCSES’s National Survey of Recent College Graduates. PREPUBLICATION COPY: UNCORRECTED PROOFS

OCR for page 109
6-10 CAPTURING CHANGE IN SCIENCE, TECHNOLOGY, AND INNOVATION: IMPROVING INDICATORS TO INFORM POLICY organizational and social sciences, on the arts, on new business processes, and on meeting consumer needs based on niche production of specialized products and services in which interesting design and appeal to individual tastes matter more than low cost or radical new technologies. REVISED AND NEW HUMAN CAPITAL INDICATORS Revised and new human capital indicators are needed in the areas of labor force mobility, the supply of STEM skills and talent, the demand for STEM skills and talent, and the growth of online STEM education. Labor Force Mobility While current NCSES indicators provide extensive information on the STEM workforce, their educational backgrounds, and development of technical skills over time, information on the mobility of STEM workers is comparatively sparse. Given the rapidly changing nature of the economy, the panel believes it is important to better understand how the STEM workforce responds to these changes, including movement between academia and the private sector, mobility across industries, changes in occupation, mobility across regions of the United States, and international mobility. Tracking career paths and mobility is relevant for a number of reasons. The first is related to the contribution of holders of doctorates to innovation in business. Questions to be addressed include what types of mobility patterns are seen outside of academia, in what sectors/industries and what types of jobs, and what motivates the choice to seek employment in the private sector. The second reason relates to the flip side of this issue: Are doctorate holders seeking employment in the private sector because they want to, or because of poor working conditions or lack of employment within academia? What types of skills are in demand for doctorate holders? How well do the skills and competences acquired through a doctoral education match skills needed in later employment? One way to develop such measures is to start with the jobs people currently hold and then ask about changes. Individuals could be tracked as they moved between the following sectors: educational institution; private, for-profit; private, nonprofit; federal and state/local government; and self-employed. Data from the 2010 SDR questions A9 though A14 could be used to track movement from the principal employer to another sector. 14 Data sources that follow individuals over time offer the best opportunity to develop statistics on STEM labor mobility. The panel’s recommendations in this area involve drawing on existing dynamic databases developed by other agencies and exploiting the sampling procedure of NCSES’s SDR, which provides the opportunity to follow doctorate holders over time. 14 See http://nsf.gov/statistics/srvydoctoratework/surveys/srvydoctoratework_nat2010.pdf [July 2013] for details on the survey questions. PREPUBLICATION COPY: UNCORRECTED PROOFS

OCR for page 109
MEASURING HUMAN CAPITAL 6-11 Survey of Doctorate Recipients The SDR is a longitudinal study of individuals who received a doctoral degree from a U.S. institution in a science, engineering, or health field. 15 The survey follows a sample of these individuals throughout their career, from the year of their degree award through age 75. The panel is refreshed in each survey cycle with a sample of new earners of doctoral degrees in these fields. The longitudinal nature of the survey makes it possible to examine mobility patterns over time for holders of these degrees. However, this dynamic feature of the SDR data typically has not been exploited to create indicators of the labor mobility of doctorate holders, nor have the data been linked to create a dynamic dataset. Instead, existing indicators are based only on single survey cycles. The SDR is used to track overall changes in statistics over time, such as changes in the number of doctorate holders within academia or in shares of doctorate holders employed in the business sector. Given the longitudinal structure of the SDR sample, however, it should also be possible to use the data from this survey to create measures on the mobility of doctorate holders across different employers, occupations, or sectors. To this end, longitudinal weights would need to be constructed that would take account of changes in the sample and target population over time. One option for tracking researcher mobility in the SDR that would not require additional survey questions would be to introduce a partial panel structure to the survey in which a group of individuals would be included in the sample over a series of survey cycles. A longitudinal database created from the SDR could be used to generate a number of useful indicators that would provide insight into the mobility of U.S. holders of doctorates in science, engineering, and health. Among these indicators would be mobility rates between the private sector and academia and from temporary to permanent positions within academia. A specific group that has received increased attention is postdoctorates. A concern is that the inability to secure a permanent position may lead many of these individuals to abandon a research career and seek other forms of employment. Longitudinal data on doctorate holders would provide a greatly enhanced ability to track this group (early-career doctorate holders are discussed in more detail below). Furthermore, these data could be analyzed to identify patterns in job mobility across occupations and across industries within the private sector. RECOMMENDATION 6-1: The National Center for Science and Engineering Statistics should do more to exploit existing longitudinal data. Specifically, NCSES should exploit the longitudinal panel structure of the Survey of Doctorate Recipients (SDR) in the following ways: create indicators of researcher mobility over time, by constructing longitudinal weights for the SDR that take account of changes in the sample and target population over time—these weights should be constructed both for subsequent survey cycles and for existing data; create a dynamic database for researcher use in which data from the SDR over time would be linked at the level of the individual; and 15 Science, engineering, and health fields include biological, agricultural, and environmental life sciences; computer and information sciences; mathematics and statistics; physical sciences; psychology; social sciences; engineering; and health (see National Science Foundation, 2012d). PREPUBLICATION COPY: UNCORRECTED PROOFS

OCR for page 109
6-12 CAPTURING CHANGE IN SCIENCE, TECHNOLOGY, AND INNOVATION: IMPROVING INDICATORS TO INFORM POLICY enhance coverage of recent doctorate recipients to better track their initial employment and career path in the first years after they receive their Ph.D., which could potentially be accomplished by including an additional module in the SDR or by exploiting that survey’s longitudinal capacities or both. Longitudinal Employer-Household Dynamics (LEHD) and Baccalaureate and Beyond Longitudinal Study (B&B) LEHD is a longitudinal, employer-employee database within the U.S. Census Bureau that is developed through the integration of a variety of data sources, including federal and state administrative data on employers and employees and Census Bureau censuses and surveys. The data, which are quarterly, follow both individuals and firms over time and thus can be used to track job flows across firms and industries, as well as labor mobility across industries and occupations. These data would thus appear to be well suited for the development of indicators of the mobility of the STEM (or since, engineering, and health) workforce. 16 The B&B study, which is conducted by the National Center for Education Statistics (NCES), 17 follows a cohort of bachelor’s degree recipients during the 10 years after they have finished college. For example, the B&B:93/03 study followed approximately 11,000 students identified in the 1992-1993 National Postsecondary Student Aid Study (NPSAS) as having earned a bachelor’s degree during the 1992-1993 academic year. These individuals were interviewed in 1994, 1997, and 2003 on a variety of topics, such as their education, job search activities, work experiences, further participation in degree and certificate programs, family formation, and other aspects of life after college. Initial B&B cohorts are a representative sample of graduating seniors in all majors. A third B&B cohort was recruited from 2008 graduates, and it has reached a third interview round, with full results forthcoming. Because it is larger than the previous two cohorts, this sample allows for more detail by industry sector. 18 These data can be useful in examining linkages between education and science, engineering, and health occupations, such as the educational background for these occupations and mobility across them for different academic areas. A particular challenge is capturing the international mobility of doctorate holders. While ample data are available on individuals that have earned a Ph.D. at a U.S. university and still reside in the United States, the same is not true for two other groups: individuals that have received a doctorate from a foreign university and U.S. doctorate holders that now reside abroad. While there are no easy solutions to closing this gap, the panel strongly supports NCSES’s efforts to identify these groups and include them in SESTAT survey populations, and notes that U.S. doctorate holders abroad have recently been included in the target population of the SDR. The LEHD and B&B databases offer opportunities for tracking the mobility patterns within sectors and types of occupations and for specific kinds of education. Data on occupations 16 See Abowd, Haltiwanger, and Lane (2004) for details on the characteristics and uses of LEHD data and Bjelland, Fallick, Haltiwanger, and McEntarfer (2011) for detail on the use of LEHD data for understanding labor mobility. 17 For more information on the B&B study, see National Center for Health Statistics (2013). 18 The second B&B cohort of about 10,000 students was drawn from the 2000 NPSAS and followed up in 2001. The third B&B cohort is the largest, with approximately 19,000 students sampled from the 2008 NPSAS and followed up in 2009 and 2012. See http://nces.ed.gov/surveys/b&b/about.asp for more detail on the types of data gathered by all three longitudinal surveys. PREPUBLICATION COPY: UNCORRECTED PROOFS

OCR for page 109
MEASURING HUMAN CAPITAL 6-13 and education can also be used to examine the demand for skills in key industries, an issue discussed in greater detail later in this chapter. To carry out this activity, NCSES will have to navigate privacy issues as data from the SDR and the LEHD or B&B are linked. However, this activity offers high potential value, particularly for understanding the correspondence between supply and demand for skill sets in S&T sectors and worker mobility. Social capital is also an important factor in the development and maintenance of human capital. For instance, similar levels of skills, knowledge, and networks developed through STEM education can produce different productivity outcomes in part as a result of differences in social capital. In cases in which skills and knowledge are fairly standardized, higher levels of social capital may lead to higher performance outcomes. Payne and colleagues (2011) synthesize the research on social capital published in the sociology and business management literatures. Although the literature contains a variety of definitions of social capital, Payne and colleagues note that “most scholars generally agree that social capital represents the resources an individual or a collective gains through a social structure or network of relationships....” Saxenian (2002, p. 28) gives an illustration of how social capital can influence entrepreneurship and productivity in high-technology industries: First-generation immigrants, such as the Chinese and Indian engineers of SiliconValley, who have the language and cultural as well as the technical skills to function well in both the United States and foreign markets are distinctly positioned to play a central role in this environment. They are creating social structures that enable even the smallest producers to locate and maintain mutually beneficial collaborations across long distances and that facilitate access to Asian sources of capital, manufacturing capabilities, skills, and markets. Payne and colleagues list several “key operational examples” that could inform indicators of social capital, a few of which may be relevant for the STEM workforce: an index measuring the extent of redundant connections for an individual or collective, an index measuring the shortest path between one node and other nodes in a network, affiliation of each actor with all other actors (e.g., membership in professional associations), access to information and resources (which may be correlated with the rank of the university attended by the individual), and embedded ties to external stakeholders. While these specific elements are not available in the LEHD or B&B, NCSES could develop proxies for these data from its SESTAT surveys. For example, the 2010 SDR and the 2010 NSCG include questions on professional associations: “During the past 12 months, did you attend any professional society or association meetings or professional conferences? Include regional, national, or international meetings.” and “To how many regional, national, or international professional societies or associations do you currently belong?” NCSES could also consider obtaining information on social capital by bridging data on individuals who appear in both the LEHD and STAR METRICS 19 databases. 19 STAR METRICS is an acronym that stands for Science and Technology for America’s Reinvestment: Measuring the Effect of Research on Innovation, Competitiveness and Science. See https://www.starmetrics.nih.gov/ [June 2013]. PREPUBLICATION COPY: UNCORRECTED PROOFS

OCR for page 109
6-14 CAPTURING CHANGE IN SCIENCE, TECHNOLOGY, AND INNOVATION: IMPROVING INDICATORS TO INFORM POLICY RECOMMENDATION 6-2: The National Center for Science and Engineering Statistics should draw on the Longitudinal Employer-Household Dynamics Program (occupations) and the Baccalaureate and Beyond Longitudinal Study (education levels) to create indicators of labor mobility. NCSES should focus in particular on industries that have been experiencing high growth and/or those in which the United States has a strong competitive advantage. Also relevant would be examining skill sets of firms with high growth. Supply of STEM Skills and Talent As noted above, the current NCSES data and indicators provide extensive coverage of college graduates and doctorate holders in S&T. Much less information is available, however, on other important groups. These include individuals with master’s degrees and those who hold degrees from community colleges or have attended community colleges and gone on to earn higher degrees. They also include recent doctorate recipients, a group for which more focus is needed on career choices in the first years after receiving the degree, when finding permanent employment may be difficult. Also needed is reliable, consistent information on STEM education and occupations by different demographic characteristics, such as gender, race/ethnicity, existence of disabilities, and age. Information on numbers and wages of postdoctorates with degrees from U.S. or foreign institutions, whether they work in the United States or abroad, is still incomplete. Needed as well is better information on foreign student stay rates, broken out by race/ethnicity and country of birth. While comprehensive statistics exist on college graduates in S&T, it would be helpful to be able to distinguish better among different levels of educational attainment. Master’s degree holders currently are identified through the NSCG, and many of the statistics on college graduates published by NCSES distinguish between those with bachelor’s and master’s degrees. The panel views this distinction as important and worthy of expanding in NCSES’s human capital indicators. To have a better understanding of the value of master’s degrees, it is important to be able to track the career paths of those who receive them. The same applies to community college degrees. Data are lacking on how many community college degrees are within STEM fields and to what extent their holders work in STEM occupations. Furthermore, because of the rising costs of college education, many students attend a community college for part of their bachelor’s education. The question then arises of whether community college attendance affects the choice of field and the later choice of occupation. NCSES currently uses the National Survey of Recent College Graduates to identify community college degrees and attendance. As that survey is phased out, it will be important for NCSES to collect this information through other surveys, such as the NSCG and the ACS. RECOMMENDATION 6-3: The National Center for Science and Engineering Statistics should enhance indicator coverage of individual science, technology, engineering, and mathematics groups such as early- career doctorate recipients, master’s degree holders, and community college graduates. NCSES already distinguishes between bachelor’s and master’s degree holders in many of its statistics. Stay rates at different education levels PREPUBLICATION COPY: UNCORRECTED PROOFS

OCR for page 109
MEASURING HUMAN CAPITAL 6-15 by demographic characteristics such as gender, race/ethnicity, disability, and country of origin should be included. RECOMMENDATION 6-4: The National Center for Science and Engineering Statistics should explore whether questions can be included in the National Survey of College Graduates and the American Community Survey that would allow the identification of community college graduates or of holders of higher university degrees that have attended a community college. New Resources The employment situation of early-career doctorate holders may be very different from that of more experienced researchers. Members of the former group must decide whether to pursue a research career in academia or employment with a private company. A number of interesting questions arise in this regard that cannot be examined with existing data, such as the main factors affecting the choice between academia and the private sector, types of occupations in the private sector and the Ph.D. skills most relevant for these occupations, and responses to potential difficulties in finding a permanent position at a university or public research institution. Ph.D. dissertations themselves contain valuable information on the areas in which research is being conducted and thus also on the skills and research competencies being developed. Research topics do not always fit well within standard fields of study. They may transcend different fields or represent a new specialized subfield or a combination of topics normally not seen as closely related. To a large extent, a database and the necessary text mining tools with which to address these issues already exist, and with further adaptation and improvement could be used to develop a number of useful indicators on current doctoral work. It would clearly be advantageous if all Ph.D. dissertations were publically available for analysis. Text mining tools could then be developed to identify emerging research topics and to measure the closeness of key research areas. Privately managed services cover a large portion, but not all, of U.S. dissertations. 20 RECOMMENDATION 6-5: The National Center for Science and Engineering Statistics should explore methods for exploiting the full-text resources of dissertation databases to create indicators on selected topics both within and across scientific fields and on the relatedness of different fields. Potential for Reducing the Number and Frequency of Surveys Implementing the recommendations offered in this chapter, as well as others in this report, will require substantial resources. If their implementation is to be feasible, savings will need to be achieved in NCSES’s current activities. The panel identified in particular two areas for potential cost savings. The first concerns further exploitation of the ACS toward the production of S&T statistics. NCSES has adopted the ACS for drawing the sampling frame for 20 The ProQuest Dissertations & Theses Database, for example, contains an extensive number of dissertations, and many, though not all, universities routinely submit accepted dissertations to this database. PREPUBLICATION COPY: UNCORRECTED PROOFS

OCR for page 109
6-16 CAPTURING CHANGE IN SCIENCE, TECHNOLOGY, AND INNOVATION: IMPROVING INDICATORS TO INFORM POLICY the NSCG, an important element in this transition being the inclusion in the ACS of a question on field of study for bachelor’s degree holders. Perhaps the ACS could be used not just to draw a sampling frame, but also as a data source to produce statistics on college graduates. A comparison of the NSCG and ACS questionnaires shows that they solicit fairly similar information on main employer and main job, albeit with some differences. For example, the NSCG contains additional information on college graduates that is not available in the ACS, including (1) the extent to which the principal job is related to the highest degree, (2) reasons for changing jobs, and (3) training and motivation for taking it. Hence, while there are differences in the coverage of relevant topics in the two surveys, there is also a significant degree of overlap. Use of the ACS as a source for statistics on college graduates could permit an increase in the frequency of indicators. Less immediate reliance on the NSCG could allow NCSES to reduce the frequency of data collection for that survey and potentially eliminate it altogether in the longer term. A reduction in frequency, for example, from every 2 to every 4 years would yield substantial cost savings that could be reallocated to many of the new indicators recommended in this report. Second, as mentioned earlier, the panel also views the SED as a candidate for reduction in the frequency of data collection, for the following reasons. Primarily, an increased focus on early careers of doctorate holders would enable the production of statistics on the first job choices of recent graduates. Data on the first jobs of recent Ph.D.’s can be seen as a substitute for data from the SED on postgraduation plans. Data on actual outcomes (first jobs) may also be more useful than data on expectations of employment that can gained from the SED. The sampling frame of the SDR is replenished by the population of newly earned Ph.D.’s from the SED. However, the SED is not needed to update the frame population of the SDR. Basic data on recent Ph.D.’s can be (and in effect already are 21) supplied directly from the awarding universities. Finally, in terms of developing indicators on new Ph.D.’s, the panel believes alternative data sources, such as the ProQuest and WorldCat databases, can prove more useful for characterizing Ph.D.’s themselves. NCSES should conduct a benefit-cost analysis to determine the impact of the reduced frequency of data collection for the SED on NCSES’s staff allocation and the cost of administering the survey. NCSES could analyze the requests of researchers to the NORC Enclave to determine how the SED has been used and the potential extent to consequences of changes in its frequency or coverage for NCSES’s clients. In sum, the panel believes NCSES would benefit from reducing the frequency of the NSCG and SED, instead producing statistics on college graduates based on the ACS. Regarding earned doctorates and the SED, data on the early careers of doctorate holders, particularly their first jobs, reduce the need for data on the plans of new doctorate recipients. In addition, data on the number of earned doctorates can be obtained using other data sources, freeing up resources needed for the development of the new indicators recommended in this report. Demand for STEM Skills and Talent Data are lacking on the demand for human capital. Which types of skills are, or will be, in greatest demand, and which are most important for innovation performance? In terms of S&T employment, the focus is typically on high-tech manufacturing, an industry that, while important, does not appear to be a large source of growth for U.S. jobs and the U.S. economy based on current statistics. A look at trade balances suggests an increasing reliance on imports within these 21 Through the annual Graduate Student Survey. PREPUBLICATION COPY: UNCORRECTED PROOFS

OCR for page 109
MEASURING HUMAN CAPITAL 6-17 industries. In contrast, strong growth is seen in exports of knowledge-intensive services. An examination of education and/or occupation profiles for these industries (or in general, industries with strong growth, particularly with respect to international competitiveness) might indicate what types of skills are in greatest demand and are generating value (and helping to generate new jobs). A number of other data sources can provide insights on the STEM skills in demand now and in the near future, including existing data sources such as the BRDIS and nontraditional data sources such as help-wanted ads (see Chapter 4). Wage levels provide an important indicator of the demand for occupations or skill sets. Wage levels may also influence individuals’ choice of education. The panel thus views wage levels for science, engineering, and health occupations as useful indicators for gauging developments in both supply and demand for S&T-related skills. RECOMMENDATION 6-6: The National Center for Science and Engineering Statistics should consider using American Community Survey data to produce indicators that can be used to track the salaries of science, technology, engineering, and mathematics occupations and/or college graduates receiving degrees in different fields and at different degree levels. The BRDIS covers both R&D and innovation activities, including questions on the number of R&D personnel involved in R&D activities. R&D personnel do not include only researchers but also other employees with a variety of other types of skills and educational backgrounds. This is particularly true when one considers businesses’ innovation activities and not just their R&D. Little is known, however, about the skills sets needed by different types of businesses for their innovation activities. This is a central question entailed in examining whether the skill sets and competencies needed for the development of today’s and tomorrow’s innovations are being acquired. Self-reporting on what industry wants has been shown to be an unreliable proxy for what industry actually does. To address this problem, NCSES could ask firms to identify their most recent innovation and for that innovation, the skill mix of the team responsible. Another way to obtain information on this issue would be to ask firms whether they have unfilled budgeted positions for people with specific skills. Lastly, firms could be asked what skills they would seek if they had to create a small team to develop an innovation they are currently considering. The main point is to ask a question that focuses on recent or current decisions on needed skill sets. RECOMMENDATION 6-7: The National Center for Science and Engineering Statistics should consider adding questions to the Business Research and Development and Innovation Survey on the types of skills sets used by businesses to develop and implement innovations. The results would provide data on and indicators of innovative firms’ demand for skills. The Growth of Online STEM Education To inform STI indicators, NCSES could develop a series of case studies on massive open online courses (MOOCs), described in Box 6-3, to address the following questions: How will the financial structure of universities be affected by distance learning activities, particularly MOOCs? Will MOOCs allow for faster diffusion of innovation PREPUBLICATION COPY: UNCORRECTED PROOFS

OCR for page 109
6-18 CAPTURING CHANGE IN SCIENCE, TECHNOLOGY, AND INNOVATION: IMPROVING INDICATORS TO INFORM POLICY (e.g., trained workers in developing countries that can utilize new products and methods for local concerns)? Will this encourage more inter- or intracountry research collaboration? What types of courses are being offered online? And more important, what is the decay rate for people finishing the course, that is, the share of people who view the material online but do not finish the course, or registrations versus completions? Is there a higher decay rate for YouTube viewing than for sites where students actually sign up for a course, receive course credit, and may engage in on-site course-related activities? BOX 6-3 Massive Open Online Courses: Steps in Progress Massive open online courses (MOOCs) offered free over the Internet could revolutionize science, technology, engineering, and mathematics (STEM) education worldwide. They are descendants of the 1960s Sunrise Semester (United States) and Open University (United Kingdom) courses that offered distance learning to viewers on public television. The Internet allows for worldwide exposure of academic material. The Khan Academy offers free short courses to students from K-12 to college without offline links, credit, or degrees. Academic organizations offer free lectures on the Internet, some on highly abstruse subjects. Many colleges and universities, some for-profit and some nonprofit, offer online courses for credit. Only in the past 2-3 years, however, when leading universities around the world began offering their courses free over the Internet, did it strike many that MOOCs would likely alter the future of education. Some 160,000 people signed up for a Stanford University course in artificial intelligence. More than 90,000 people registered for the first MITx course, and some 100,000 registered for Harvard’s first free courses a few months later. Most registrants drop out of free courses, but enough complete them to mark a massive expansion of the courses’ reach. The growth of MOOCs raises important questions about the best way to combine online and offline education. Online education readily fulfills its promise of great scale at low cost, but otherwise has not proven the panacea that many hoped it would be. Experimentation and research are needed to determine the appropriate mix of face-to-face offline education and MOOCs. Studies of STEM education find that lectures are less effective in getting students to understand science and math than working collaboratively and meeting with instructors and fellow students face to face. Much of the future of education may reside on the World Wide Web, but it appears that the greatest benefits will come from combining this form of learning with other, more traditional ways of approaching material. Examples: UDACITY: “Learn. Think. Do. Higher Education for Free”—formed by the creator of the Stanford artificial intelligence course. As of December 2012, 19 courses were being offered. COURSERA: “Take the World's Best Courses, Online, For Free”—33 universities offering 210 courses, with student meet-ups in 1,020 cities. edX: “The future of online education—for anyone, anywhere, anytime”—nine courses, with universities and courses increasing rapidly. The plan is to offer a sanctioned certificate at a “modest fee.” Participating universities are Harvard; Massachusetts Institute of Technology (MIT); and the University of California, Berkeley. Students around the world who never dreamed of having the chance for an Ivy League-level education can take these courses. The portal for learning is edx.org. SOURCES: Agarwal, 2012; The New York Times, 2012. PREPUBLICATION COPY: UNCORRECTED PROOFS

OCR for page 109
MEASURING HUMAN CAPITAL 6-19 NCSES could undertake three data gathering efforts related to MOOCs. First, it would be helpful to understand the current inventory of these courses and track trends over time. A Google search shows that many courses and many free-standing lectures or lecture series are offered online. Most are free, without course credit being offered. For example, before edX at Harvard University22 existed, the university offered extension course lectures free online; likewise, the Massachusetts Institute of Technology (MIT) had its open courseware available before its MOOC initiative. To obtain an inventory of what exists and related trends, NCSES could conduct an online search similar to the Bureau of Labor Statistics’ effort to construct price indices using web-based data. NCSES could then choose a few specific subjects in different STEM fields, say, differential equations in math, and count those courses on the Internet. NCSES then could obtain figures on downloads of courses that are for credit or not for credit. These numbers would be useful for understanding how pervasive MOOCs are and how important they will be in the future for generating credentialed STEM workers. Another metric NCSES could develop is how frequently people access the online material. For example, MIT’s differential equations course on YouTube had 490,783 views for the first lecture but 26,871 for the last lecture—a decay rate of 94.5 percent. A Khan Academy course registered 569,954 students at first, but 45,090 accessed the last lecture in the series—a decay rate of 92.0 percent. Lewin (2012) reports that the famous Stanford University course in artificial intelligence had 160,000 students from 190 countries enrolled at the beginning in 2011, but just 20,000 successfully completed the course. Stanford’s Machine Learning course had 104,000 registered, and 13,000 completed the course; Introduction to Databases had 92,000 registered, but was completed by just 7,000. NCSES could obtain these data from the universities or firms offering online courses. Data on decay rates would complement those on persistence rates for traditional college degrees. This metric would help university administrators understand the potential impact of distance learning courses on the financial structure of their institutions, or help policy makers understand the impact of innovation in the distribution of knowledge on knowledge diffusion and which countries are the likely benefactors of the distribution of knowledge in STEM fields. NCSES could also link data from administrative records on people taking courses for credit online to LEHD earnings and employment data, NCSES’s SDR data, other data on the STEM workforce, or even patent and publication data (see Chapter 5). For instance, it would be possible to develop a metric indicating the percentage of people who took MOOCs in engineering and changed their trajectory from or to a different field. One could also follow differences in earnings trajectories between traditional bricks-and-mortar and distance learning environments. Careful identification techniques would be necessary, but these data could yield informative analytical output on returns to different types of education investments and organizational structures. This is an example of a project that researchers could undertake using NCSES data along with data from other sources. A variety of users of NCSES’s indicators would be interested in the findings from this research. Therefore, this would be an interesting topic for a box inserted item in NCSES’s reports on indicators of human capital. KEY OPPORTUNITIES Since NCSES’s indicators inform a rich set of topics on human capital, the panel chose to focus on a few of these in this chapter. 22 See https://www.edx.org/ [December 2012]. PREPUBLICATION COPY: UNCORRECTED PROOFS

OCR for page 109
6-20 CAPTURING CHANGE IN SCIENCE, TECHNOLOGY, AND INNOVATION: IMPROVING INDICATORS TO INFORM POLICY First, it is important to obtain better measures of rapid changes in the STI workforce, including job mobility. Worker mobility between jobs, occupations, and nations is important to measure since it indicates how knowledge flows and spillovers occur and how knowledge markets operate. For instance, it is important to know in what industries people with degrees in science—in both STEM fields and the social and behavioral sciences—work throughout their lifetimes. It is also of national interest to have information on the flows of undergraduate and graduate students, postdoctorates, and workers between states and countries. Second, it is important to determine where datasets on STEM education and workforce statistics overlap among the federal statistical agencies. NCSES has long-term relationships with several statistical agencies that provide education data for S&E indicators. Some education indicators are collected through NCSES surveys, while others are reported by NCSES based on statistics generated by other agencies and organizations. In particular, NCSES and NCES both collect data on holders of higher education degrees. Recently, NCSES and NCES staff agreed to undertake a joint gap analysis to examine education data gathering activities. The results of this analysis will enable both agencies, as well as others, to determine where there is useful overlap, where there are voids, and where efficiencies can be achieved by streamlining efforts. A series of workshops on potential linkages, interoperability, and rationalization of datasets on human capital will further improve efficiencies between the two agencies. NCSES will then be able to plan for a greater role than its current position of a data clearinghouse specifically on STI education and workforce data. Third, NCSES has clear near-term opportunities to mine the BRDIS for indicators related to the STEM labor force. The agency already has several activities under way or planned to update its portfolio of education and workforce statistics. For instance, NCSES is rethinking its collection of data on the S&E workforce as the NSCG transitions to a sampling frame built entirely from the ACS, implementing the International Survey of Doctorate Recipients and integrating it into the SDR, developing an early-career doctorate project, and considering earnings of STI workers as a potential new indicator. One possibility would be to develop an S&E wage index, which could facilitate international comparisons and become another explanatory variable for international flows of S&E students and workers. All of these efforts could yield new indicators that would address user needs. SUMMARY In this chapter, the panel has offered seven recommendations, which fall into four categories: (1) rationalization of existing human resource surveys, (2) measures of student and labor mobility that can be developed by using NCSES surveys alone or by linking NCSES data with data from other agencies, (3) measures of industry skill mix revealing demand and supply for STEM talent by sector, and (4) new datasets that can be developed without new surveys. The first priority should be efficiency principles for rationalization of SESTAT datasets along with the development of indicators from the existing, rich longitudinal database at NCSES. Over time, NCSES can collaborate with other agencies to deliver highly useful human capital indicators that link educational inputs to employment and wage outcomes. PREPUBLICATION COPY: UNCORRECTED PROOFS