7
The ACS and SESTAT in the Future

The addition of a field of bachelor’s degree question to the American Community Survey (ACS) will quickly, profoundly, and permanently change the prospects for analysis of the science and engineering (S&E) workforce in the United States. It will facilitate analysis of several of the key science and engineering workforce questions directly from ACS data, enable more efficient design of the National Survey of College Graduates (NSCG) and specially targeted surveys, and open the door to targeted surveys of specific subgroups, such as immigrants or dual career households with both spouses in the S&E workforce. Having a field-of-degree question on the ACS will also provide the National Science Foundation (NSF) with a unique opportunity for strategic planning regarding its priorities for studying the S&E workforce. This planning will necessarily involve all of the components of the Scientist and Engineers Statistical Data System (SESTAT) and may involve rethinking the relevant questions to ask.

In this chapter the panel discusses some of the exciting opportunities raised and suggests promising venues of research and analysis of the S&E workforce that will be possible when the ACS field-of-degree information is available.

THE ACS AS A DATA SOURCE

It is important to note that ACS data do not presently contain the rich set of S&E background and outcome variables now collected in the



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 68
7 The ACS and SESTAT in the Future T he addition of a field of bachelor’s degree question to the American Community Survey (ACS) will quickly, profoundly, and perma- nently change the prospects for analysis of the science and engi- neering (S&E) workforce in the United States. It will facilitate analysis of several of the key science and engineering workforce questions directly from ACS data, enable more efficient design of the National Survey of College Graduates (NSCG) and specially targeted surveys, and open the door to targeted surveys of specific subgroups, such as immigrants or dual career households with both spouses in the S&E workforce. Having a field-of-degree question on the ACS will also provide the National Sci- ence Foundation (NSF) with a unique opportunity for strategic planning regarding its priorities for studying the S&E workforce. This planning will necessarily involve all of the components of the Scientist and Engineers Statistical Data System (SESTAT) and may involve rethinking the relevant questions to ask. In this chapter the panel discusses some of the exciting opportunities raised and suggests promising venues of research and analysis of the S&E workforce that will be possible when the ACS field-of-degree information is available. THE ACS AS A DATA SOURCE It is important to note that ACS data do not presently contain the rich set of S&E background and outcome variables now collected in the 8

OCR for page 68
 THE ACS AND SESTAT IN THE FUTURE SESTAT surveys. Nonetheless, the ACS data do contain basic information about the occupations and earnings of people with bachelor’s degrees in science and engineering as well as other fields. With the addition of the field-of-degree data, the ACS information could be tabulated to directly support NSF’s mandated indicator reports. The use of the ACS for this purpose will bring advantages of increased reliability from large sample sizes and significantly improved timeliness, though these advantages are counterbalanced by some loss in detail. The large ACS sample sizes will be particularly valuable when addressing issues of relative employment conditions among rare populations, such as scientists or engineers from groups that have been traditionally under- represented in S&E fields and occupations. This feature is important, given the NSF mandate to monitor the employment status of women, minorities, and persons with disabilities who have college-level training in S&E fields. Recommendation 7.1: The National Science Foundation should use current data from the American Community Survey to evaluate the degree to which the American Community Survey with the field- of-degree question would allow for the production of mandated indicator reports in the future. The ability of NSF to address questions about the S&E workforce beyond the production of indicators will also be enhanced by having the ACS with a field-of-degree question. In the past, analysis of relationships among college major, career choices, and career access were limited to snapshots provided by occasional large surveys. A sufficiently detailed field-of-degree question will enable NSF to track the employment status of rare populations of bachelor’s level scientists and engineers, with rela- tive levels, trends, and fluctuations in both employment and earnings on a continual basis, using the ACS alone. Because the ACS is a household survey, it further expands the types of analyses that can be done with respect to the S&E workforce. For example, the study of dual-career households is possible with existing ACS data, but only for groups defined by occupation and education level. The field-of-degree question on the ACS will add a new dimension, allowing special consideration of dual-career households in S&E fields. The household data will provide additional analytic opportunities as well, especially with in-depth household information not currently collected on the NSCG. The timeliness of ACS data will provide annual information on the S&E workforce close to the reference period. These more timely data pro- vide NSF with a powerful new ability to analyze the effects of real-time

OCR for page 68
0 USING THE AMERICAN COMMUNITY SURVEY events on the college-educated population. For example, does a particular labor market fluctuation have different effects on workers with different fields of college training? The large sample sizes and ongoing design mean that the ACS data will be useful to many researchers interested in understanding different aspects of the S&E workforce over time. For example, what are the effects of recent immigration on the S&E workforce? Are the effects different in different fields? Why do gender differences in career choices persist, and are there places or time periods in which these differences narrow? The ACS could also be used in other ways to improve the timeliness and relevance of the S&E workforce information. For example, the ACS could provide a frame to do more frequent (and smaller) special topic surveys on topics and groups of special interest. Moreover, the Census Bureau is considering adding supplemental questions that appear one time or rotate in and out of use on subjects of current interest for the whole sample or for subpopulations of special interest. Recommendation 7.2: If the American Community Survey is selected to produce indicator reports, the National Science Foundation and the Census Bureau should develop a supplemental program of spe- cial, targeted surveys to obtain information on topics and groups of interest. ACS EFFECTS AND SURVEY DESIGN Chapter 6 details how the ACS with a field-of-degree question as a sample frame could positively affect the NSCG. As noted, the ACS offers the ability to allocate resources efficiently in designing the sampling frame for the NSCG’s more detailed questions. Not only can people with sci- ence or engineering college majors be oversampled from the ACS frame, it will be also possible to identify specific majors from within this group for particular attention. In thinking beyond the improvements that can accrue to the current NSCG, the ACS itself could be used to suggest improvements to NSCG content or to suggest specific targeted surveys. In the future, issues that cannot be identified with data from the Current Population Survey (CPS) or the decennial census data should be visible from the ACS in time to allow each NSCG wave to add new questions or to target particular groups to address current policy concerns. For example, if it is noted that a given economic contraction has strong effects on workers with a particu- lar undergraduate major, such as when computer sciences were affected by the technology sector retrenchment in the early 2000s, that group could be oversampled in the following wave of the new NSCG. Similarly, if

OCR for page 68
 THE ACS AND SESTAT IN THE FUTURE large changes in migration or immigration in one scientific field appear, those changes could be monitored continuously in new waves of ACS data, and relevant questions could be added to the new NSCG to fill in the details. In the same vein, the ACS can help identify dual-career S&E families and provide information to design a supplemental sample to the NSCG to provide deeper information on dual-career issues. The avail- ability of these data will enhance the ability of NSF to provide a timely picture of a wide variety of emerging workforce issues. In much the same way, the ACS lends itself to serving as the basis for drawing samples of subgroups of interest in order to test and evalu- ate questions (much as is currently accomplished through the content test program) and sample design and methodology improvements. Such testing and evaluation of NSCG content and methodology is especially natural if NSF and the Census Bureau adopt the panel’s recommendation for a rotating panel design. Use of the ACS for the NSCG affords NSF a unique opportunity for continuous improvement. An ongoing program of developing carefully crafted experimental panels would provide the basis for testing the next generation of advances in collection and estimation methodology for the NSCG (and other surveys of its kind). Recommendation 7.3: The National Science Foundation and the Cen- sus Bureau should consider establishing a continuing experimental panel program to support testing and development of techniques and methods for the National Survey of College Graduates. LINKING CENSUS, ACS, AND NSCG DATA There are many reasons for researchers to want NSCG data that are linked to census and ACS data for the same person. Such a linkage was successfully implemented in a match of the 1993 NSCG with informa- tion from the 1990 census. The decision of how much of the linked data to release involves tradeoffs between the competing goals of produc- ing data that can be used for meaningful statistical analysis, protecting the confidentiality of participants, and avoiding the necessity of asking participants to answer the same questions they have already answered on a previous survey. Linking data from the NSCG to outside sources provides an efficient means to study labor market dynamics on short time scales and to understand how NSCG respondents compare with other college graduates. It would be particularly useful to be able to link NSCG responses to common labor market measures, including occupa- tion (using census categories), previous year’s earnings, and the number of weeks and hours per week worked in the previous year. Demographic

OCR for page 68
 USING THE AMERICAN COMMUNITY SURVEY information is easier to collect in a new survey wave, but detailed census demographic measures (race, ethnicity, immigration status, and timing of immigration) are valuable in some studies. Finally, it is vital in some analyses to have access to the census variables that are considered in the decision to include people in the NSCG sample, e.g., educational attain- ment at the time of the ACS interview, age, sex, and broadly aggregated race and ethnicity. The panel recognizes that the data involved in the linking operation are highly confidential and that access must be carefully controlled by the Census Bureau to ensure that the data are protected. Today, such protec- tions are afforded through the Census Bureau Research Data Centers. In the future, alternative means of improving access to the data in a manner that assures that data confidentiality is protected, such as data enclaves, may be judged adequate by the Census Bureau. Recommendation 7.4: The National Science Foundation should sponsor the development of a matched sample of American Com- munity Survey and National Survey of College Graduates respon- dents for research purposes with access provided to researchers through the Census Bureau’s Research Data Centers. STRATEGIC PLANNING FOR SESTAT The field-of-degree question on the ACS, in addition to using the ACS for sampling purposes, provides a unique opportunity for NSF to engage in strategic planning for the SESTAT system. Therefore, it seems appro- priate to examine each element of the SESTAT data system to determine how to best integrate and configure data collections and optimally expend available resources given the available resources. The bulk of the panel’s report addresses integration of the ACS and the NSCG. In this section the committee discusses the potential effects of the ACS on the other SESTAT components. Science and Engineering Doctorates The stock and flow of science, engineering, and health doctorates are well covered by the Survey of Doctorate Recipients (SDR). The SDR has great value as a stand-alone survey, enabling longitudinal analysis of the careers of these doctorate holders. Previous sample frame research conducted for NSF has recognized that the SDR is different, with separate sampling justified by the desire to increase the sample of earned doctorates above the small number of U.S. doctorates in the NSCG and to elicit comprehensive information about

OCR for page 68
 THE ACS AND SESTAT IN THE FUTURE this group (Fecso et al., 2007a, p. 4). Currently, those with U.S. doctor- ates contained in the NSCG are not included in the SESTAT integrated database; people with U.S. doctorates are drawn from the SDR survey. Only those sample cases in the NSCG who have doctorates from institu- tions outside the United States are included in the SESTAT integrated database. The use of the ACS as a frame for the NSCG is unlikely to change the value of and need for the SDR survey for several reasons: • The ACS will not provide a sufficient sample of doctorate recipi- ents unless multiple ACS years are combined, and such a combin- ing of multiple years would nullify many of the quality enhanc- ing features of using the ACS. • The desire for small-domain estimates for these people (e.g., doc- toral field by race or ethnicity by sex) and the ready availability of the Survey of Earned Doctorates (SED), a census of all U.S.-earned science, engineering, and health doctorates, for a sampling frame makes continued use of a separate SDR survey a very efficient approach for the doctorates of interest. • The ACS does not contain information about the date a person receives a degree. Data on age, which are available from the ACS, are not a good proxy for determining year of degree. Data from the 2003 National Survey of Recent College Graduates (NSRCG) show that more than 50 percent of bachelor’s degree recipients in science, engineering, and health in 2001 and 2002 earned their degrees when they were 25 years of age or older, and more than 50 percent of master’s degree recipients in those fields earned their degrees when they were 30 years of age or older. Therefore, age is an inefficient indicator for selecting recent graduates. Recent College Graduates The NSRCG has twin objectives. One, discussed in Chapter 2, is to generate data to inform the public of the flow of new bachelor’s and master’s degree recipients (from U.S. institutions) into a science, engi- neering or health field. NSRCG data are useful for employers and gov- ernment to understand and predict trends in graduate school enrollment, employment opportunities, and salaries for recent graduates in S&E fields. The NSRCG provides direct information about the employment and continuation into further education of recent bachelor’s and master’s recipients in those fields. Another objective of the NSRCG data is to serve as a key component of the SESTAT system, in that it provides the flow of new U.S.-educated science, engineering, and health bachelor’s

OCR for page 68
 USING THE AMERICAN COMMUNITY SURVEY scientists and engineers to add to the stock of scientists and engineers from the NSCG. The availability of annual ACS data with field-of-degree informa- tion raises the question of how much the ACS and the NSRCG overlap. Clearly, there is some overlap between the NSRCG and the ACS (with a field-of-degree question) in that the ACS will naturally incorporate recent graduates, but there are differences, too. For example, the ACS alone has no questions that can directly discern recent college graduates or those with a science, engineering, and health master’s degree. The advisability of implementing a new ACS-based approach to cap- ture the population currently covered by the NSRCG must take into account not only the relative importance of the two functions that the NSRCG currently serves in the set of SESTAT surveys, but also techni- cal issues. For instance, if the rotating panel sample design option rec- ommended in this report is adopted, it would be possible to fulfill the second function of the NSRCG with the NSCG alone because the flow of new degree recipients would occur naturally. It would also be possible to fulfill this function if subsamples from the ACS were drawn with some frequency to capture new graduates. (It should be kept in mind that data on recent college graduates need to be collected with some frequency because the workforce experience of this group is particularly sensitive to labor market conditions.) However, the use of the ACS to identify recent college graduates would not fulfill a primary function of the NSRCG— providing detailed information on this population. The NSRCG is now used to make estimates for small domains, such as recent college gradu- ates by field, race and ethnicity, and gender. Using the ACS frame to produce direct information about recent grad- uates would require a larger sample for the NSCG and oversampling (and screening) to identify a sufficient number of recent graduates to derive estimates that approach the precision of those currently yielded by the NSRCG. The ACS does not have information on the year of a degree, so to maintain the current coverage of the NSRCG, NSF would have to sub- stantially oversample older people whose highest degree was a bachelor’s or master’s degree. Most of these older graduates would not have recent degrees. To avoid such costly and inefficient oversampling would require an undesirable revision in the scope of the NSRCG. Currently, the NSRCG samples approximately 9,000 graduates from each graduating class (about two-thirds from bachelor’s degree recipients and one-third from master’s degree recipients). Obtaining a sample of this size with appropriate demographic oversampling may be difficult to achieve with 1 or 2 years of ACS data, even with a field-of-degree ques- tion. For sampling purposes, the NSRCG obtains reasonably accurate degree data from colleges and universities of recent graduates in science,

OCR for page 68
 THE ACS AND SESTAT IN THE FUTURE engineering, and health in order to target sampling of recent graduates. Although only limited demographic information on each school is avail- able as part of the sampling characteristics that are used, there is sufficient information on past graduates and each school’s profile to target the pop- ulation accurately. In addition, schools are able to provide relatively cur- rent contact information on recent graduates, along with detailed degree information. The foregoing discussion suggests that there would be some benefit to considering how the ACS can be used to improve the efficiency of the NSRCG as a part of an overall reconsideration of the design of the SESTAT data system. Recommendation 7.5: The National Science Foundation should use the opportunity afforded by the introduction of the American Com- munity Survey as a sampling frame to reconsider the design of the Scientists and Engineers Statistical Data System (SESTAT) Program and the content of its component surveys. FUTURE OPPORTUNITIES The previous section discusses gains in efficiency and analytical power for understanding the S&E workforce as currently defined that arise from the addition of the field-of-degree question on the ACS. In this section the committee discusses the potential of the ACS with field-of-degree question to contribute to a rethinking of the economic concept of the S&E workforce and the science of measuring and tracking this workforce. In a previous review of NSF’s efforts to track the national infrastruc- ture of human capital in science and technology, Kelly et al. (2004) raised several issues that should be taken into account in developing a data system that would enhance understanding of the science and technology enterprise.1 One major point that the authors raise is that the composi- tion of the human capital pool used by the S&E sector reflects choices that are made by both firms and workers. Based on the remuneration in other sectors and the costs of other productive inputs, S&E industries will change the field of study and the intensity of training they require for the employees they hire. As wages in different sectors change and rise or fall relative to one another, workers may switch out of or into S&E sectors. Given these employment flows, the S&E employment pool will change over time. 1Although the Kelly et al. (2004) study addressed what they characterized as “science and technology,” it is a concept very similar to what is characterized in this report as “science and engineering,” and the basic arguments apply.

OCR for page 68
 USING THE AMERICAN COMMUNITY SURVEY With limited resources, NSF has to date appropriately focused on a clearly identifiable and highly relevant slice of the labor force—those with bachelor’s or higher degrees in S&E and S&E-related fields. To its credit, NSF was able to expand the scope of the NSCG to include non-S&E fields in recent years but much more needs to be done to collect information on a larger part of the workforce in more detail and in a more timely manner. NSF does not currently have the capability to inventory S&E skills in all workplaces, nor is it apparent that SESTAT can produce what is and will be desired to analyze the patterns and trends that are embedded in the many choices made by workers and employers. The availability of the ACS with field-of-degree information offers a breakthrough opportunity for gaining a better understanding of the total S&E enterprise and all of the national labor force resources that contrib- ute to innovation and technical change and the economic growth they generate. This requires thinking expansively about the S&E workforce. Occupations and training levels that might have once put a worker unam- biguously inside or outside the S&E sector may no longer serve such a discriminant function. For example, a narrowly trained technician from a community college may, under the current definition, be counted as a science and technology worker if working in the right industry. It may not take a remarkably higher wage to move this worker from a job in the science and technology sector to one in retail trade, in a technical support position, and hence no longer a technology worker. Similarly, if that same worker is employed in retail trade as a support person and hence not a technology worker, a change of sector as a result of a higher wage offer could move that person into classification as a technology worker. NSF’s ability to study the S&E infrastructure in the United States using the expanded concept of a large potential S&E workforce would be enhanced with the availability of additional descriptive data, such as data on earnings. This larger view of the workforce and associated data would permit better analysis of such issues as the S&E worker shortage hypothesis. A professed shortage of workers alongside a decline in their earnings relative to workers with similar levels of education in different fields would help understanding of whether a perceived shortage of S&E workers was localized to a field, part of a more widespread phenomenon, or reflected not a shortage but the presence of better opportunities in dif- ferent industries. The ACS and, as discussed below, the CPS offer data on workers and earnings that provide such additional analytic opportunities for studying the condition of the S&E labor market. The availability of the ACS with field of degree will finally allow ana- lysts to explore the dynamism of the concept of the S&E workforce. Most current analyses rely on comparing workforce participants based on the occupations they hold, leading to what is sometimes known as choice-

OCR for page 68
 THE ACS AND SESTAT IN THE FUTURE based sampling. That is to say, the determination of whether someone belongs to a category is a result of choices made by that individual. The statistical properties of such samples can be somewhat complex. If NSF oriented its sampling around degree and field to define S&E workers, then NSF would be able to devote resources to sampling a larger number of potential S&E workers. (As the panel notes above, degree and field also reflect individual choices; however, once a degree is earned, the possession of that degree and its fields are characteristics of an individual that do not change in response to market dynamics.) The resulting ability to broadly focus on fields without the distraction of current occupation will enhance understanding of how changing relative wages and chang- ing demands in the markets can change the number and composition of S&E workers and, in the process, help to explain innovation. The ACS could also be used to generate a very powerful longitudinal study capacity, much more robust than the current limited longitudinal capacity of the NSCG. At the present time, the ability to track the move- ment of workers through the labor force is the province of such studies as the National Longitudinal Survey, the Panel Study of Income Dynam- ics, and the New Immigrant Survey. These sources certainly help under- standing of the dynamics of all workers, but their limited sample sizes and scopes circumscribe their ability to understand the dynamics of the S&E workforce. The SED and similar data resources help understand the flow of highly trained people into the labor force, but they lack the panel attribute that would enable them to contribute to an understanding of dynamics. A small, well-executed panel survey of college graduates below the doctoral level, drawn from those identified in the ACS, would aid in understanding the labor force dynamics of highly trained workers, even if it did not have the large sample size needed to explore these dynamics in detail for all demographic categories. If supplemented with a longitu- dinal sample of immigrants with advanced degrees, obtained with the cooperation of the Customs and Immigration Services, a comprehensive understanding of the dynamic labor market would evolve. In the process of moving away from the selection of respondents based on occupations that are choice-based (that is, influenced by current and potential earnings and training cost considerations), cross-sectional, general-purpose surveys such as the CPS could be enlisted to provide a broad picture of the employment status for scientists and engineers and also make direct comparisons with other fields. As part of the evolution from a focus on occupations to a focus on fields, it would be useful to enhance the ability of the CPS to supplement the new information from the ACS on a more current basis and contribute to the study of the S&E workforce by adding a field-of-degree question to the survey. The addi- tion of a field-of-degree question to the CPS would enable NSF to assess

OCR for page 68
8 USING THE AMERICAN COMMUNITY SURVEY S&E workforce trends on a monthly basis in the same manner as the aggregation of ACS cases will provide on an annual analysis, although it is recognized that the CPS will not provide the sort of detail on S&E workers by disability and gender that the current mandates require and thus will not substitute for the ACS. In sum, the ACS is a promising option for efficiently fulfilling a major part of NSF’s mandate. However, the ACS affords much more than an opportunity to do business-as-usual better. It provides an opportunity to repurpose SESTAT to overcome some of the current conceptual limitations that cause analysts to view S&E workers as a well-defined and time- invariant segment of the labor force. The ACS would permit measuring all workers who could, at some set of wage rates and product demands, become S&E workers. In the process, NSF would be able to fulfill its man- date to understand the S&E labor force in a more efficient and timely man- ner that would release resources to invest in more policy-relevant work. CONCLUSION The ACS with a field-of-degree question can affect the mission of NSF with regard to S&E workforce data; indeed, the ACS with field-of-degree information may eventually provide much of the data needed by NSF to produce its mandated reports. If so, then NSF would be afforded a unique opportunity to redesign SESTAT in support of innovative analy- sis to enhance understanding of the key issues regarding S&E human resources. A redesigned SESTAT may include its current components, such as the NSCG, for which the ACS could have a large impact as a more effi- cient sample frame; or SESTAT may become more reliant on the ACS, supplemented by a series of targeted surveys based on trends visible with ACS data, or it may include both. A redesigned SESTAT may even integrate CPS (if a field-of-degree question were added) to provide timely information on income data to frame the important questions of the S&E workforce. It is not too early to begin thinking about the many exciting possibilities that are emerging with the inclusion of a field-of-degree ques- tion on the ACS. Recommendation 7.6: The National Science Foundation should conduct a careful assessment of internal and user priorities for studying the science and engineering workforce to capitalize on the expanded analytical opportunities afforded by the addition of field- of-degree question to the American Community Survey.