CHAPTER 6

SUMMARY AND RECOMMENDATIONS

Summary

The committee was charged to make recommendations on the government 's optimal role in forecasting the supply and demand of scientists and engineers, and in particular whether NSF itself should be involved in forecasting and related activities such as data collection. Throughout the workshop, speakers, discussants, and participants addressed a number of salient issues. These issues are synthesized here in the form of five questions followed by the consensus responses that developed throughout the workshop.

  1. Who are the clients for forecasts of demand and supply of doctoral scientists and engineers, and what is it that they really need and want? Two primary groups have been identified: (1) students who are deciding on careers and (2) funding agencies, such as NSF and NIH, that must decide how to allocate funds for traineeships and research assistantships. Other clients include universities that face decisions on the size of research and teaching programs and faculty recruitment and members of Congress who make policy decisions that affect research and the labor market. These other clients need data and forecasts that are subsets of those needed by NSF and NIH.

  2. What is the appropriate scope for forecasts? Of course the answer will be different depending on who the client is, but six questions that help define the scope have been identified.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 49
FORECASTING DEMAND AND SUPPLY OF DOCTORAL SCIENTISTS AND ENGINEERS: Report of a Workshop on Methodology CHAPTER 6 SUMMARY AND RECOMMENDATIONS Summary The committee was charged to make recommendations on the government 's optimal role in forecasting the supply and demand of scientists and engineers, and in particular whether NSF itself should be involved in forecasting and related activities such as data collection. Throughout the workshop, speakers, discussants, and participants addressed a number of salient issues. These issues are synthesized here in the form of five questions followed by the consensus responses that developed throughout the workshop. Who are the clients for forecasts of demand and supply of doctoral scientists and engineers, and what is it that they really need and want? Two primary groups have been identified: (1) students who are deciding on careers and (2) funding agencies, such as NSF and NIH, that must decide how to allocate funds for traineeships and research assistantships. Other clients include universities that face decisions on the size of research and teaching programs and faculty recruitment and members of Congress who make policy decisions that affect research and the labor market. These other clients need data and forecasts that are subsets of those needed by NSF and NIH. What is the appropriate scope for forecasts? Of course the answer will be different depending on who the client is, but six questions that help define the scope have been identified.

OCR for page 49
FORECASTING DEMAND AND SUPPLY OF DOCTORAL SCIENTISTS AND ENGINEERS: Report of a Workshop on Methodology First, should the forecasts be conditional or unconditional? Unconditional means forecasting what is going to happen in the future, all things taken together. Conditional means a what-if analysis; if NIH increased funding for some areas of biology, how would that impact the market? Both kinds of forecasts may be needed, but the distinction must be made. Second, at what level should forecasts be disaggregated? There has been discussion about the need for very disaggregated forecasts, such as the demand for people in informatics. There have been other discussions about the impossibility of disaggregation when there is so much cross-field substitutability that only at an aggregate level does the forecast make sense. Third, what variables should be part of a forecast? The NSF forecasts that have been criticized for concentrating only on numbers of jobs or the supply of job seekers. Other important variables should be considered (1) salaries, which economists identify as the most important element in the operation of markets; (2) the quality of the people that are going into these markets; and (3) the nature of the work that they do, whether it is in bench science, administration, marketing, or whatever. Fourth, how much detail is needed to track career paths and labor market changes by sector of employment? The greatest attention has been concentrated on the academic market, obviously a very important market for scientists and engineers. However, the industrial market is a large and growing market. The level of detail required in forecasts of industrial employment of scientists and engineers has not been resolved. For example, is it useful to produce separate forecasts of demand for biologists in the agricultural and pharmaceutical industries, or does a forecast of overall industrial demand suffice? Fifth, what is to be described? Are you forecasting a series of snapshots of the market in the future? Are you trying to describe the flows and the transitions, or are you trying to tell a story about future career expectations for scientists and engineers?

OCR for page 49
FORECASTING DEMAND AND SUPPLY OF DOCTORAL SCIENTISTS AND ENGINEERS: Report of a Workshop on Methodology Those different goals will have different implications in the establishment of models. Sixth, how should uncertainty be modeled and presented? Many of the variables we have discussed are distributions, such as the distribution of salaries. Making a point forecast of mean salary may not meet all the clients' needs. The dispersion of salaries may need to be modeled. How is information about distributions to be presented, and what is the best way to present information about the inherent uncertainty in the models? Complaints about the quality of the data that are available were a recurring theme during the workshop. What data do we actually have now, and what are their deficiencies? What can be done to improve the utility of the data that we are currently collecting? In addition, what new data are needed? What are the critical gaps in the current data collection programs? Are data access and data documentation adequate? What major steps must be taken to make the data that we currently collect more useful? Finally, what about data management and coordination between different collectors of data? What needs to be changed there? The fourth question relates to modeling issues. To some extent, the variables included, the details of the model, and what is being forecasted will determine much about the model itself, but there are still additional modeling issues. What are the drivers, those things that are outside of science and engineering of which we need to take account? Supply demographics and research and development funding have been emphasized in this regard What variables should be included? Economists talk about prices, salaries, jobs, and the quality of the people in the market. In addition, should job vacancies or underemployment be considered? Economists view these phenomena as temporary symptoms of market disequilibrium that market forces are always

OCR for page 49
FORECASTING DEMAND AND SUPPLY OF DOCTORAL SCIENTISTS AND ENGINEERS: Report of a Workshop on Methodology pushing to eliminate rather than “permanent” characteristics of markets. What should the nature of the model be? Gap models that exclude equilibrating mechanisms (through salary adjustments, changes in job description and tasks, acceleration of research programs, and so forth) and orthodox economic models that (in principle) include these equilibrating mechanisms are perceived as more different than they actually are. Gap models identify the size of the problem that equilibrating mechanisms must handle but often fail to specify which equilibrating mechanisms are operating and how quickly they will work. Users of gap models sometimes make the de facto assumption that equilibrating mechanisms are very slow. Orthodox models often assume that equilibration is rapid without articulating the specific mechanisms that produce equilibrating adjustments or determining that they do indeed work quickly. The resolution of the differences between these modeling approaches requires careful identification and empirical analysis of equilibrating mechanisms and their speed of operation. Regarding model complexity, there is a tendency in all of science and engineering to make things more complex on the grounds that complexity is necessary for realism or accuracy. Certainly forecasting models have become very complex. Considering whether that is the best approach may be useful, especially since forecast users are not economists. Openness in models with some transparency in their structure that allows users to access those models at various levels is also desirable. Having a more realistic but highly complex model may work at cross-purposes with client needs. A good model of this market clearly needs to draw on behavior. People respond to incentives in this market, and we need to understand the quantitative nature of those responses. There are many ways to do this. Economists traditionally prepare a mathematical, econometric model and use historical data to infer

OCR for page 49
FORECASTING DEMAND AND SUPPLY OF DOCTORAL SCIENTISTS AND ENGINEERS: Report of a Workshop on Methodology what the behavioral response parameters might be. Another approach, more like a classical scientific field experiment, examines the impact of an increase in the number of traineeships on Ph.D. completion rates, the behavior of postdoctorals, and so forth. Finally, what is the best administrative structure for forecasting? A fundamental question is whether the government should be making official forecasts in this market. In particular, should NSF and its Science Resources Studies (SRS) group be in the business of making an official forecast? Does an official forecast do more harm than good in encouraging market adjustment? If there is an official forecast, how should it be structured to protect NSF from either the fact or the appearance of political interference? On a related issue, if there is no official forecast, what role should the SRS section of NSF and the government play in forecasting? Data collection on scientific and engineering personnel has been in the governmental domain and presumably will remain there. Should a clear division be made between the agencies that collect data on this market and agencies that are engaged in forecasting or policy analysis? Could this division then facilitate the independence of the agencies involved or maintain the integrity of the data collection efforts? In one model, for example, NSF would not make an official forecast, but similar to blue-chip indicators would present a variety of forecasts prepared by others. Recommendations The forecasting process is ongoing. Forecasters must learn from their mistakes. The whole forecasting exercise needs to be placed within an administrative framework that facilitates an evaluation process and a process to correct errors. The Science Resources Study Division (SRS) of the NSF is the locus of action in the federal government to bring about the improvement of data and forecasts of the market for doctoral scientists and engineers. The

OCR for page 49
FORECASTING DEMAND AND SUPPLY OF DOCTORAL SCIENTISTS AND ENGINEERS: Report of a Workshop on Methodology committee has therefore directed most of its recommendations to SRS, since this division should be able to encourage improvements in the construction and use of forecasting models for highly trained scientists and engineers, even when SRS should not carry out the work itself. Recommendation 1. The producers of forecasts should take into account the variety of consumers of forecasts of demand and supply for scientists and engineers. NSF should recognize that there are five distinct communities of clients for data and projections on the supply and demand of scientists and engineers, each with different needs and interests. NSF's data collection and forecasting activities should keep the needs of these different communities in mind. They are: students making career decisions, federal, state, and private funding agencies, industrial and academic employers of scientists, Congress in its role as funder and policymaker, and the scientific community that conducts research and produces studies of the market and its participants. Representatives of all these user communities spoke at the workshop. All expressed dissatisfaction with the current state of data and forecasting. Students need qualitative projections of likely career outcomes and probabilities of success, with particular attention to the state of the job market in a few years when they will seek employment. These qualitative projections require timely data, but they need not be based on broad surveys or censuses. Research funding agencies and Congress need relatively long-term projections of supply and demand factors by specific discipline that can be used to guide policy on training support and

OCR for page 49
FORECASTING DEMAND AND SUPPLY OF DOCTORAL SCIENTISTS AND ENGINEERS: Report of a Workshop on Methodology institutional development. These projections should take careful account of the ease with which one kind of labor can be substituted for another and the incentives and behavioral responses that operate. The importance of contingencies and of forecast uncertainty should be emphasized. The needs of employers are varied, but they also benefit from an early warning about shortages. As producers of Ph.D.s, academic institutions can take steps to expand or contract Ph.D. enrollment given convincing evidence of emerging labor market trends. Finally, the forecasting community should be more forthcoming about the appropriate use of forecasts and the nature of their underlying assumptions. Recommendation 2. The NSF should not produce or sponsor “official” forecasts of supply and demand of scientists and engineers, but should support scholarship to improve the quality of underlying data and methodology. NSF should not produce or sponsor “official” forecasts of supply and demand in the markets for scientists and engineers, but should continue to take the lead in collecting and making available data on these markets. A clear organizational separation should be made between data collection and modeling/forecasting activities undertaken for NSF's own policy use or for use by federal agencies. For example, convert the SRS into a National Center for Science Statistics on the model of the National Center for Health Statistics (NCHS), the National Center for Educational Statistics (NCES), or the Energy Information Agency (EIA) and remove modeling and forecasting activities to a separate policy unit. Or, for example, forecasts could be produced by an outside agency with statistical expertise. Agencies such as the Bureau of the Census or the Bureau of Economic Analysis may be well suited to undertake such forecasts.

OCR for page 49
FORECASTING DEMAND AND SUPPLY OF DOCTORAL SCIENTISTS AND ENGINEERS: Report of a Workshop on Methodology If asked to produce forecasts of scientific and engineering personnel for its own use or the use of other agencies, the NSF policy unit should avoid endorsing or emphasizing “gap” models that do not incorporate behavioral adjustment to demand and supply and consequently may give unwary users a misleading impression of likely market outcomes. NSF should avoid suggesting that there is a single best level of detail and model complexity for the forecasts needed by various users and should instead maintain that model structure will depend on user needs and objectives. The committee reviewed the history of NSF projections, most notably those in The State of Academic Science and Engineering. In response to the perception that there could be a connection between NSF funding and its projections, especially regarding projections of shortages, the committee believed that NSF should limit itself to data collection and dissemination and use external “arms-length ” forecasts for its policy needs to avoid the possible conflict of interest that might occur if it produced its own forecasts. Recommendation 3: Undertake a comprehensive review of data collection in the light of forecasting needs. NSF's SRS should undertake a comprehensive review of its data management program, preferably in coordination with Bureau of Labor Statistics (BLS), NCHS, and NCES. It should be to seek the production of more timely and useful data on the market for scientists and engineers. In addition, it should coordinate definitions and categories across agencies to facilitate a consistent picture of the different stages in the market, from student training and degree choice to mid-career transitions across and out of science and engineering fields. Moreover, sample sizes have been reduced since the late 1970s, which makes modeling difficult for small fields, specific employment sectors, and for rare events (such as mid-career changes).

OCR for page 49
FORECASTING DEMAND AND SUPPLY OF DOCTORAL SCIENTISTS AND ENGINEERS: Report of a Workshop on Methodology Recommendation 4. Data that enhance forecasts should be widely available and be disseminated on a timely basis. NSF's SRS should establish three high-priority data objectives. These are: (1) production of timely, descriptive statistics on employment and salaries by field of training, occupation, and sector, in a consistent time-series format that permits tracking and projections of trends; (2) production of an individual-level Public Use Sample, containing a consistent time series of cross-sections of doctoral recipients, and when available, nondoctoral recipients; and (3) production of a Public Use dataset panel of scientists and engineers to analyze transitions from the educational system to employment and transitions across fields, occupations, and activities. NSF should process these data, establish rules for access so that they are widely available, and institute less burdensome mechanisms to protect confidentiality. For example, in producing a Public Use panel, NSF might recruit respondents who are willing to provide vitae in a standard format without confidentiality restrictions. This format could be made available on the seb; to make the process even easier for respondents, NSF could code respondent vitae. This would enhance modeling of individual and institutional behavior in response to changes in funding, demographically driven demand, compensation, etc. Standard coding of vitae would permit collection of more detailed data than the Survey of Doctorate Recipients form. Both modelers and policy advisors at the workshop complained that data were not timely. At present, there is no way of knowing in a timely way whether market mechanisms are working to alleviate shortages or gluts. Policies calibrated to outdated evidence may provide too little too late, or too much too late. Furthermore, if indicators show that market mechanisms are impacting rapidly, the need for policy change might be obviated altogether. NSF's/SRS should collect and disseminate data that

OCR for page 49
FORECASTING DEMAND AND SUPPLY OF DOCTORAL SCIENTISTS AND ENGINEERS: Report of a Workshop on Methodology enables a variety of forecasting exercises that differ along some or all of the following dimensions: Unconditional vs. conditional (“What if”) forecasts. Multiple levels of field disaggregation, e.g., physical sciences, physics, solid state physics, materials science. Optional variables to forecast, e.g., jobs, salaries, quality, productivity, occupation. Various sectors to forecast, e.g., academic tenure track/ other, postdoctoral positions, or industrial sectors. What to forecast, e.g., stocks, flows, transitions, or careers. Permit forecasts to go beyond means and standard errors to more complete descriptions of the distributions of possible outcomes. Various time horizons for forecasts. Finally, the NSF's SRS Website currently focuses almost exclusively on providing relatively simple tabulations that are useful for casual policy analysis but not very useful for either career planning by students or for research on the science and engineering market. The data management program and website should be redesigned to service these neglected user communities (or in the case of students, the public and private organizations and associations that provide career guidance) and to provide links to other data sources from BLS and NCES that are important for analysis of the markets for scientists and engineers. Recommendation 5. NSF should develop a research program to improve forecasting. The Directorate for Social, Behavioral and Economic Sciences of the NSF should commission behavioral studies of scientists and engineers early in their careers, as well as studies of forecasting methods and evaluations of past forecasting exercises.

OCR for page 49
FORECASTING DEMAND AND SUPPLY OF DOCTORAL SCIENTISTS AND ENGINEERS: Report of a Workshop on Methodology Particular emphasis should be placed on critical parameters of market response, such as wage-sensitivity of field and occupation switching, and the determinants of the ability of employers to restructure research jobs in response to supply conditions in various fields. Emphasis should be placed on the difficult issues of measuring the quality and productivity of scientists and engineers, the quality of worker-to-job matches, and quality of life for scientists and engineers. This work should be conducted through the ordinary peer-reviewed research support process already in place at NSF. SRS should facilitate the dissemination of results of these studies but should stop short of sponsoring or endorsing specific forecasts or methods.

OCR for page 49
FORECASTING DEMAND AND SUPPLY OF DOCTORAL SCIENTISTS AND ENGINEERS: Report of a Workshop on Methodology This page in the original is blank.