Measuring the Science and Engineering Enterprise: Priorities for the Division of Science Resources Studies (2000)

The Division of Science Resources Studies (SRS) of the National Science Foundation is one of fourteen agencies in the federal statistical system, as represented on the Interagency Council on Statistical Policy. Like these other federal statistical agencies—such as the Bureau of Labor Statistics, the Bureau of the Census, the National Center for Education Statistics, and the National Center for Health Statistics—SRS is charged with providing data and analysis on a broad policy area for public- and private-sector constituents. The activities of SRS, authorized by the National Science Foundation Act of 1950 as amended, are important for informing science and technology policymakers, managers, educators, and researchers.

To meet the information needs of its constituents in science and engineering, SRS collects and acquires data on national patterns of research and development (R&D) funding and performance and on the education and careers of scientists and engineers. The division publishes tabulations and analyses of data in these areas and makes its data sets publicly available for analysis by others. SRS also supports the National Science Board in the production of the biennial Science and Engineering Indicators. This latter volume, a congressionally-mandated report, is a key reference for science and technology policymakers and analysts.

Federal statistical agencies must continuously monitor developments in the broad subject area about which they collect data to ensure that they are providing information that addresses policy and research needs. Substantial recent developments in both the science and technology policy context and the science and engineering enterprise suggest that the need for SRS to review and update its portfolio is especially important at this time. These trends, some of which have been in progress since the early 1980s, include:

• An increase in industry's share of R&D funding in the United States from about half in 1980 to almost two-thirds today. This shift is largely due to decreases in federal funding for development activities and a high growth rate in industrial R&D spending.

• An increase in service sector R&D which made up just 5 percent of industrial R&D in the early 1980s and now accounts for almost a quarter of it.

• The role of technological innovation in the apparent resurgence in American industrial performance during the 1990s though this is no better understood or quantified than the apparent decline in American industrial performance during the 1970s and 1980s.

• Changing patterns of federal research and development spending following the end of the Cold War characterized by decreases in defense R&D and an acceleration of the long-term growth of federal spending on biomedical research.

• A difficult job market for many new science Ph.D.s in some fields in the early 1990s, characterized by increased numbers of recent Ph.D.s in postdoctoral positions and increased numbers of others working outside of their fields.

• Substantial increases in the number of Ph.D.s awarded in the United States since the mid-1980s, due largely to increases in the number of non-U.S. citizens earning Ph.D.s from U.S. institutions.

• An increase in the percentage of Ph.D.s in science and engineering holding positions outside of educational institutions, with more than half of Ph.D.s so employed for the first time in 1991.

• Trends suggesting an increasing number of intra- and inter-sectoral alliances in research and development.

• Anecdotal information suggesting that an increasing percentage of research, especially ''cutting-edge" research, is inter- or multidisciplinary in nature.

• The emergence of new fields, such as biotechnology and information technology, that have changed our perceptions of the way scientific research is conducted and translated into practical innovations.

• The emergence of a global economy and the globalization of the science and engineering enterprise.

In light of these and other developments and their potential for raising important policy issues, SRS asked the National Research Council (NRC) to undertake a review of the SRS portfolio of data collection, acquisition, and analysis activities and to assist SRS in revising these activities to meet the information needs of policymakers, managers, educators, and researchers. The scope of this study was developed through discussions among SRS, the NRC's Office of Scientific and Engineering Personnel, and the NRC's Committee on National Statistics.

The Committee to Assess the Portfolio of the Science Resources Studies Division of NSF was charged with identifying gaps in NSF surveys and providing prioritized recommendations for addressing those data and information gaps. Potential means for filling these gaps could include recommending new questions for existing surveys, advocating the development of new surveys, suggesting ways for SRS to combine its data with those from other sources in creative ways, and proposing new approaches to data analysis that would benefit those who need these data and information. Other ways to address such gaps include in-depth study of emerging issues to better understand these phenomena and the data about them that would be useful to collect in the future. The committee was also charged with looking for SRS activities that are obsolete and could be discontinued.

The committee convened for this study included experts in a range of fields. The committee members brought expertise in R&D economics, labor economics, graduate education, statistics, and survey methods. They also brought experience in federal

science and technology policymaking, managing a federal statistical agency, administration of graduate education, directing industrial R&D, and overseeing university-sponsored research programs. The disciplinary backgrounds of the committee included social sciences, life sciences, physical sciences, and statistics.

Given the breadth of its charge and the short time frame for this study, the committee sought to identify key science and technology resource issues to guide its recommendations in an efficient, yet informative, manner. The input for this study, gathered in 1998, included:

• presentations to the committee in April 1998, by the SRS Division Director and senior SRS managers on their operations and data collection challenges

• individual interviews with these managers

• focus groups with SRS staff in the Research and Development Statistics Program, the Human Resources Statistics Program, the Science and Engineering Indicators Program, the Integrated Studies Program, and the Information Services Group

• the reports of SRS customer surveys in 1994 and 1996

• interviews with 42 individuals in government, industry, and academia to solicit their views on current issues in science and engineering and the data available to address these issues

• a two-day NRC conference entitled "A Workshop on Data to Describe Resources for the Changing Science and Engineering Enterprise," held in Washington, D.C. on September 18–19, 1998

• the conclusions and recommendations from a number of recent reports on issues in science and technology policy

In addition, the committee relied on its own members with extensive experience in research and development, graduate education, and the analysis of science and engineering labor markets to assess the importance of various issues raised in the presentations, focus groups, interviews, the two-day conference, and national reports. The committee also looked at recent data trends to substantiate the importance of issues raised. It then selected for treatment in this report those issues raised through this process that it believes are of the highest priority for SRS to address.

The process of updating the portfolio of a federal statistical agency is an ongoing and iterative process. The committee, therefore, also set out to provide SRS with guidance on how to organize both for investigating the issues identified as discussed above and for continuous renewal thereafter. Despite a long history of federal data collection, dating back to the provision for a decennial census in the U.S. Constitution, the literature on criteria for the effective operation of federal statistical agencies is small. The committee found the Committee on National Statistics' Principles and Practices for a Federal Statistical Agency useful in outlining issues in this area (NRC 1992). However, the literature specifically on the relationship between a federal statistical agency and its data users in the process of revising and updating a portfolio of data collection and analysis activities is even smaller. Here, the committee relied on the insights of its members who have served in or worked with federal statistical agencies, as well as those of its members who have substantial experience as users of federal

statistical data. The committee selected as its highest priorities for SRS two recommendations that focus on strengthening the dialogue and interactions between SRS and its data users.

In Part I of this report, the committee discusses SRS as a federal statistical agency. Chapter 2 reviews the evolution of the SRS portfolio and relates its current operations to standards for federal statistical agency practices. In Chapter 3, we provide recommendations to SRS for a process that will promote an ongoing interaction between SRS and its data users, allowing SRS to harvest information on the issues for which its customers need data. This process is designed to allow SRS to continuously monitor and revise its portfolio. The committee also recognizes that in some instances, better integration of existing survey data rather than collection of new data may be the most effective way to meet unfilled data needs. Thus, we strongly recommend that SRS seek to link data sets where possible. We also discuss in Chapter 3 the need for SRS to improve the timeliness with which it makes its data available to the public.

In Part II, the committee addresses current issues in graduate education, the science and engineering workforce, and research and development funding and how additional understanding of or new data on theses issues could support policymakers. Chapter 4 discusses current policy issues regarding graduate education, the transition to employment for new Ph.D.s, career paths of scientists and engineers, and the international flow of scientists and engineers. Chapter 5 discusses the changing organization of R&D in the United States and globally and then examines specific issues dealing with industrial research and development and the allocation of federal resources for science and technology.

A summary of conclusions and recommendations is presented in Chapter 6.