Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 7
Data on Federal Research and Development Investments: A Pathway to Modernization 1 Introduction Two surveys of the National Science Foundation (NSF)—the Survey of Federal Funds for Research and Development (the federal funds survey) and the Survey of Federal Science and Engineering Support to Universities, Colleges, and Nonprofit Institutions (the federal support survey)—provide some of the most significant data available to understand federal research and development (R&D) investment trends and patterns, illuminating science policy in the United States. Indeed, theses two surveys are building blocks for virtually every analysis of publicly sponsored U.S. scientific and technical activity. They are used by government, academia, industry, and a host of nonprofit analytical and advocacy groups as the primary source of information about federal spending on research and development. For example, they are used by the National Science Board as a basis for its statutorily mandated biannual report on science and engineering (S&E) indicators. In addition, they are one of the primary sources for the analyses of the federal R&D budget prepared regularly by the American Association for the Advancement of Science and other organizations (National Research Council, 2005b, p. 102). Indeed, only the federal budget documents issued by the Office of Management and Budget are as important a source of information on federal R&D spending. The surveys are used to help reach conclusions about important and fundamental policy questions, such as whether a given field of research is being adequately funded, whether funding is balanced among fields, whether deficiencies in funding may be contributing to a loss of U.S. scientific or economic competitiveness, and which agencies are most important for the health of a scientific discipline.
OCR for page 8
Data on Federal Research and Development Investments: A Pathway to Modernization Users of the survey results typically consider the information to be straightforward, accurate, and complete. However, none of these descriptors is quite the case. Federal agencies that report their R&D spending to NSF treat the surveys with differing degrees of attention to timeliness and accuracy. Some agencies periodically change their internal classifications and the ways in which they account for R&D spending, so the data have problems at their source.1 Even more problematically, the surveys ask for information in categories that are not used by all agencies for their own internal purposes, so the information provided to NSF is often a rough estimate, frequently based on unexamined assumptions that originated years earlier. A key component of the reporting of federal R&D spending is the fields of S&E taxonomy, and Macro International recently conducted a study for NSF of its use by federal agencies (Macro International, 2008, p. 5). The study found that some of the major R&D agencies estimate spending by fields of science using staff judgment calls, rules of thumb, percentage distributions, or mapping of the codes to the agency’s plans or organizational structure. Some of the agency decisions made in these ways are rather arbitrary. For example, for purposes of federal funds survey reporting, the Centers for Disease Control and Prevention reports its entire research portfolio under one category, Life Sciences–Medical Sciences (Macro International, 2008, p. 7). With these issues and others in mind, NSF asked the Committee on National Statistics of the National Research Council to review the two main surveys that are used to collect data on federal R&D spending and to consider ways to improve their accuracy and timeliness. Accordingly, the Panel on Modernizing the Infrastructure of the National Science Foundation Federal Funds Survey was established to consider the uses of the NSF federal R&D spending data and, in view of those uses, the quality of the data on federal funds for research and development and to recommend future directions for the program. The panel was asked to include the fields of science classification structure underlying the Survey of Federal Funds for Research and Development in its review. In approaching this task, the panel has reached out to both senior officials of federal agencies that provide the federal funds data and key data users, solicited advice from providers of complementary and competing data sources, and reviewed past studies on federal funds data. This report, with recommendations on modernizing the infrastructure of the survey, is the primary product of the study. 1 For example, in fiscal year (FY) 2000, the National Aeronautics and Space Administration reclassified as research activities that had previously been classified as development, and in FY 2004 the agency implemented a new budget approach. Both actions introduced major discontinuities in the R&D data series (National Science Foundation, 2008a, p. 6).
OCR for page 9
Data on Federal Research and Development Investments: A Pathway to Modernization CONTEXT FOR THIS REVIEW The work of the panel took place during a dynamic time for the U.S. government’s S&E policy. In recent years, there has been increasing concern about the role that federal investment in research and development plays in generating innovation and concomitant growth in the U.S. economy. In view of the perceived interconnection between investment in R&D and innovation, there is strong interest in the adequacy of that investment. Furthermore, the focus of top-level decision makers has shifted in recent years from concern about the supply of new knowledge in particular fields and disciplines to a concern about meeting the demands for knowledge to help resolve critical societal challenges, such as infectious diseases, climate change, energy, and food safety. The data now collected by NSF do not readily allow analyses to illuminate these sorts of questions. Moreover, these concerns are prompting research managers and analysts alike to refocus attention on the metrics that describe the S&E enterprise. The users of R&D data are now raising larger, longer term questions about how to develop a suite of data that would better inform policy debates without losing the information and historical record encapsulated in the two NSF surveys. More specifically, the changing research environment is leading to questions about whether current measures adequately capture the increasingly multidisciplinary nature of research, whether the current taxonomy of fields of S&E accurately describes the research landscape, whether the old division of S&E activities into basic and applied research and development makes for useful categories, and whether the data as now collected permit a comprehensive analysis of the role of the federal government in innovation and growth. The ultimate goal of data collection on R&D funding should be to enable science policy researchers to draw a much richer picture of federally funded research and its connections to economic growth and other societal goals. That would include being better able to connect research inputs with outcomes, slice the spending data in many different ways, and understand the links among researchers in academia, government, and industry. Growing Interdisciplinary Research One concern, for example, that has both immediate and long-term consequences, is the inability of the surveys to account for the growing trend toward interdisciplinary (or cross-disciplinary or transdisciplinary) research. There is a growing belief that demand-driven, problem-solving R&D is often interdisciplinary in character, and there has been a growing discussion in the scientific and science policy communities about whether
OCR for page 10
Data on Federal Research and Development Investments: A Pathway to Modernization such work is being sufficiently promoted or possibly even discouraged (National Research Council, 2005a). It is also important to understand the trends in interdisciplinarity: That is, whether research is most fertile at the boundaries of disciplines, which appears to be the basis for many federal programs seeking to encourage such research, or whether interdisciplinary change is occurring within the core of contemporary disciplines in ways that fundamentally change them. The federal R&D funding and support surveys would be an obvious place to try to get data to help understand these questions, yet the current surveys are likely to obscure the matter by forcing investments in interdisciplinary research to be reported either within a single field or in a miscellaneous (“not elsewhere classified”) category. The difficulty of portraying the growing interdisciplinary nature of federal R&D is exacerbated by the fact that much of the interdisciplinary work takes place across agencies as well as across disciplines. Climate change research is an example. It is important that, to the extent possible, these cross-agency R&D initiatives be described in the same way, so they can be identified and aggregated so as to give a view of the totality of the investment throughout the federal government. Failure of the Fields of Science and Engineering Taxonomy to Describe Research and Development in Useful Ways Another issue at the heart of the problem is that the surveys provide little help in drawing connections between the research agenda and either public goods or industrial innovation and competitiveness. Most federal agencies manage programs that are defined by categories related to topic areas. For example, several agencies have common breakouts for cross-cutting programs as defined by the Office of Science and Technology Policy and the Office of Management and Budget, including programs categorized by topic areas, such as nanotechnology, climate change, and homeland security. The federal funds survey, however, classifies data by disciplinary fields (such as chemistry, physics, and life sciences); there is no collection of data by topic area. Furthermore, as new fields emerge, old fields are joined in new combinations or decline in importance altogether, and therefore a taxonomy developed around the dominant fields of an earlier era may not provide for an adequate depiction of the relevant data on current federal R&D spending. Beyond that, the currently used taxonomy is now quite uneven with respect to the level of detail reported in various clusters of disciplines, such as the social sciences, the life sciences, and information sciences, in comparison to the physical sciences and engineering.
OCR for page 11
Data on Federal Research and Development Investments: A Pathway to Modernization Antiquated Characterization of Research and Development The very building blocks of the model of R&D investment are increasingly questioned. Characterizing activities to understand and affect the natural and human environment as basic research, applied research, and development enshrines a linear model that has never been more than a rough approximation of the way R&D actually works, and it is increasingly inadequate as a representation of contemporary reality. In the period following World War II, the U.S. government S&E investment policies increasingly favored basic research. This emphasis was said to have created an organizational disconnect between the federal government’s technologically inspired systems for basic research and industry’s use-inspired systems for development (Stokes, 1997). A major subsequent preoccupation of science and technology policy since the 1980s has been to bridge the so-called valley of death between research and development and commercialization2 (Branscomb and Auerswald, 2001, 2002). The data on categories of R&D now collected are useful and still necessary to understand the federal R&D enterprise, but they are not fully adequate to portray that enterprise in today’s environment. Likewise, better metrics are needed to identify and describe the impact of R&D as a source of economic growth. The U.S. economy has significantly altered since NSF first began to compile R&D data. For example, the role of the central industrial R&D laboratory focused on fundamental research has faded and the federal role in basic research has expanded as private basic research has contracted. Studies indicate that federally funded research is now cited in a majority of industrial patent applications, and it underlies many innovations that become successfully commercialized (Broad, 1997; Block and Keller, 2008). The innovation wave in information technology, for example, was largely underpinned by federal R&D support (National Research Council, 1999, pp. 85-157; Ruttan, 2006, pp. 91-127). Thus, while there is still a strong justification for the conduct of basic research to provide the knowledge basis for improved health, security, and prosperity—the theme of Vannevar Bush’s report, Science the Endless Frontier (Bush, 1945)—the nation has become more focused on a dual rationale for research: both knowledge and innovation. To the extent that the federal role in the innovation process is growing, more information about the relationship between federal research and subsequent innovation is appropriate. The R&D enterprise is increasingly a matter of interdependence among government, academia, and industry. The role of the federal government in fostering 2 The “valley of death” refers to the gap between basic research, which is largely federally funded, and applied research and development, which is often industry-funded.
OCR for page 12
Data on Federal Research and Development Investments: A Pathway to Modernization breakthrough innovation needs to be better understood, and more useful data on federal investment in R&D would help. Increasing Utilization of Administrative Data While sympathetic to the need for more information to illuminate the trends in research and development and the federal role in that enterprise, the panel is also sensitive to the concern that federal agencies not be over-burdened with requirements for additional data reporting. A principal reason the current data are less reliable than desirable is that, in the view of many agencies, filling out NSF reports is labor-intensive and difficult and the benefits barely justify the high cost in labor and other resources (Touhy, 1998). Rather than simply increasing the reporting burden, it would be preferable to consider new data search and analysis technologies tied to expanding efforts to make government data accessible (such as http://www.data.gov), which might gradually make it easier to obtain the raw data for the R&D surveys in ways less burdensome than the current individually conducted compilation processes undertaken separately at each agency. ORGANIZATION OF THE REPORT With these user needs and challenges in mind, the panel set about to identify a step-by-step process that would lead to improvement in the federal data on R&D spending. Following this introduction, Chapter 2 provides a description and critique of the current status of the two surveys that now provide the information used to portray federal R&D spending. Chapter 3 focuses on current problems and makes suggestions for a few relatively modest improvements that could be made in the short term (the next four years or so) to the current system of surveys. We then urge NSF to focus attention on a medium-term solution (over roughly 4 to 10 years) that would make use of new technologies and maturing automated databases and set the stage for long-term changes in the collection system—beyond 10 years. Chapter 4 describes the potential and limitations of the use of administrative data for collecting and compiling information on federal R&D spending and identifies opportunities for transitioning to a new system of data collection. In Chapter 5, we explore some cutting-edge possibilities for long-term changes in the way in which R&D is viewed and the manner in which information about it is collected. In Chapter 6, these threads are gathered into a recommended course of action, which would take NSF through the process of making small short-term improvements in the surveys, undertaking an initiative to build a much fuller, more useful administrative records–based system, and laying the basis for even more revolutionary changes in the long term.