The National Center for Science and Engineering Statistics (NCSES) of the National Science Foundation (NSF) is involved in the collection, analysis, and dissemination of data on science and engineering. NCSES conducts a number of surveys covering various aspects of this topic area, and the agency produces 30 reports a year, including the National Patterns of R&D Resources (hereafter referred to as National Patterns). Based primarily on input from five major NCSES research and development (R&D) surveys, this annual publication provides an overview of a calendar year’s distribution of R&D funds broken out by (1) the type of research carried out—basic, applied, or development; (2) the type of provider; and (3) the type of recipient. The publication consists of tabular representations of R&D expenditure and funding time series. It also provides some sub-national geographic detail. Tables of the amounts and associated time plots facilitate comparison of the funding and expenditure patterns both over time and with selected countries.
Looking forward, NCSES is interested in retaining or enhancing the future relevance of National Patterns. Changes and enhancement might include
• changing the main tabular presentations currently used;
• changing which of the current portfolio of variables collected on the five major input surveys are tabulated;
• adding new variables on the questionnaires for these surveys;
• making use of administrative records sources of data; and
BOX 1-1
Statement of Task
An ad hoc committee will plan and organize commissioned papers for, and conduct a public workshop to review and consider, future directions for the National Patterns of Research and Development Program of the National Center for Science and Engineering Statistics of the National Science Foundation. The workshop sessions will feature invited presentations and discussions on such issues as:
• the responsiveness of the information content to users’ needs;
• targets for expanded content;
• the usefulness of adjustments for consistency;
• timeliness vs. data quality;
• the appropriateness of the methods for estimated values (e.g., estimation of R&D performance by nonprofit organizations, business R&D expenditures, and federal and state intramural R&D expenditures);
• consistency of indicators from other sources (e.g., Bureau of Economic Analysis, OECD);
• international comparability; and
• categorization of R&D by socioeconomic objective and science/engineering field.
The committee will develop the workshop agenda, commissioned papers, select and invite speakers and discussants, and moderate the discussion.
• adding new tabulations, statistics, or graphics that include information from new sources.
This enhancing of National Patterns could also involve improvement of the statistical methods used to create these products and thereby improve the quality of the statistical products in National Patterns.
The charge to the Committee on National Statistics (CNSTAT) of the National Research Council (NRC) was to organize a workshop to examine whether and how these various dimensions of National Patterns could be improved: see Box 1-1. To address the charge, CNSTAT formed the Steering Committee on Future Directions for the NSF National Patterns of Research and Development Reports.
Statistics on R&D are important indicators of innovation, the transfer of ideas and knowledge, which in turn are key drivers of economic growth (see, e.g., Romer, 1986, 1990), and R&D expenditures are one indicator of the generation and diffusion of knowledge (see Rosenberg, 1994). A key aspect of the role of R&D in the process of economic growth is that it generates spillover benefits. Given the contribution of R&D to economic growth and, consequently, its role in policy decisions, NSF began to measure it in the 1950s.
Although the idea of measuring R&D seems straightforward, there are complexities to carrying it out. As pointed out by Jaffe and Trajtenberg (2005), knowledge, innovation, and technical change are elusive notions. They are hard to define and harder to measure. To its credit, for more than 60 years NCSES (and its predecessors) has collected and produced consistent statistical tables and graphs on domestic R&D expenditures, producing the statistics in a timely fashion. Underlying the publication process are the efforts undertaken by the agency to overcome the complexities of definitions and various efforts to address issues such as missing data and international comparability.
National Patterns of R&D Resources is a compendium of five annual surveys. Each publication in the series integrates and synthesizes the data from these periodic surveys of R&D expenditures by U.S. R&D performers in order to analyze current patterns of R&D activity in the United States in relation to the historical record and to the reported R&D levels of other industrialized countries. The goal of this effort is to aid NCSES in fulfilling the legislative mandate of the NSF: see Box 1-2.
BOX 1-2
The National Science Foundation Act of 1950
The National Science Foundation was created in 1950 with a mission “to promote the progress of science; to advance the national health, prosperity, and welfare; and to secure the national defense.” That 1950 act that created the agency also made it a statistical agency. In that vein, it is also charged with providing “a central clearinghouse for the collection, interpretation, and analysis of data on scientific and engineering resources, and to provide a source of information for policy formulation by other agencies of the Federal government.”
The steering committee identified seven topics for presentations and subsequent discussions at the workshop:
1. the purposes and uses of National Patterns;
2. advances in international comparability of the statistical outputs in National Patterns;
3. the nature and estimation of R&D expenditure data for nonprofit organizations;
4. the benefits of collecting and reporting on additional variables relevant to R&D funds;
5. improving communication in National Patterns;
6. potential methodological uses of administrative records for R&D estimation; and
7. the use of small-area estimation techniques for estimating R&D amounts for small domains such as states crossed with industrial categories.
The reader should note that the agenda items are in accord with the issues mentioned in the above statement of task. A workshop is not a consensus activity, and so no recommendations or other consensus findings are offered in this report.
The purpose of the workshop and this summary are to explore a set of possible next steps for improving the relevance, the content, and the presentation of National Patterns reports by better understanding the demands of users, the constraints of the data producers, the techniques used to address data limitations, the purpose of the current strategies for data dissemination, and the challenges in using administrative data. The workshop provided a variety of views and suggested a range of possibilities for improvements to the sponsoring agency. The workshop agenda is presented in Appendix B. The workshop presentations are available on CNSTAT’s webpage: http://sites.nationalacademies.org/dbasse/cnstat/dbasse_070728 [June 2013].
This report generally follows the workshop structure, summarizing the presentations and the discussions on each topic. Chapter 2 covers the historical background of National Patterns, including a description of the surveys that feed into the report, the variables collected on those surveys and tabulated in National Patterns, and other NSF publications that make use of the R&D data. This presentation also mentions comparable publications from international agencies or organizations. Chapter 3 addresses the user perspective, domestically and internationally: namely, what modifications could be made to National Patterns that could address currently unmet or anticipated future user needs. Chapter 4 highlights the missing gaps in the
data that are currently collected by NCSES and points out areas in which techniques or new data sources might be used to improve current estimates. Chapter 5 looks at a key methodological issue of how to produce R&D statistics for small domains. Finally, Chapter 6 concerns how to present or display the information in National Patterns data so that readers can recognize the patterns more easily.
We stress that this summary is limited to the views expressed either during the activities undertaken in planning the workshop or at the workshop itself. Therefore, all views expressed are those of the workshop presenters or other workshop attendees.
