Advisers to the Nation on Science, Engineering, and Medicine

The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences.

The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Charles M. Vest is president of the National Academy of Engineering.

The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Harvey V. Fineberg is president of the Institute of Medicine.

The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. Charles M. Vest are chair and vice chair, respectively, of the National Research Council.

www.national-academies.org

This page intentionally left blank.

Many people contributed time and expertise to the information-gathering efforts of the panel, which, together with its own deliberations, form the basis for this report. The panel appreciates their cooperation and assistance.

The staff of the Division of Science Resources Statistics of the National Science Foundation (NSF) was exceptionally cooperative and forthcoming with information necessary for the panel to conduct its business. Under the leadership of Lynda Carlson, who addressed the panel at its first meeting and helped to establish the framework for our inquiries, and her deputy, Mary Frase, the division staff went to great lengths to assemble information and present it to the panel in a concise and useful manner. John Jankowski, who manages these survey operations, gave three informative orientation presentations to the panel in the first meeting and the workshop the panel held September 5-6, 2008. These presentations summarized the issues with the surveys and the taxonomy of fields of science and engineering, permitting a frank and fruitful discussion of those issues. From her perspective as a staff mathematical statistician, Jeri Mulrow provided a helpful discussion of her work in assessing the implementation of the fields of science and engineering and the findings of her investigation into the use of the fields in the work of NSF. Melissa Pollack coordinated between the panel and NSF in her service as the person responsible for the grant that supported this activity.

The presentations in the workshop provided much of the basis for the analysis and recommendations in this report. The panel expresses its appreciation to Diane DiEuliis of the Office of Science and Technology Policy;

James Wilson, then majority staff director of the Research and Science Education Subcommittee of the U.S. House of Representatives’ Committee on Science and Technology; and John Sargent of the Congressional Research Service for representing data users in a session that expanded awareness of important uses of the data from the surveys and offered some very solid suggestions for making the data more relevant and useful. Robert Gropp of the Institute of Biological Sciences provided his perspective on the use of these data and suggested improvements on the spur of the moment when a scheduled speaker was unable to participate. Although not able to participate in the workshop, Louis Lanzerotti, chairman of the National Science Board’s Subcommittee on Science and Engineering Indicators, provided letter input to the panel that outlined conclusions of the National Science Board on data resources for federal research and development (R&D) allocation decisions.

Two individuals represented the data providers in the federal agencies with responsibility for conducting research and development programs: Israel Lederhendler, a director in the Office of Extramural Research at the National Institutes of Health (NIH), and Tom Russell, director of aerospace, chemistry, and materials in the Air Force Office of Scientific Research. Subsequent to the workshop, Timothy Hayes of NIH provided additional information on its Research, Condition, and Disease Categorization System. Julia Lane of NSF provided a valuable discussion of the important Science of Science and Innovation Policy initiative and, on the basis of her own extensive work in using administrative data for research and analysis, opened the way to a subsequent discussion of opportunities to capitalize on government-wide initiatives to use administrative data for assessing R&D investments.

To assist the panel in focusing on long-term opportunities to use federal government administrative data and other data sources for measuring R&D investments, Mark Bussow represented the agency with responsibility for overall coordination of these efforts, the Office of Management and Budget. Andrew Reamer of the Brookings Institution provided a discussion and assessment of the various sources of information that represent the federal government’s inventory of administrative data on grants and contracts, and Jeff Alexander, then of New Economy Strategies (now associated with SRI International), summarized the utility of those data sources from the prospective of a user of the information. On the second day of the workshop, Gretchen Gano of New York University discussed the state of the science with regard to classification of the fields of science and engineering and summarized several new initiatives that use the power of computing to classify items and domains.

The panel also acknowledges the excellent work of the staff of the Committee on National Statistics (CNSTAT) and the National Research

Council (NRC) for support in developing and organizing the workshop and this report. Under the direction of Constance Citro, director of CNSTAT, Tom Plewes, the study director, provided valuable assistance to the panel in organizing the meetings and preparing this report. He was ably assisted by Michael Siri, also on the staff of CNSTAT.

This report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the Report Review Committee of NRC. The purpose of this independent review is to provide candid and critical comments that assist the institution in making its report as sound as possible, and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process.

The panel thanks the following individuals for their review of this report: Rita R. Colwell, Center for Bioinformatics and Computational Biology, University of Maryland; Richard B. Freeman, Department of Economics, National Bureau of Economic Research, Harvard University; Monica Gaughan, Department of Health Policy and Management, University of Georgia; James Hendler, Tetherless World Constellation, Rensselaer Polytechnic Institute; Julia Melkers, School of Public Policy, Georgia Institute of Technology; Roger Pielke, Jr., Center for Science and Technology Policy Research, University of Colorado; and Hal S. Stern, Department of Statistics, University of California, Irvine.

Although the reviewers listed above have provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations, nor did they see the final draft of the report before its release. The review of this report was overseen by Lawrence D. Brown, Department of Statistics, The Wharton School, University of Pennsylvania. Appointed by the NRC, he was responsible for making certain that the independent examination of this report was carried out in accordance with institutional procedures and that all review comments were carefully considered. Responsibility for the final content of the report rests entirely with the authoring committee and the NRC.

Christopher Hill, Chair

Panel on Modernizing the Infrastructure of the National Science Foundation Federal Funds Survey

This page intentionally left blank.

This page intentionally left blank.