Data on Federal Research and Development Investments
A PATHWAY TO MODERNIZATION
THE NATIONAL ACADEMIES PRESS
Washington, D.C.
www.nap.edu
THE NATIONAL ACADEMIES PRESS
500 Fifth Street, N.W. Washington, DC 20001
NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance.
Support of the work of the Committee on National Statistics is provided by a consortium of federal agencies through a grant from the National Science Foundation (award number SES-0453930). Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reflect the views of the organizations or agencies that provided support for the project.
International Standard Book Number 13: 978-0-309-14523-7
International Standard Book Number 10: 0-309-14523-6
Additional copies of this report are available from
National Academies Press,
500 Fifth Street, N.W., Lockbox 285, Washington, DC 20055; (800) 624-6242 or (202) 334-3313 (in the Washington metropolitan area); Internet, http://www.nap.edu.
Copyright 2010 by the National Academy of Sciences. All rights reserved.
Printed in the United States of America
Suggested citation: National Research Council. (2010). Data on Federal Research and Development Investments: A Pathway to Modernization. Panel on Modernizing the Infrastructure of the National Science Foundation Federal Funds Survey, Committee on National Statistics. Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.
THE NATIONAL ACADEMIES
Advisers to the Nation on Science, Engineering, and Medicine
The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Ralph J. Cicerone is president of the National Academy of Sciences.
The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Charles M. Vest is president of the National Academy of Engineering.
The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Harvey V. Fineberg is president of the Institute of Medicine.
The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Ralph J. Cicerone and Dr. Charles M. Vest are chair and vice chair, respectively, of the National Research Council.
PANEL ON MODERNIZING THE INFRASTRUCTURE OF THE NATIONAL SCIENCE FOUNDATION FEDERAL FUNDS SURVEY
CHRISTOPHER T. HILL (Chair),
School of Public Policy, George Mason University
WILLIAM B. BONVILLIAN,
Washington, DC, Office, Massachusetts Institute of Technology
KATY BÖRNER,
School of Library and Information Science, Indiana University
MARY K. FEENEY,
College of Planning and Public Affairs, University of Illinois at Chicago
DAVID GOLDSTON,
Center for the Environment, Harvard University
NANCY J. KIRKENDALL, Independent Consultant,
Washington, DC
JULIE THOMPSON KLEIN,
Department of English, Wayne State University
KEI KOIZUMI,
Office of Science and Technology Policy, Executive Office of the President
J. DAVID ROESSNER,
SRI International, Prescott, AZ
MARTHA M. TAYLOR,
Office of Sponsored Programs, Auburn University
THOMAS J. PLEWES, Study Director
MICHAEL J. SIRI, Program Associate
COMMITTEE ON NATIONAL STATISTICS 2009
WILLIAM F. EDDY (Chair),
Department of Statistics, Carnegie Mellon University
KATHARINE ABRAHAM,
Department of Economics and Joint Program in Survey Methodology, University of Maryland
ALICIA CARRIQUIRY,
Department of Statistics, Iowa State University
WILLIAM DUMOUCHEL,
Phase Forward, Inc., Waltham, MA
JOHN HALTIWANGER,
Department of Economics, University of Maryland
V. JOSEPH HOTZ,
Department of Economics, Duke University
KAREN KAFADAR,
Department of Statistics, Indiana University
DOUGLAS MASSEY,
Department of Sociology, Princeton University
SALLY MORTON,
Statistics and Epidemiology, RTI International, Research Triangle Park, NC
JOSEPH NEWHOUSE,
Division of Health Policy Research and Education, Harvard University
SAMUEL H. PRESTON,
Population Studies Center, University of Pennsylvania
HAL STERN,
Department of Statistics, University of California, Irvine
ROGER TOURANGEAU,
Survey Research Center, University of Michigan, and Joint Program in Survey Methodology, University of Maryland
ALAN ZASLAVSKY,
Department of Health Care Policy, Harvard Medical School
CONSTANCE F. CITRO, Director
Acknowledgments
Many people contributed time and expertise to the information-gathering efforts of the panel, which, together with its own deliberations, form the basis for this report. The panel appreciates their cooperation and assistance.
The staff of the Division of Science Resources Statistics of the National Science Foundation (NSF) was exceptionally cooperative and forthcoming with information necessary for the panel to conduct its business. Under the leadership of Lynda Carlson, who addressed the panel at its first meeting and helped to establish the framework for our inquiries, and her deputy, Mary Frase, the division staff went to great lengths to assemble information and present it to the panel in a concise and useful manner. John Jankowski, who manages these survey operations, gave three informative orientation presentations to the panel in the first meeting and the workshop the panel held September 5-6, 2008. These presentations summarized the issues with the surveys and the taxonomy of fields of science and engineering, permitting a frank and fruitful discussion of those issues. From her perspective as a staff mathematical statistician, Jeri Mulrow provided a helpful discussion of her work in assessing the implementation of the fields of science and engineering and the findings of her investigation into the use of the fields in the work of NSF. Melissa Pollack coordinated between the panel and NSF in her service as the person responsible for the grant that supported this activity.
The presentations in the workshop provided much of the basis for the analysis and recommendations in this report. The panel expresses its appreciation to Diane DiEuliis of the Office of Science and Technology Policy;
James Wilson, then majority staff director of the Research and Science Education Subcommittee of the U.S. House of Representatives’ Committee on Science and Technology; and John Sargent of the Congressional Research Service for representing data users in a session that expanded awareness of important uses of the data from the surveys and offered some very solid suggestions for making the data more relevant and useful. Robert Gropp of the Institute of Biological Sciences provided his perspective on the use of these data and suggested improvements on the spur of the moment when a scheduled speaker was unable to participate. Although not able to participate in the workshop, Louis Lanzerotti, chairman of the National Science Board’s Subcommittee on Science and Engineering Indicators, provided letter input to the panel that outlined conclusions of the National Science Board on data resources for federal research and development (R&D) allocation decisions.
Two individuals represented the data providers in the federal agencies with responsibility for conducting research and development programs: Israel Lederhendler, a director in the Office of Extramural Research at the National Institutes of Health (NIH), and Tom Russell, director of aerospace, chemistry, and materials in the Air Force Office of Scientific Research. Subsequent to the workshop, Timothy Hayes of NIH provided additional information on its Research, Condition, and Disease Categorization System. Julia Lane of NSF provided a valuable discussion of the important Science of Science and Innovation Policy initiative and, on the basis of her own extensive work in using administrative data for research and analysis, opened the way to a subsequent discussion of opportunities to capitalize on government-wide initiatives to use administrative data for assessing R&D investments.
To assist the panel in focusing on long-term opportunities to use federal government administrative data and other data sources for measuring R&D investments, Mark Bussow represented the agency with responsibility for overall coordination of these efforts, the Office of Management and Budget. Andrew Reamer of the Brookings Institution provided a discussion and assessment of the various sources of information that represent the federal government’s inventory of administrative data on grants and contracts, and Jeff Alexander, then of New Economy Strategies (now associated with SRI International), summarized the utility of those data sources from the prospective of a user of the information. On the second day of the workshop, Gretchen Gano of New York University discussed the state of the science with regard to classification of the fields of science and engineering and summarized several new initiatives that use the power of computing to classify items and domains.
The panel also acknowledges the excellent work of the staff of the Committee on National Statistics (CNSTAT) and the National Research
Council (NRC) for support in developing and organizing the workshop and this report. Under the direction of Constance Citro, director of CNSTAT, Tom Plewes, the study director, provided valuable assistance to the panel in organizing the meetings and preparing this report. He was ably assisted by Michael Siri, also on the staff of CNSTAT.
This report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the Report Review Committee of NRC. The purpose of this independent review is to provide candid and critical comments that assist the institution in making its report as sound as possible, and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process.
The panel thanks the following individuals for their review of this report: Rita R. Colwell, Center for Bioinformatics and Computational Biology, University of Maryland; Richard B. Freeman, Department of Economics, National Bureau of Economic Research, Harvard University; Monica Gaughan, Department of Health Policy and Management, University of Georgia; James Hendler, Tetherless World Constellation, Rensselaer Polytechnic Institute; Julia Melkers, School of Public Policy, Georgia Institute of Technology; Roger Pielke, Jr., Center for Science and Technology Policy Research, University of Colorado; and Hal S. Stern, Department of Statistics, University of California, Irvine.
Although the reviewers listed above have provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations, nor did they see the final draft of the report before its release. The review of this report was overseen by Lawrence D. Brown, Department of Statistics, The Wharton School, University of Pennsylvania. Appointed by the NRC, he was responsible for making certain that the independent examination of this report was carried out in accordance with institutional procedures and that all review comments were carefully considered. Responsibility for the final content of the report rests entirely with the authoring committee and the NRC.
Christopher Hill, Chair
Panel on Modernizing the Infrastructure of the National Science Foundation Federal Funds Survey