National Academies Press: OpenBook
Suggested Citation:"Front Matter." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2000. Experiments in International Benchmarking of US Research Fields. Washington, DC: The National Academies Press. doi: 10.17226/9784.
×

EXPERIMENTS IN INTERNATIONAL BENCHMARKING OF US RESEARCH FIELDS

Committee on Science, Engineering, and Public Policy

National Academy of Sciences

National Academy of Engineering

Institute of Medicine

NATIONAL ACADEMY PRESS
Washington, D.C.

Suggested Citation:"Front Matter." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2000. Experiments in International Benchmarking of US Research Fields. Washington, DC: The National Academies Press. doi: 10.17226/9784.
×

NATIONAL ACADEMY PRESS
2101 Constitution Ave., NW Washington, DC 20418

NOTICE: This volume was produced as part of a project approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. It is a result of work done by the Committee on Science, Engineering, and Public Policy (COSEPUP) as augmented, which has authorized its release to the public. This report has been reviewed by a group other than the authors according to procedures approved by COSEPUP and the Report Review Committee.

The Committee on Science, Engineering, and Public Policy (COSEPUP) is a joint committee of NAS, NAE, and IOM. It includes members of the councils of all three bodies.

Financial Support: The development of this report was supported by the Sloan Foundation and the National Research Council.

International Standard Book Number 0-309-06898-3

Internet Access: This report is available on COSEPUP's World Wide Web site at http://www4.nationalacademies.org/pd/cosepup.nsf

Order from:
National Academy Press,
2101 Constitution Ave., NW, Washington, DC 20418. All orders must be prepaid with delivery to a single address. No additional discounts apply. Prices are subject to change without notice. To order by credit card, call 1-800-624-6242 or 202-334-3313 (in Washington metropolitan area).

Copyright 2000 by the National Academy of Sciences. All rights reserved. This document may be reproduced solely for educational purposes without the written permission of the National Academy of Sciences.

Printed in the United States of America.

Suggested Citation:"Front Matter." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2000. Experiments in International Benchmarking of US Research Fields. Washington, DC: The National Academies Press. doi: 10.17226/9784.
×

Committee on Science, Engineering, and Public Policy

MAXINE F. SINGER (Chair), President,

Carnegie Institution of Washington

BRUCE M. ALBERTS,* President,

National Academy of Sciences

ENRIQUETA C. BOND, President,

The Burroughs Wellcome Fund

LEWIS BRANSCOMB, Professor Emeritus,

Center for Science and International Affairs, John F. Kennedy School of Government, Harvard University

PETER DIAMOND, Institute Professor and Professor of Economics,

Massachusetts Institute of Technology

GERALD DINNEEN,* Retired Vice President,

Science and Technology, Honeywell, Inc.

MILDRED S. DRESSELHAUS, Institute Professor of Electrical Engineering and Physics,

Massachusetts Institute of Technology

JAMES J. DUDERSTADT, President Emeritus and University Professor of Science and Engineering,

Millennium Project, University of Michigan

MARYE ANNE FOX, Chancellor,

North Carolina State University

RALPH E. GOMORY, President,

Alfred P. Sloan Foundation

RUBY P. HEARN, Vice President,

The Robert Wood Johnson Foundation

BRIGID L. M. HOGAN, Investigator,

Howard Hughes Medical Institute, and

Hortense B. Ingram Professor,

Department of Cell Biology, Vanderbilt University School of Medicine

SAMUEL PRESTON, Dean,

University of Pennsylvania School of Arts and Sciences

KENNETH I. SHINE,* President,

Institute of Medicine

MORRIS TANENBAUM, Retired Vice Chairman and Chief Financial Officer,

AT&T

IRVING L. WEISSMAN, Karel and Avice Beekhuis Professor of Cancer Biology,

Stanford University School of Medicine

SHEILA E. WIDNALL, Abbey Rockefeller Mauze Professor of Aeronautics,

Massachusetts Institute of Technology

WILLIAM JULIUS WILSON, Lewis P. and Linda L. Geyer University Professor,

Harvard University

WILLIAM A. WULF,* President,

National Academy of Engineering

_______________

RICHARD E. BISSELL, Director

DEBORAH D. STINE, Associate Director

MARION RAMSEY, Administrative Associate

*  

 Ex officio member

Suggested Citation:"Front Matter." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2000. Experiments in International Benchmarking of US Research Fields. Washington, DC: The National Academies Press. doi: 10.17226/9784.
×

COSEPUP Benchmarking Guidance Group

MARYE ANNE FOX (Chair), Chancellor,

North Carolina State University

DAVID CHALLONER, Director,

Institute for Science and Health Policy, University of Florida

ELLIS COWLING, University Distinguished Professor At-Large,

North Carolina State University

GERALD DINNEEN, Retired Vice President,

Science and Technology, Honeywell, Inc.

MILDRED S. DRESSELHAUS, Institute Professor of Electrical Engineering and Physics,

Massachusetts Institute of Technology

ALEXANDER FLAX, Consultant

RALPH E. GOMORY, President,

Alfred P. Sloan Foundation

_______________

DEBORAH D. STINE, Study Director

ALAN ANDERSON, Consultant Writer

REBECCA BURKA, Administrative Associate

AUBREY SABALA, Project Assistant

NORMAN GROSSBLATT, Editor

Suggested Citation:"Front Matter." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2000. Experiments in International Benchmarking of US Research Fields. Washington, DC: The National Academies Press. doi: 10.17226/9784.
×

PREFACE

The American people, through their elected representatives, support the nation's research enterprise in the expectation of substantial returns on their investment: a higher standard of living, a healthier society, an environmentally sustainable economy, and a strong national security. Knowing the power of research in addressing national objectives, the nation has committed itself to a broad set of investments to uphold its research capability.

The National Research Council has already prepared studies that describe the effectiveness of research investments in addressing national concerns. Research investments affect the quality of research done. The present study asks how to evaluate research-leadership status. COSEPUP proposed contributing a set of experiments in international benchmarking. International benchmarking compares the quality and impact of research in one country (or region) with world standards of excellence. The use of international benchmarking was also advocated in 1995 by the "Press report," Allocating Federal Funds for Science and Technology (see appendix B), for the purpose of providing objective information for the executive branch and Congress. The need for objective evaluations has intensified since the passage in 1993 of the Government Performance and Results Act (GPRA), which requires annual performance reports by all federal agencies, including those which support research. GPRA is discussed in the 1999 COSEPUP report Evaluating Federal Research Programs: Research and the Government Performance and Results Act .

Although the use of international benchmarking was not new, it had not been attempted on a scale large enough to contribute to national

Suggested Citation:"Front Matter." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2000. Experiments in International Benchmarking of US Research Fields. Washington, DC: The National Academies Press. doi: 10.17226/9784.
×

policy. Accordingly, in 1997, COSEPUP decided to undertake a set of experiments to test the efficacy of international benchmarking. The committee chose three areas of research—mathematics, immunology, and materials science and engineering—that are quite different from one another in size, funding, numbers of subdisciplines, and other qualities. COSEPUP appointed a panel for each field and sought to experiment with providing information for relatively modest commitments of time and money. Once the panels had completed their reports, the committee held a workshop with agency representatives, congressional staff, and oversight bodies to discuss the findings. (See appendix C for a summary of the workshop.)

This report describes the background, methodology, experimental results, and findings of the international benchmarking experiments and concludes that international benchmarking by a panel of experts can be efficient and reasonably objective. International benchmarking might also be a valuable assessment tool for those seeking to implement GPRA.

Maxine Singer

Chair

Committee on Science, Engineering, and Public Policy

Suggested Citation:"Front Matter." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2000. Experiments in International Benchmarking of US Research Fields. Washington, DC: The National Academies Press. doi: 10.17226/9784.
×

ACKNOWLEDGMENTS

This report is the product of many individuals. First, COSEPUP acknowledges those who made presentations at the Workshop on International Benchmarking: Arden Bement, chair, Panel on Materials Science and Engineering; Irving Weissman, chair, Panel on Immunology; Peter Lax, chair, Panel on Mathematics; and the discussants: Richard Russell, House Committee on Science; Don Lewis, director, Division of Mathematical Sciences, National Science Foundation; Patricia Dehmer, associate director, Office of Science, Office of Basic Energy Sciences, Department of Energy; Helen Quill, chief, Basic Immunology Branch, Division of Allergy, Immunology and Transplantation, National Institute of Allergy and Infectious Diseases, National Institutes of Health; and Irwin Feller, director, Institute for Policy Research and Evaluation, and professor of economics, Pennsylvania State University.

The report was reviewed by persons chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the National Research Council's Report Review Committee. The purposes of this independent review are to provide candid and critical comments that will assist COSEPUP in making its report as sound as possible and to ensure that the report meets institutional standards of objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process. We wish to thank the following for their participation in the review of this report: Arden L. Bement, Jr., Robert M. White, Margaret H. Wright, Paul A. Fleury, Susan Cozzens, Arthur Bienenstock, and the report review coordinator, Anita

Page viii Cite
Suggested Citation:"Front Matter." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2000. Experiments in International Benchmarking of US Research Fields. Washington, DC: The National Academies Press. doi: 10.17226/9784.
×

Jones, and the report review monitor, John Ahearne. Although those persons provided many constructive comments and suggestions, responsibility for the content of the report rests solely with COSEPUP.

The production of this report was the result of hard work by the committee as a whole and by the extra effort of the Guidance Group, consisting of current and former COSEPUP members Marye Anne Fox (Chair), David Challoner, Ellis Cowling, Gerald Dinneen, Mildred S. Dresselhaus, Alexander Flax, and Ralph E. Gomory.

The project was aided by the invaluable help of COSEPUP professional staff: Deborah D. Stine, associate director of COSEPUP and study director; Rebecca Burka, administrative associate; Aubrey Sabala, project assistant; Alan Anderson, consultant writer; and Norman Grossblatt, editor. The committee also thanks Richard Bissell, executive director of the Policy Division and director of COSEPUP, who oversaw the committee's activities.

Suggested Citation:"Front Matter." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2000. Experiments in International Benchmarking of US Research Fields. Washington, DC: The National Academies Press. doi: 10.17226/9784.
×
Suggested Citation:"Front Matter." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2000. Experiments in International Benchmarking of US Research Fields. Washington, DC: The National Academies Press. doi: 10.17226/9784.
×
Page R1
Suggested Citation:"Front Matter." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2000. Experiments in International Benchmarking of US Research Fields. Washington, DC: The National Academies Press. doi: 10.17226/9784.
×
Page R2
Suggested Citation:"Front Matter." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2000. Experiments in International Benchmarking of US Research Fields. Washington, DC: The National Academies Press. doi: 10.17226/9784.
×
Page R3
Suggested Citation:"Front Matter." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2000. Experiments in International Benchmarking of US Research Fields. Washington, DC: The National Academies Press. doi: 10.17226/9784.
×
Page R4
Suggested Citation:"Front Matter." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2000. Experiments in International Benchmarking of US Research Fields. Washington, DC: The National Academies Press. doi: 10.17226/9784.
×
Page R5
Suggested Citation:"Front Matter." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2000. Experiments in International Benchmarking of US Research Fields. Washington, DC: The National Academies Press. doi: 10.17226/9784.
×
Page R6
Suggested Citation:"Front Matter." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2000. Experiments in International Benchmarking of US Research Fields. Washington, DC: The National Academies Press. doi: 10.17226/9784.
×
Page R7
Page viii Cite
Suggested Citation:"Front Matter." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2000. Experiments in International Benchmarking of US Research Fields. Washington, DC: The National Academies Press. doi: 10.17226/9784.
×
Page R8
Suggested Citation:"Front Matter." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2000. Experiments in International Benchmarking of US Research Fields. Washington, DC: The National Academies Press. doi: 10.17226/9784.
×
Page R9
Suggested Citation:"Front Matter." National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. 2000. Experiments in International Benchmarking of US Research Fields. Washington, DC: The National Academies Press. doi: 10.17226/9784.
×
Page R10
Next: Executive Summary »
Experiments in International Benchmarking of US Research Fields Get This Book
×
Buy Paperback | $87.00 Buy Ebook | $69.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

How can the federal government gauge the overall health of scientific research--as a whole and in its parts--and determine whether national funding adequately supports national research objectives? It is feasible to monitor US performance with field-by-field peer assessments. This might be done through the establishment of independent panels consisting of researchers who work in a field, individuals who work in closely related fields, and research "users" who follow the field closely. Some of these individuals should be outstanding foreign scientists in the field being examined. This technique of comparative international assessments is also known as international benchmarking.

Experiments in International Benchmarking of U.S. Research Fields evaluates the feasibility and utility of the benchmarking technique. In order to do this, the report internationally benchmarks three fields: mathematics, immunology, and materials science and engineering, then summarizes the results of these experiments.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!