MAJOR AWARD DECISIONMAKING AT THE NATIONAL SCIENCE FOUNDATION

Panel on NSF Decisionmaking for Major Awards

Committee on Science, Engineering, and Public Policy

National Academy of Sciences

National Academy of Engineering

Institute of Medicine

NATIONAL ACADEMY PRESS
Washington, D.C.
1994



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page R1
Major Award Decisionmaking at the National Science Foundation MAJOR AWARD DECISIONMAKING AT THE NATIONAL SCIENCE FOUNDATION Panel on NSF Decisionmaking for Major Awards Committee on Science, Engineering, and Public Policy National Academy of Sciences National Academy of Engineering Institute of Medicine NATIONAL ACADEMY PRESS Washington, D.C. 1994

OCR for page R1
Major Award Decisionmaking at the National Science Foundation National Academy Press 2101 Constitution Ave., NW Washington, DC 20418 NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. It is a result of work done by an independent panel appointed by the Committee on Science, Engineering, and Public Policy, which has authorized its release to the public. The members of the panel responsible for this report were chosen for their special competencies and with regard for appropriate balance. This report has been reviewed by a group other than the authors according to procedures approved by a Report Review Committee and by the Committee on Science, Engineering, and Public Policy. Both consist of members of the National Academy of Sciences, National Academy of Engineering, and Institute of Medicine. The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Bruce M. Alberts is president of the National Academy of Sciences. The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Robert M. White is president of the National Academy of Engineering. The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences in its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Kenneth I. Shine is president of the Institute of Medicine. The Committee on Science, Engineering, and Public Policy is a joint committee of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. It includes members of the councils of all three bodies. Sponsor: This study was funded with Federal funds from the National Science Foundation (NSF) under contract number LPA-9123428. The contents of this report do not necessarily reflect the views or policies of the NSF, nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. Government. Library of Congress Catalog Card Number 94-66065 International Standard Book Number 0-309-05029-4 Copyright 1994 by the National Academy of Sciences. All rights reserved. Available from: National Academy Press, 2101 Constitution Avenue, N.W., Washington, D.C. 20418 B-274 Printed in the United States of America

OCR for page R1
Major Award Decisionmaking at the National Science Foundation PANEL ON NSF DECISIONMAKING FOR MAJOR AWARDS ROBERT H. RUTFORD (Chair), President, University of Texas at Dallas, and Chairman of the Polar Research Board, National Research Council CLARENCE R. ALLEN, Professor of Geology and Geophysics, Emeritus, California Institute of Technology ALBERT A. BARBER, Special Assistant to the Chancellor, University of California, Los Angeles—Washington D.C. Office HARVEY BROOKS, Gordon McKay Professor of Applied Physics, Emeritus, in the Division of Applied Sciences, and Benjamin Peirce Professor of Technology and Public Policy, Emeritus, John F. Kennedy School of Government, Harvard University CHRISTOPHER COBURN, Director, Public Technology Programs, Battelle Memorial Institute SUSAN E. COZZENS, Associate Professor, Science and Technology Studies Department, Rensselaer Polytechnic Institute FRANK D. DRAKE, Professor of Astronomy and Astrophysics, University of California, Santa Cruz, and President, SETI Institute DONALD S. FREDRICKSON, President, D.S. Fredrickson Associates, Inc. FREDRICK S. HUMPHRIES, President, Florida A&M University ANITA K. JONES, Chair, Department of Computer Science, University of Virginia (resigned May 31, 1993, to become Director of Defense Research and Engineering, Department of Defense) LARRY K. MONTEITH, Chancellor, North Carolina State University DOUGLAS D. OSHEROFF, Professor, Department of Physics, Stanford University JUDITH A. RAMALEY, President, Portland State University LYLE H. SCHWARTZ, Director, Materials Science and Engineering Laboratory, National Institute of Standards and Technology Staff MICHAEL McGEARY, Study Director ELIZABETH BLOUNT, Senior Project Assistant

OCR for page R1
Major Award Decisionmaking at the National Science Foundation COMMITTEE ON SCIENCE, ENGINEERING, AND PUBLIC POLICY PHILLIP A. GRIFFITHS (Chair), Director, Institute for Advanced Study ROBERT McCORMICK ADAMS, Secretary, Smithsonian Institution BRUCE M. ALBERTS, President, National Academy of Sciences (Ex-Officio) ELKAN BLOUT, Harkness Professor, Department of Biological Chemistry and Molecular Pharmacology, Harvard Medical School FELIX BROWDER, Department of Mathematics, Rutgers University ROBERT A. BURT, Alexander M. Bickel Professor of Law, Yale Law School DAVID R. CHALLONER, M.D., Vice President of Health Affairs, University of Florida ALBERT M. CLOGSTON,* Member, Center for Material Sciences, Los Alamos National Laboratory F. ALBERT COTTON, Distinguished Professor of Chemistry; Director, Laboratory for Molecular Structure and Bonding, Texas A&M University ALEXANDER H. FLAX, Senior Fellow, National Academy of Engineering RALPH E. GOMORY, President, Alfred P. Sloan Foundation THOMAS D. LARSON, Consultant JOHN L. McLUCAS,* Aerospace Consultant MARY JANE OSBORN, Department of Microbiology, University of Connecticut Health Center C. KUMAR N. PATEL, Vice Chancellor, Research Programs, University of California, Los Angeles PHILLIP A. SHARP, Head, Department of Biology, Center for Cancer Research, Massachusetts Institute of Technology KENNETH I. SHINE, President, Institute of Medicine (Ex-Officio) ROBERT M. SOLOW, Institute Professor, Department of Economics, Massachusetts Institute of Technology H. GUYFORD STEVER, Member, Carnegie Commission on Science and Technology MORRIS TANENBAUM, Vice President, National Academy of Engineering

OCR for page R1
Major Award Decisionmaking at the National Science Foundation ROBERT M. WHITE, President, National Academy of Engineering (Ex-Officio) SHEILA E. WIDNALL,* Associate Provost and Abby Rockefeller Mauze Professor of Aeronautics and Astronautics, Massachusetts Institute of Technology Staff LAWRENCE E. McCRAY, Executive Director BARBARA A. CANDLAND, Administrative Coordinator *   Term expired June 30, 1993

OCR for page R1
Major Award Decisionmaking at the National Science Foundation This page in the original is blank.

OCR for page R1
Major Award Decisionmaking at the National Science Foundation Preface Under the guidance of the National Science Board (NSB), the National Science Foundation (NSF) supports science and engineering research and education projects. NSF does not carry out these projects itself. It chooses the best proposals submitted by researchers in universities, colleges, and other research institutions. NSF uses a merit review process to identify the most promising projects to receive funding awards. Merit review has two distinctive features: it relies on independent outside peer reviewers to assess the quality of proposals, and it uses criteria that emphasize technical quality and also promote other goals of the nation’s research base such as equal opportunity, human resource development, and a broader geographic and institutional infrastructure. Most of the awards made by NSF are to individuals or to small groups of scientists and engineers. This report addresses a small but important set of awards—very large awards for major research facilities, interdisciplinary research centers, and other large-scale research-related activities. Because of their budgetary impact and importance, it is critical that these major projects be carefully chosen on the basis of their contributions to the nation’s research enterprise and not according to political, bureaucratic, or other considerations. To achieve this, major award proposals are subjected to a merit review process. Merit review of major awards is more complicated and sometimes more controversial than that for individual investigator and small group awards. This report is based on an 18-month study of the NSF-NSB system for making major awards. The study was undertaken by a broad-based expert group, which makes a series of recommendations

OCR for page R1
Major Award Decisionmaking at the National Science Foundation for improving the planning, selection, and renewal of such awards. The recommendations appear in the chapters on these topics and are summarized in the executive summary. The panel would like to thank the individuals who took the time to meet with us and share their knowledge, experiences, and views. Special thanks go to Alan M. Gaines, assistant for science and technology to the director of NSF, and NSF liaison official for this study, who made sure we had full and timely access to the publicly available information needed for the study. The panel was briefed on the decisionmaking process for major awards at its first meeting by NSF and NSB officials: Frederick M. Bernthal, deputy director (chair, Director's Action Review Board); William C. Harris, assistant director for mathematical and physical sciences; Mary E. Clutter, assistant director for biological sciences; Joseph Kull, chief financial officer (executive secretary, NSB Committee on Programs and Plans); and Marta Cehelsky, NSB executive officer. Then-director Walter E. Massey met with the panel at a later meeting. Warren J. Baker, chair, NSB Committee on Programs and Plans, also briefed the panel on how the NSB reviews major award proposals. Former NSF director John B. Slaughter, who recently chaired the site selection committee for the NSF-supported Laser Interferometer Gravitational Wave Observatory, graciously provided his views on the evolution of the major award review process in an interview with a panel member. The staff would also like to thank others at NSF who provided information and insight: Robert P. Abel, Charles N. Brownstein, Thomas N. Cooley, Peter W. House, Madeleine E. Hymowitz, James M. McCullough, Lynn Preston, and Joanna E. Rom. Susan E. Fannoney of the NSB staff was especially helpful in locating and providing NSB documents relating to NSB review and approval of 10 case study awards; Florence Heckman, NSF librarian, pointed the way to materials on the history of proposal review at NSF; and George Mazuzan, NSF historian, provided access to the historical files of NSF. The panel appreciates the efforts of Michael McGeary, the study director, who pulled together a remarkable amount of information on

OCR for page R1
Major Award Decisionmaking at the National Science Foundation NSF's merit review policies, procedures, and practices, as well as the 10 case study award decisions, and assisted the panel in drafting the report. Elizabeth Blount, staff associate, took care of the many administrative details of panel meetings and report production with skill, energy, and unfailing good cheer. Jeffrey D. Porro, consultant, edited early drafts of the report. We are also grateful for the support and assistance of the staff of the Committee on Science, Engineering, and Public Policy, including Lawrence E. McCray, executive director, and Barbara Candland, executive assistant, and of Philip M. Smith, executive officer, National Academy of Sciences. Florence Poullin was copy editor. National Academy Press staff who helped turn the report into a book included Stephen Mautner, Dawn Eichenlaub, and James Gormley. Finally, I would like to thank the panel members for their willingness to devote considerable time to the study and for their contributions to this report. The recommendations reflect their vast experience and wisdom and their desire to give NSF and the NSB constructive advice for better decisionmaking on major awards. Panel member Anita K. Jones was able to participate substantially in the drafting of the report before resigning in May 1993 to become the director of defense research and engineering in the Department of Defense. Robert H. Rutford Chairman of the Study Panel

OCR for page R1
Major Award Decisionmaking at the National Science Foundation This page in the original is blank.

OCR for page R1
Major Award Decisionmaking at the National Science Foundation Contents     EXECUTIVE SUMMARY   1 1   MAJOR AWARDS AT NSF   13     Overview of Major Awards   15     Major Awards and Merit Review   19     Major Awards and the NSB   22     NSF Organization and Staffing for Merit Review   24     Overall Conclusions   31 2   PLANNING MAJOR PROJECTS   39     Background: Project Planning and Budgeting at NSF   39     Long-Range Planning at NSF   40     Annual Budget Process   43     Major Project Planning and Budgeting   44     Capital Facilities Planning   49     Findings and Recommendations on Planning and Budgeting   51     Recommendation 1: Justification for Major Project Awards   54     Recommendation 2: Involvement and Support of Research Community in Planning   57 3   AWARDING MAJOR PROJECTS: CRITERIA AND REVIEW PROCEDURES   61     Background: The Merit Review Process at NSF   62     Current Review Criteria   63     Review and Selection for Major Project Awards   66     Findings and Recommendations on Criteria   70     Recommendation 3: Primacy of Technical Merit Criteria   71     Recommendation 4: Human Resource Development and Equal Opportunity as a Criterion   73

OCR for page R1
Major Award Decisionmaking at the National Science Foundation     Recommendation 5: Cost Sharing as a Criterion   74     NSF Procedures for Reviewing Proposals   76     Proposal Review Process   76     Policies and Procedures for Dealing with Bias and Conflict of Interest   80     Award Decisionmaking   80     Findings and Recommendation on Review Procedures   83     Recommendation 6: A Two-Phase Merit Review Process   84 4   AWARDING MAJOR PROJECTS: NSB ROLE, REVIEW PROCESS DESIGN, AND DECISION DOCUMENTATION   89     NSB Role and Procedures   89     Findings and Recommendations on the NSB Role   93     Recommendation 7: Reorienting the NSB Workload   93     Designing the Review and Solicitation Process   94     Proposal Review Planning Requirements   94     NSB Approval of Solicitation Announcements   97     Findings and Recommendations on Proposal Review Planning   101     Recommendation 8: Planning the Review Process and Criteria   101     Documenting Award Decisions   104     Findings and Recommendations on Award Documentation   106     Recommendation 9: More and Better Public Documentation of Award Decisions   106 5   RECOMPETITION OF AWARDS   109     Project Continuation at NSF   109     Findings and Recommendations   113     Recommendation 10: More Recompetitions   113 6   LOOKING TO THE FUTURE   117     APPENDIXES     A   Biographical Sketches of Panel Members   121 B   Major Awards Supported by NSF   127 C   Awards Approved by National Science Board, FY 1986–1992   137 D   Major Award Criteria from Recent Solicitation Announcements   150 E   The Ten Case Studies   155     REFERENCES   156

OCR for page R1
Major Award Decisionmaking at the National Science Foundation MAJOR AWARD DECISIONMAKING AT THE NATIONAL SCIENCE FOUNDATION

OCR for page R1
Major Award Decisionmaking at the National Science Foundation This page in the original is blank.