| ||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||
| Copyright © 2010. National Academy of Sciences. All rights reserved. Terms of Use and Privacy Statement |
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page R1
Major Award Decisionmaking at the National Science Foundation
MAJOR AWARD DECISIONMAKING AT THE NATIONAL SCIENCE FOUNDATION
Panel on NSF Decisionmaking for Major Awards
Committee on Science, Engineering, and Public Policy
National Academy of Sciences
National Academy of Engineering
Institute of Medicine
NATIONAL ACADEMY PRESS
Washington, D.C.
1994
OCR for page R2
Major Award Decisionmaking at the National Science Foundation
National Academy Press
2101 Constitution Ave., NW Washington, DC 20418
NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. It is a result of work done by an independent panel appointed by the Committee on Science, Engineering, and Public Policy, which has authorized its release to the public. The members of the panel responsible for this report were chosen for their special competencies and with regard for appropriate balance.
This report has been reviewed by a group other than the authors according to procedures approved by a Report Review Committee and by the Committee on Science, Engineering, and Public Policy. Both consist of members of the National Academy of Sciences, National Academy of Engineering, and Institute of Medicine.
The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Bruce M. Alberts is president of the National Academy of Sciences.
The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Robert M. White is president of the National Academy of Engineering.
The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences in its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Kenneth I. Shine is president of the Institute of Medicine.
The Committee on Science, Engineering, and Public Policy is a joint committee of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. It includes members of the councils of all three bodies.
Sponsor: This study was funded with Federal funds from the National Science Foundation (NSF) under contract number LPA-9123428. The contents of this report do not necessarily reflect the views or policies of the NSF, nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. Government.
Library of Congress Catalog Card Number 94-66065
International Standard Book Number 0-309-05029-4
Copyright 1994 by the National Academy of Sciences. All rights reserved.
Available from:
National Academy Press,
2101 Constitution Avenue, N.W., Washington, D.C. 20418
B-274
Printed in the United States of America
OCR for page R3
Major Award Decisionmaking at the National Science Foundation
PANEL ON NSF DECISIONMAKING FOR MAJOR AWARDS
ROBERT H. RUTFORD (Chair), President,
University of Texas at Dallas, and
Chairman of the Polar Research Board,
National Research Council
CLARENCE R. ALLEN, Professor of Geology and Geophysics, Emeritus,
California Institute of Technology
ALBERT A. BARBER, Special Assistant to the Chancellor,
University of California, Los Angeles—Washington D.C. Office
HARVEY BROOKS, Gordon McKay Professor of Applied Physics, Emeritus, in the Division of Applied Sciences, and Benjamin Peirce Professor of Technology and Public Policy, Emeritus,
John F. Kennedy School of Government, Harvard University
CHRISTOPHER COBURN, Director,
Public Technology Programs, Battelle Memorial Institute
SUSAN E. COZZENS, Associate Professor,
Science and Technology Studies Department, Rensselaer Polytechnic Institute
FRANK D. DRAKE, Professor of Astronomy and Astrophysics,
University of California, Santa Cruz, and
President,
SETI Institute
DONALD S. FREDRICKSON, President,
D.S. Fredrickson Associates, Inc.
FREDRICK S. HUMPHRIES, President,
Florida A&M University
ANITA K. JONES, Chair,
Department of Computer Science, University of Virginia (resigned May 31, 1993, to become Director of Defense Research and Engineering, Department of Defense)
LARRY K. MONTEITH, Chancellor,
North Carolina State University
DOUGLAS D. OSHEROFF, Professor,
Department of Physics, Stanford University
JUDITH A. RAMALEY, President,
Portland State University
LYLE H. SCHWARTZ, Director,
Materials Science and Engineering Laboratory, National Institute of Standards and Technology
Staff
MICHAEL McGEARY, Study Director
ELIZABETH BLOUNT, Senior Project Assistant
OCR for page R4
Major Award Decisionmaking at the National Science Foundation
COMMITTEE ON SCIENCE, ENGINEERING, AND PUBLIC POLICY
PHILLIP A. GRIFFITHS (Chair), Director,
Institute for Advanced Study
ROBERT McCORMICK ADAMS, Secretary,
Smithsonian Institution
BRUCE M. ALBERTS, President,
National Academy of Sciences
(Ex-Officio)
ELKAN BLOUT, Harkness Professor,
Department of Biological Chemistry and Molecular Pharmacology, Harvard Medical School
FELIX BROWDER,
Department of Mathematics, Rutgers University
ROBERT A. BURT, Alexander M. Bickel Professor of Law,
Yale Law School
DAVID R. CHALLONER, M.D., Vice President of Health Affairs,
University of Florida
ALBERT M. CLOGSTON,* Member,
Center for Material Sciences, Los Alamos National Laboratory
F. ALBERT COTTON, Distinguished Professor of Chemistry; Director,
Laboratory for Molecular Structure and Bonding, Texas A&M University
ALEXANDER H. FLAX, Senior Fellow,
National Academy of Engineering
RALPH E. GOMORY, President,
Alfred P. Sloan Foundation
THOMAS D. LARSON, Consultant
JOHN L. McLUCAS,* Aerospace Consultant
MARY JANE OSBORN,
Department of Microbiology, University of Connecticut Health Center
C. KUMAR N. PATEL, Vice Chancellor,
Research Programs, University of California, Los Angeles
PHILLIP A. SHARP, Head,
Department of Biology, Center for Cancer Research, Massachusetts Institute of Technology
KENNETH I. SHINE, President,
Institute of Medicine
(Ex-Officio)
ROBERT M. SOLOW, Institute Professor,
Department of Economics, Massachusetts Institute of Technology
H. GUYFORD STEVER, Member,
Carnegie Commission on Science and Technology
MORRIS TANENBAUM, Vice President,
National Academy of Engineering
OCR for page R5
Major Award Decisionmaking at the National Science Foundation
ROBERT M. WHITE, President,
National Academy of Engineering
(Ex-Officio)
SHEILA E. WIDNALL,* Associate Provost and Abby Rockefeller Mauze Professor of Aeronautics and Astronautics,
Massachusetts Institute of Technology
Staff
LAWRENCE E. McCRAY, Executive Director
BARBARA A. CANDLAND, Administrative Coordinator
*
Term expired June 30, 1993
OCR for page R6
Major Award Decisionmaking at the National Science Foundation
This page in the original is blank.
OCR for page R7
Major Award Decisionmaking at the National Science Foundation
Preface
Under the guidance of the National Science Board (NSB), the National Science Foundation (NSF) supports science and engineering research and education projects. NSF does not carry out these projects itself. It chooses the best proposals submitted by researchers in universities, colleges, and other research institutions. NSF uses a merit review process to identify the most promising projects to receive funding awards. Merit review has two distinctive features: it relies on independent outside peer reviewers to assess the quality of proposals, and it uses criteria that emphasize technical quality and also promote other goals of the nation’s research base such as equal opportunity, human resource development, and a broader geographic and institutional infrastructure.
Most of the awards made by NSF are to individuals or to small groups of scientists and engineers. This report addresses a small but important set of awards—very large awards for major research facilities, interdisciplinary research centers, and other large-scale research-related activities. Because of their budgetary impact and importance, it is critical that these major projects be carefully chosen on the basis of their contributions to the nation’s research enterprise and not according to political, bureaucratic, or other considerations. To achieve this, major award proposals are subjected to a merit review process. Merit review of major awards is more complicated and sometimes more controversial than that for individual investigator and small group awards.
This report is based on an 18-month study of the NSF-NSB system for making major awards. The study was undertaken by a broad-based expert group, which makes a series of recommendations
OCR for page R8
Major Award Decisionmaking at the National Science Foundation
for improving the planning, selection, and renewal of such awards. The recommendations appear in the chapters on these topics and are summarized in the executive summary.
The panel would like to thank the individuals who took the time to meet with us and share their knowledge, experiences, and views. Special thanks go to Alan M. Gaines, assistant for science and technology to the director of NSF, and NSF liaison official for this study, who made sure we had full and timely access to the publicly available information needed for the study.
The panel was briefed on the decisionmaking process for major awards at its first meeting by NSF and NSB officials: Frederick M. Bernthal, deputy director (chair, Director's Action Review Board); William C. Harris, assistant director for mathematical and physical sciences; Mary E. Clutter, assistant director for biological sciences; Joseph Kull, chief financial officer (executive secretary, NSB Committee on Programs and Plans); and Marta Cehelsky, NSB executive officer. Then-director Walter E. Massey met with the panel at a later meeting. Warren J. Baker, chair, NSB Committee on Programs and Plans, also briefed the panel on how the NSB reviews major award proposals. Former NSF director John B. Slaughter, who recently chaired the site selection committee for the NSF-supported Laser Interferometer Gravitational Wave Observatory, graciously provided his views on the evolution of the major award review process in an interview with a panel member.
The staff would also like to thank others at NSF who provided information and insight: Robert P. Abel, Charles N. Brownstein, Thomas N. Cooley, Peter W. House, Madeleine E. Hymowitz, James M. McCullough, Lynn Preston, and Joanna E. Rom. Susan E. Fannoney of the NSB staff was especially helpful in locating and providing NSB documents relating to NSB review and approval of 10 case study awards; Florence Heckman, NSF librarian, pointed the way to materials on the history of proposal review at NSF; and George Mazuzan, NSF historian, provided access to the historical files of NSF.
The panel appreciates the efforts of Michael McGeary, the study director, who pulled together a remarkable amount of information on
OCR for page R9
Major Award Decisionmaking at the National Science Foundation
NSF's merit review policies, procedures, and practices, as well as the 10 case study award decisions, and assisted the panel in drafting the report. Elizabeth Blount, staff associate, took care of the many administrative details of panel meetings and report production with skill, energy, and unfailing good cheer. Jeffrey D. Porro, consultant, edited early drafts of the report. We are also grateful for the support and assistance of the staff of the Committee on Science, Engineering, and Public Policy, including Lawrence E. McCray, executive director, and Barbara Candland, executive assistant, and of Philip M. Smith, executive officer, National Academy of Sciences. Florence Poullin was copy editor. National Academy Press staff who helped turn the report into a book included Stephen Mautner, Dawn Eichenlaub, and James Gormley.
Finally, I would like to thank the panel members for their willingness to devote considerable time to the study and for their contributions to this report. The recommendations reflect their vast experience and wisdom and their desire to give NSF and the NSB constructive advice for better decisionmaking on major awards. Panel member Anita K. Jones was able to participate substantially in the drafting of the report before resigning in May 1993 to become the director of defense research and engineering in the Department of Defense.
Robert H. Rutford
Chairman of the Study Panel
OCR for page R10
Major Award Decisionmaking at the National Science Foundation
This page in the original is blank.
OCR for page R11
Major Award Decisionmaking at the National Science Foundation
Contents
EXECUTIVE SUMMARY
1
1
MAJOR AWARDS AT NSF
13
Overview of Major Awards
15
Major Awards and Merit Review
19
Major Awards and the NSB
22
NSF Organization and Staffing for Merit Review
24
Overall Conclusions
31
2
PLANNING MAJOR PROJECTS
39
Background: Project Planning and Budgeting at NSF
39
Long-Range Planning at NSF
40
Annual Budget Process
43
Major Project Planning and Budgeting
44
Capital Facilities Planning
49
Findings and Recommendations on Planning and Budgeting
51
Recommendation 1: Justification for Major Project Awards
54
Recommendation 2: Involvement and Support of Research Community in Planning
57
3
AWARDING MAJOR PROJECTS: CRITERIA AND REVIEW PROCEDURES
61
Background: The Merit Review Process at NSF
62
Current Review Criteria
63
Review and Selection for Major Project Awards
66
Findings and Recommendations on Criteria
70
Recommendation 3: Primacy of Technical Merit Criteria
71
Recommendation 4: Human Resource Development and Equal Opportunity as a Criterion
73
OCR for page R12
Major Award Decisionmaking at the National Science Foundation
Recommendation 5: Cost Sharing as a Criterion
74
NSF Procedures for Reviewing Proposals
76
Proposal Review Process
76
Policies and Procedures for Dealing with Bias and Conflict of Interest
80
Award Decisionmaking
80
Findings and Recommendation on Review Procedures
83
Recommendation 6: A Two-Phase Merit Review Process
84
4
AWARDING MAJOR PROJECTS: NSB ROLE, REVIEW PROCESS DESIGN, AND DECISION DOCUMENTATION
89
NSB Role and Procedures
89
Findings and Recommendations on the NSB Role
93
Recommendation 7: Reorienting the NSB Workload
93
Designing the Review and Solicitation Process
94
Proposal Review Planning Requirements
94
NSB Approval of Solicitation Announcements
97
Findings and Recommendations on Proposal Review Planning
101
Recommendation 8: Planning the Review Process and Criteria
101
Documenting Award Decisions
104
Findings and Recommendations on Award Documentation
106
Recommendation 9: More and Better Public Documentation of Award Decisions
106
5
RECOMPETITION OF AWARDS
109
Project Continuation at NSF
109
Findings and Recommendations
113
Recommendation 10: More Recompetitions
113
6
LOOKING TO THE FUTURE
117
APPENDIXES
A
Biographical Sketches of Panel Members
121
B
Major Awards Supported by NSF
127
C
Awards Approved by National Science Board, FY 1986–1992
137
D
Major Award Criteria from Recent Solicitation Announcements
150
E
The Ten Case Studies
155
REFERENCES
156
OCR for page R13
Major Award Decisionmaking at the National Science Foundation
MAJOR AWARD DECISIONMAKING AT THE NATIONAL SCIENCE FOUNDATION
OCR for page R14
Major Award Decisionmaking at the National Science Foundation
This page in the original is blank.
Items in cart [0]
