The Measure of STAR
Review of the U.S. Environmental Protection Agency’s Science To Achieve Results (STAR) Research Grants Program
THE NATIONAL ACADEMIES PRESS
Washington, D.C.
www.nap.edu
THE NATIONAL ACADEMIES PRESS
500 Fifth Street, NW Washington, DC 20001
NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance.
This project was supported by Contract 68-C-01-119 between the National Academy of Sciences and the U.S. Environmental Protection Agency. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the authors and do not necessarily reflect the view of the organizations or agencies that provided support for this project.
International Standard Book Number 0-309-08938-7 (Book)
International Standard Book Number 0-309-51674-9 (PDF)
Additional copies of this report are available from:
The National Academies Press
500 Fifth Street, NW Box 285 Washington, DC 20055 800-624-6242 202-334-3313 (in the Washington metropolitan area) http://www.nap.edu
Copyright 2003 by the National Academy of Sciences. All rights reserved.
Printed in the United States of America
THE NATIONAL ACADEMIES
Advisers to the Nation on Science, Engineering, and Medicine
The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Bruce M. Alberts is president of the National Academy of Sciences.
The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Wm. A. Wulf is president of the National Academy of Engineering.
The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Harvey V. Fineberg is president of the Institute of Medicine.
The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy’s purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Bruce M. Alberts and Dr. Wm. A. Wulf are chair and vice chair, respectively, of the National Research Council
COMMITTEE TO REVIEW EPA’S RESEARCH GRANTS PROGRAM
Members
HAROLD A. MOONEY (Chair),
Stanford University, Stanford, CA
RAYMOND C. LOEHR (Vice Chair),
University of Texas, Austin
ANDERS W. ANDREN,
University of Wisconsin, Madison
EDWIN H. CLARK, II,
Clean Sites, Inc., Washington, DC
COSTEL D. DENSON,
University of Delaware, Newark
JOHN C. ELSTON,
Spring Lake Heights, NJ, retired from New Jersey State Department of Environmental Protection
CAROL J. HENRY,
American Chemistry Council, Arlington, VA
MARTHA A. KREBS,
Science Strategies, Los Angeles, CA
RICHARD F. LEE,
Skidaway Institute of Oceanography, Savannah, GA
GERALD VAN BELLE,
University of Washington, Seattle
TERRY F. YOUNG,
Environmental Defense, Oakland, CA
LAUREN A. ZEISE,
California Environmental Protection Agency, Oakland
Staff
EILEEN N. ABT, Project Director
ROBERTA M. WEDGE, Program Director for Risk Analysis
NORMAN GROSSBLATT, Senior Editor
KELLY CLARK, Assistant Editor
JENNIFER E. SAUNDERS, Research Assistant
MIRSADA KARALIC-LONCAREVIC, Research Assistant
LUCY V. FUSCO, Senior Project Assistant
Sponsor
U.S. ENVIRONMENTAL PROTECTION AGENCY
BOARD ON ENVIRONMENTAL STUDIES AND TOXICOLOGY1
Members
GORDON ORIANS (Chair),
University of Washington, Seattle
JOHN DOULL (Vice Chair),
University of Kansas Medical Center, Kansas City
DAVID ALLEN,
University of Texas, Austin
THOMAS BURKE,
Johns Hopkins University, Baltimore, MD
JUDITH C. CHOW,
Desert Research Institute, Reno, NV
CHRISTOPHER B. FIELD,
Carnegie Institute of Washington, Stanford, CA
WILLIAM H. GLAZE,
Oregon Health and Science University, Beaverton
SHERRI W. GOODMAN,
Center for Naval Analyses, Alexandria, VA
DANIEL S. GREENBAUM,
Health Effects Institute, Cambridge, MA
ROGENE HENDERSON,
Lovelace Respiratory Research Institute, Albuquerque, NM
CAROL HENRY,
American Chemistry Council, Arlington, VA
ROBERT HUGGETT,
Michigan State University, East Lansing
BARRY L. JOHNSON
Emory University, Atlanta, GA
JAMES H. JOHNSON,
Howard University, Washington, DC
JAMES A. MACMAHON,
Utah State University, Logan
PATRICK V. O’BRIEN,
Chevron Research and Technology, Richmond, CA
DOROTHY E. PATTON,
International Life Sciences Institute, Washington, DC
ANN POWERS,
Pace University School of Law, White Plains, NY
LOUISE M. RYAN,
Harvard University, Boston, MA
JONATHAN M. SAMET,
Johns Hopkins University, Baltimore, MD
KIRK SMITH,
University of California, Berkeley
LISA SPEER,
Natural Resources Defense Council, New York, NY
G. DAVID TILMAN,
University of Minnesota, St. Paul
CHRIS G. WHIPPLE,
Environ Incorporated, Emeryville, CA
LAUREN A. ZEISE,
California Environmental Protection Agency, Oakland
Senior Staff
JAMES J. REISA, Director
DAVID J. POLICANSKY, Associate Director
RAYMOND A. WASSEL, Senior Program Director for Environmental Sciences and Engineering
KULBIR BAKSHI, Program Director for the Committee on Toxicology
ROBERTA M. WEDGE, Program Director for Risk Analysis
K. JOHN HOLMES, Senior Staff Officer
SUSAN N.J. MARTEL, Senior Staff Officer
SUZANNE VAN DRUNICK, Senior Staff Officer
EILEEN N. ABT, Senior Staff Officer
ELLEN K. MANTUS, Senior Staff Officer
RUTH E. CROSSGROVE, Managing Editor
OTHER REPORTS OF THE BOARD ON ENVIRONMENTAL STUDIES AND TOXICOLOGY
Cumulative Environmental Effects of Alaska North Slope Oil and Gas Development (2003)
Estimating the Public Health Benefits of Proposed Air Pollution Regulations (2002)
Biosolids Applied to Land: Advancing Standards and Practices (2002)
Ecological Dynamics on Yellowstone’s Northern Range (2002)
The Airliner Cabin Environment and Health of Passengers and Crew (2002)
Arsenic in Drinking Water: 2001 Update (2001)
Evaluating Vehicle Emissions Inspection and Maintenance Programs (2001)
Compensating for Wetland Losses Under the Clean Water Act (2001)
A Risk-Management Strategy for PCB-Contaminated Sediments (2001)
Acute Exposure Guideline Levels for Selected Airborne Chemicals (3 volumes, 2000-2003)
Toxicological Effects of Methylmercury (2000)
Strengthening Science at the U.S. Environmental Protection Agency (2000)
Scientific Frontiers in Developmental Toxicology and Risk Assessment (2000)
Ecological Indicators for the Nation (2000)
Modeling Mobile-Source Emissions (2000)
Waste Incineration and Public Health (1999)
Hormonally Active Agents in the Environment (1999)
Research Priorities for Airborne Particulate Matter (4 volumes, 1998-2003)
Ozone-Forming Potential of Reformulated Gasoline (1999)
Arsenic in Drinking Water (1999)
The National Research Council’s Committee on Toxicology: The First 50 Years (1997)
Carcinogens and Anticarcinogens in the Human Diet (1996)
Upstream: Salmon and Society in the Pacific Northwest (1996)
Science and the Endangered Species Act (1995)
Wetlands: Characteristics and Boundaries (1995)
Biologic Markers (5 volumes, 1989-1995)
Review of EPA's Environmental Monitoring and Assessment Program (3 volumes, 1994-1995)
Science and Judgment in Risk Assessment (1994)
Pesticides in the Diets of Infants and Children (1993)
Dolphins and the Tuna Industry (1992)
Science and the National Parks (1992)
Human Exposure Assessment for Airborne Pollutants (1991)
Rethinking the Ozone Problem in Urban and Regional Air Pollution (1991)
Decline of the Sea Turtles (1990)
Copies of these reports may be ordered from The National Academies Press
(800) 624-6242 or (202) 334-3313
Preface
The Environmental Protection Agency (EPA) is a mission agency established in 1970 to protect human health and to safeguard the natural environment. EPA’s regulatory and decision-making role requires that the agency have access to the best available science that is relevant to its mission. In an effort to improve the scientific foundation of its decision-making process, the agency established the Science To Achieve Results (STAR) research grants program in 1995.
The STAR program is a competitive, peer-reviewed, extramural research grants program created to encourage interagency collaboration and to increase EPA’s access to the nation’s best scientists and engineers in academic and nonprofit research institutions. The program supports research in a variety of fields relevant to EPA’s mission, ranging from human health protection to environmental preservation. It is designed to maximize the independence of the researchers supported and to provide an equal opportunity for all researchers to qualify for support.
In 2000, EPA asked the National Research Council to conduct an independent assessment of the STAR program. In response, the Research Council established the Committee to Review EPA’s Research Grants Program. In this report, the committee analyzes information provided by EPA, STAR grant recipients and fellows, and other sources to assess the program’s scientific merit, effect on the agency’s policies and decisions, and overall relevance to EPA’s mission. In addition, the committee compares some of the procedural aspects of the STAR program with those of basic and applied research grant programs of other agencies. Finally, the committee recom
mends ways to enhance the program and improve data collection for future program evaluations.
This report has been reviewed in draft form by persons chosen for their diverse perspectives and technical expertise in accordance with procedures approved by the Research Council’s Report Review Committee. The purpose of this independent review is to provide candid and critical comments that will assist the institution in making its published report as sound as possible and to ensure that the report meets institutional standards of objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process. We wish to thank the following people for their review of this report: William Glaze, University of North Carolina, Chapel Hill, North Carolina; Bernard D. Goldstein, University of Pittsburgh, Pittsburgh, Pennsylvania; Mark A. Harwell, Florida A&M University, Tallahassee, Florida; George Lucier (retired), National Institute of Environmental Health Sciences, Pittsboro, North Carolina; Perry L. McCarty, Stanford University, Stanford, California; Paul G. Risser, The Oklahoma State System of Higher Education, Oklahoma City, Oklahoma; Joan B. Rose, Michigan State University, East Lansing, Michigan; Jane Warren, Health Effects Institute, Boston, Massachusetts.
Although the reviewers listed above have provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations, nor did they see the final draft of the report before its release. The review of this report was overseen by Bailus Walker, Jr., Howard University Medical Center, Washington, D.C. Appointed by the Research Council, he was responsible for making certain that an independent examination of the report was carried out in accordance with institutional procedures and that all review comments were carefully considered. Responsibility for the final content of the report rests entirely with the committee and the institution.
The committee gratefully acknowledges the following people for making presentations to it: Peter Preuss, Paul Gilman, Timothy Oppelt, John Bachmann, Patricia Bradley, Elizabeth Bryan, William Farland, and James Hanlon, EPA; Robert Huggett, Michigan State University; John Wanska, General Accounting Office; Jerry Elwood, Department of Energy; Peter Johnson, U.S. Department of Agriculture; Claudia Thompson, National Institute of Environmental Health Sciences; Susan Cozzens, Ann Bostrom, and Alan Porter, Georgia Institute of Technology; Penny Firth, National Science Foundation; and Nils Newman, IISCO. The committee also wishes to thank the following, who provided further background information:
Theodore Just, Jack Puzak, Jeffrey Harris, Manju Gupta, Terry Simpson, Matthew Clark, Gina Perovich, Shirley Hamilton, and James Gentry, EPA; Claudia Magdalena Abendroth, Office of Management and Budget; Jeanne Powell, National Institute of Standards and Technology; Julia Melkers, University of Chicago; Christopher Allen, University of Vermont; Robert Selden, Air Force Scientific Advisory Board; Leslie Peasant, Air Force Office of Scientific Research; James Coleman, Nevada National Science Foundation; and Deborah Stine and Scott Weidman, National Research Council. In addition, the committee gives special thanks to the EPA project officers in three research programs—Endocrine Disruptors, Elaine Francis and David Reese; Particulate Matter, Gail Robarge and Stacey Katz; and Ecological Indicators, Barbara Levinson—who were available to discuss their programs in detail. We are also grateful to the many National Center for Environmental Research (NCER) staff who invested extensive time and effort in responding to all the committee’s requests for information. Finally, we appreciate the contributions of the numerous EPA STAR grantees and fellowship recipients who provided input on the program.
The committee is grateful for the assistance of the Research Council staff in preparing this report: Eileen Abt, project director; James Reisa, director of the Board on Environmental Studies and Toxicology; Roberta Wedge, program director for risk analysis; Jennifer Saunders and Mirsada Karalic-Loncarevic, research assistants; Ruth E. Crossgrove, managing editor; Norman Grossblatt, senior editor; Kelly Clark, assistant editor; Lucy Fusco and Bryan Shipley, senior project assistants; and Robert Policelli and Tamara Dawson, project assistants.
Finally, I thank the members of the committee for their dedicated efforts throughout the development of this report.
Harold Mooney
Chair, Committee to Review EPA’s Research Grants Program