3
The National Research Initiative Competitive Grants Program

The defining attributes of a successful competitive grants program are quality, fairness, relevance, and flexibility (Chubin and Hackett, 1990; Chubin, 1994; Kostoff, 1997a,b). High-quality research is novel, valuable, feasible, technically sound, and on occasion elegant. In practice, the quality of basic research is conveyed by the publication of research results in a peer-reviewed scientific journal. The quality of applied research might be demonstrated by a patent or by the successful implementation of research results, perhaps as new management practices, new products, new institutional arrangements, or new public policies. High-quality research also impacts science and technology themselves—that is, on the direction and development of the scientific enterprise and its technical implementation—an attribute closely related to relevance.

Fairness refers to the likelihood that a proposal will be evaluated with strict adherence to a set of evaluation criteria related to the quality and relevance of the proposed research. Race, sex, age, geography, and institutional affiliation must be effectively ignored when one is evaluating a proposal. Fairness also means that the review process must be open to independent examination and that each proposal is considered seriously and appropriately by a well-qualified group of reviewers. In practice, this works by ensuring that reviewers and panel members are broadly representative of the entire scientific community in a given



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 29
National Research Initiative: A Vital Competitive Grants Program in Food, Fiber, and Natural-Resources Research 3 The National Research Initiative Competitive Grants Program The defining attributes of a successful competitive grants program are quality, fairness, relevance, and flexibility (Chubin and Hackett, 1990; Chubin, 1994; Kostoff, 1997a,b). High-quality research is novel, valuable, feasible, technically sound, and on occasion elegant. In practice, the quality of basic research is conveyed by the publication of research results in a peer-reviewed scientific journal. The quality of applied research might be demonstrated by a patent or by the successful implementation of research results, perhaps as new management practices, new products, new institutional arrangements, or new public policies. High-quality research also impacts science and technology themselves—that is, on the direction and development of the scientific enterprise and its technical implementation—an attribute closely related to relevance. Fairness refers to the likelihood that a proposal will be evaluated with strict adherence to a set of evaluation criteria related to the quality and relevance of the proposed research. Race, sex, age, geography, and institutional affiliation must be effectively ignored when one is evaluating a proposal. Fairness also means that the review process must be open to independent examination and that each proposal is considered seriously and appropriately by a well-qualified group of reviewers. In practice, this works by ensuring that reviewers and panel members are broadly representative of the entire scientific community in a given

OCR for page 29
National Research Initiative: A Vital Competitive Grants Program in Food, Fiber, and Natural-Resources Research field.1 A fair process also ensures that grant applications are solicited from as wide a variety of applicants as possible. The remaining two attributes of a successful competitive grants program are relevance and flexibility. A relevant grants program provides funding for research that will most effectively further the goals of the program and meet national needs. Flexibility refers to the program’s capacity to shift in response to emerging fields of research. Almost by definition, emerging fields are highly relevant. However, flexibility also should be intrinsic to the research enterprise as a whole. Achieving flexibility can be difficult because of institutional inertia—the addition of individual programs adversely affect the resources remaining for other programs. Thus, a mechanism for periodically evaluating and revising programmatic areas is crucial in a successful competitive grants program. Other attributes of a successful program are related to specific practical aspects of the program’s implementation. For example, the program must give awards of sufficient size, duration, and number to attract high-quality scientists and support important research. If the awards are too small or too short, many highly qualified scientists are likely to ignore the program in favor of other funding sources. Similarly, grants must be numerous enough to attract high-quality scientists, especially those at the beginning of their research careers. Grant acceptance rates below 10% suggest low chances for success and discourage many scientists from participating as either grant writers or reviewers. At very low funding rates, the effort expended by scientists in writing unsuccessful applications exceeds that of the scientists whose research is supported. Some have argued that such a program is a net burden rather than an asset to the scientific community as a whole (Chubin, 1998). Clearly, there are tradeoffs in the management of any research program (Chubin, 1994; Baldwin and McCardle, 1996). Implementation issues are analyzed in more detail in chapter 6. QUALITY AND VALUE Quality and value are terms commonly used to rank types of activities, and research is no exception. Specific metrics can be used to assess quality; alternatively, testimonials can be obtained from various sources to tap perceptions of quality. The latter approach generally was used by this committee to assess the quality and value of NRI-supported research. The former approach is addressed later in a committee finding on evaluation of quality and program accountability. Evaluation of research has been a long-term challenge for the scientific community (NRC, 1998). In assessing the value of fundamental research, the private sector largely avoids such standard tools as return on investment and 1   Government science agencies use peer-review in many ways. For additional information on the use of peer-review, see Peer-review Practicies at Federal Science Agencies Vary, General Accounting Office, 1999.

OCR for page 29
National Research Initiative: A Vital Competitive Grants Program in Food, Fiber, and Natural-Resources Research instead focuses on scientific merit and accomplishments of the researcher in fields relevant to the funding unit. The value of research in academe also includes the learning experience for the investigator and the student seeking answers to compelling questions. Value can also reflect the intended use of the research. Government agencies motivated by the Government Performance and Results Act use various approaches to measure value (Kostoff, 1997b). The Department of Energy Office of Basic Energy Sciences uses scientific excellence, relevance to energy future, stewardship, and program management. The National Science Foundation (NSF) uses expected outcomes, including discoveries at the frontiers of science, connections between discoveries and their use, and development of a diverse globally oriented workforce of scientists and engineers. The National Institutes of Health (NIH) considers both broad outcomes—such as understanding of biologic functions and behaviors, and improvement in prevention, diagnosis, and treatment of diseases—and specific descriptions of the known and unknown to understand and improve human health as evidence of research value. Measuring the Quality and Value of NRI-Supported Research The measurements of quality and value of NRI-supported research that the committee reviewed or generated included surveys of applicants, awardees, and institutions (see appendixes B and C); testimony from chief scientists, representatives of the private sector, government agency staff, and other constituents; interviews with NRI staff (see appendix D) and NSF officials; and the experience of members of the committee (appendix E) and their colleagues. The numerical quality indicators included the proportion of applications funded and successful renewal rates (appendix F). Documentation of successfully completed projects and their use and application added to the overall assessment (see boxes 3-1, 3-2, 3-3 and appendix G). With regard to proposal quality, most of the surveys and testimonials agree in spirit with Harold D.Coble, representing the Council for Agricultural Science and Technology, who testified before the committee that panel reviewers consistently rated proposal quality as excellent. Similarly, past chief scientists unanimously concluded that NRI proposals were of high quality. Funded NRI projects generally have come from the top of the priority pool established by the review panels. For example (this is typical of most categories), the category “Plant Responses to the Environment” received 1,196 applications over the 8-year period 1991–1998. Of those, 52 (4%) were considered outstanding and were funded; 208 (17%) were regarded as having “high priority”, and 178 (86%) of them were funded; and 254 “medium-priority” applications (21%) were received, and 59 (23%) of them were funded. Only four “low-priority” projects out of about 300 so classified were funded over the 8-year period. An additional 341 applications were judged as “having some merit” or “do not fund” and were not funded. Over the 8-year period, 24% of applications were funded at some level. A total of $259 million in applications was received in the “Plant Responses to the Environment” category

OCR for page 29
National Research Initiative: A Vital Competitive Grants Program in Food, Fiber, and Natural-Resources Research over the period. The funds requested for approved grants totaled $59 million; but only $40 million was actually awarded. In all, only 15% of the $259 million requested was awarded (NRI Program Office, personal communications, September 1, 1998). In the committee’s experience, high-quality applications generally lead to high-quality research. John W.Suttie, past president of the Federation of American Societies for Experimental Biology, testified that the quality of research selected through peer-review has always been high. Richard A.Herrett, of the Agricultural Research Institute, indicated that the NRI has been productive in providing new techniques. He cited the American Society of Plant Physiologists observation on the importance of NRI funding in developing environmentally benign insecticides. Robert G.Zimbelman, chairman of the Coalition of Funding Agricultural Research Missions, testified that “the record shows the NRI has supported very high quality research and the results have been meaningful.” Kenneth E.Olsen, dairy and animal health specialist with the American Farm Bureau Federation, testified that “within the scientific community the NRI is well respected for top quality basic research.” Tony Cavalieri, of Pioneer Hi-Bred International, Inc. testified that we believe [NRI] is a very sound program and, in fact, may be the most effective example of USDA using their research money effectively. They have been effective in funding important work and in funding researchers who can do the work…. As far as the quality of funded research…it is obviously among the best work done in plant sciences. In individual discussions with the committee, several NSF personnel cited NRI research as excellent. A previous National Research Council report on the NRI (NRC, 1994) attested to the quality of the research by noting the “consensus among NRI staff and panel members and managers that ‘good to high’ characterizes the overall quality and relevance of the proposals being received and that the quality has been increasing each year” (p. 21). The report also stated that “the contribution of the NRI extensive review process to the quality of science should not be overlooked” (p. 20). In short, the stewardship of the NRI has been unquestionably high. BOX 3–1 Ethanol from Biomass: An Example of a Significant Scientific Advance from NRI Research If renewable biomass sources are to supply tomorrow’s energy needs, cost-effective technologies are needed. Researchers at the University of Florida have been making significant strides toward removing barriers to ethanol production from biomass. Using biologic approaches, Lonnie Ingram and colleagues are laying the foundation for ethanol production from biomass that is less costly and less capital-intensive. The general approach has been to develop different microorganisms in which useful traits for cellulose hydrolysis and sugar metabolism are combined with genes for ethanol production. With NRI support, genetically engineered Escherichia coli were developed to produce ethanol from all the monomer sugars that can be derived from plant cell walls.

OCR for page 29
National Research Initiative: A Vital Competitive Grants Program in Food, Fiber, and Natural-Resources Research The resulting strain, KO11, has been used to ferment hemicellulose sugars derived from several sources of biomass. Later awards have led to the integration of the ethanol production genes from Zymomonas mobilis into the chromosome of E. coli and the engineering of Klebsiella oxytoca for the simultaneous saccharification and fermentation of cellulose. More recent efforts are seeking to improve these microorganisms further, for example, by engineering the secretion of an Erwinia endoglucanase in E. coli and Klebsiella oxytoca. The aim is to reduce the requirement for supplemental cellulases from fungi, which are costly. Other research is to develop ethanol-producing biocatalysts with increased resistance to toxic products generated during the chemical hydrolysis of lignocellulose components. These compounds, sugar and lignin degradation products, currently must be removed by an expensive multistep process. Work continues to make the biomass conversion to ethanol more competitive, but a milestone has been reached. On October 20, 1998, BC International broke ground in Jennings, Louisiana, for a commercial scale plant to produce ethanol from agricultural waste. The plant, which has the capacity to produce 20 million gallons of ethanol per year, will run on bagasse (a residue from sugarcane refining) but has flexibility to use other feedstocks as well This first-of-its-kind plant is based on the genetically engineered KO11 bacterium developed by Dr. Ingram and colleagues. Competitive Research Grants Office and NRI Awards to Dr. Ingram 1986 Constructing of Lactose Utilizing Strains of Z. mobilis, $102,000, 3 years. 1988 Genetic Engineering of Alcohol Production in E.Coli, $110,000, 3 years. 1990 Genetic Engineering of Bacteria for Ethanol Production, $120,000, 3 years. 1992 In Vitro Analysis of Plant- Pathogenic Mycoplasma Like Organisms, $180,000, 3 years. 1995 Genetic Engineering of Bacteria for Ethanol Production, $180,000, 3 years. 1998 Engineering Bacteria for Fuel Ethanol Production, $179,000, 3 years. 1998 Advanced Ethanologenic Biocatalysts for Ligno Cellulose Fermentations, $298,935, 3 years. Patents US Patent 5,000,000, Ethanol production by Escherichia coli strains co-expressing Zymomonas PDC and ADH genes, Mar. 19, 1991. US Patent 5,028,539, Ethanol production using engineered mutant Escherichia coli, July 2, 1991. US Patent 5,162,516, Cloning and sequencing of the alcohol dehydrogenase gene from Zymomonas mobilis, Nov. 10, 1992. US Patent 5,424,202, Ethanol production by recombinant hosts, Jun. 13, 1995. US Patent 5,482,846, Ethanol production in Gram-positive microbes, Jan. 9, 1996. US Patent 5,602,030, Recombinant glucose uptake system, Feb.11, 1997. US Patent 5,821,093, Recombinant cells that highly express chromosomally-integrated heterologous genes, Oct. 13, 1998. Source: US Department of Agriculture, National Research Initiative Competitive Grants Program Office, Personal Communication, March 1999.

OCR for page 29
National Research Initiative: A Vital Competitive Grants Program in Food, Fiber, and Natural-Resources Research BOX 3–2 Porcine Reproductive and Respiratory Syndrome: An Example of a Significant Scientific Advancement from NRI Research Porcine reproductive and respiratory syndrome (PRRS), formerly known as mystery swine disease, first became apparent in the United States in 1986. In the next few years, it affected a majority of herds in every state that raised pigs. Economic losses were estimated at $250–300 per breeding-age female, so a typical 600-sow farm could lose $150,000–180,000 per outbreak, excluding other costs. By 1990, PRRS had appeared in Europe and was well on the way to becoming a global epidemic. The National Pork Producers Council considered the disease “the most important animal health problem affecting pigs”. Scientists knew that PRRS was caused by a highly contagious virus but understood little about how it was transmitted and where it replicated inside the animal. With funding from the NRI and the National Pork Producers Council, David Benfield, at South Dakota State University, and James Collins, at the University of Minnesota, began to address these issues. They identified the primary targets of the virus: the lung, heart, blood vessels, and lymph nodes. They also discovered that the virus is transmitted from pig to pig by close contact, such as nose-to-nose touching, by exposure to bodily secretions, by semen to female pigs, and from mother pig to fetus. An additional fmding—that PRRS virus replicates in an unknown primary target tissue and is then released into the bloodstream—was especially important because it suggested that a vaccine could be successful in fighting the disease. In partnership with private industry, Benfield and Collins developed a vaccine using a weakened form of the virus. They also developed monoclonal antibodies for use in laboratory tests of pig serum or tissue samples. These antibodies allow quick, accurate, and economical diagnosis of the disease, thus reducing treatment costs and producer losses. Although the vaccine and diagnostic resources described above are widely used today, PRRS continues to challenge producers and scientists as new strains of the virus emerge. The research funded by the NRI represents significant progress in understanding and combating the disease and laid the groundwork for continued advances in controlling this important animal-health problem. NRI Awards to Drs. Benfield and Collins 1992 Pathogenic Mechanism of Swine Infertility and Respiratory Syndrome Virus, $150,000. 1995 Mechanisms of Persistence of Porcine Reproductive and Respiratory Syndrome Virus, $210,553. 1998 Rushmore Conference: Mechanisms in the Pathogenesis of Enteric Diseases, S5,000. Patents US Patent 5,677,429, Monoclonal antibodies to the mystery swine disease virus, Oct. 14, 1997. US Patent 5,683,865, Vaccine for mystery swine disease and method for diagnosis thereof; Nov. 4, 1997. US Patent 5,846,805, Culture of swine infertility and respiratory syndrome virus in simian cells, Dec. 8, 1998. Source: US Department of Agriculture, National Research Initiative Competitive Grants Program Office, Personal Communication, March 1999.

OCR for page 29
National Research Initiative: A Vital Competitive Grants Program in Food, Fiber, and Natural-Resources Research BOX 3–3 Reduction in Fertilizer Use Profits Farmers and the Environment: An Example of NRI Research with Great Promise Pamela Matson and colleagues, at Stanford University, used NRI funds from the Forest/Range/Crop/Aquatic Ecosystems Program to counteract the consequences of the Green Revolution. Although this greening of the world has indeed increased crop yields (through irrigation and application of nitrogen fertilizers), it has done so in conjunction with unwanted greenhouse gas accumulations (specifically nitrous oxide), increased tropospheric levels of ozone and acid rain due to increases in nitric oxide, and deposition of nitrates from soils into freshwater and marine ecosystems, often resulting in eutrophication. However, Matson and colleagues’ research confirms that high yields are possible if less fertilizer is used, resulting in both lower application costs for farmers and lower social costs for the environment. The research took place in the Yaqui Valley of Sonora, Mexico, a major wheat-producing region that has helped to foster the Green Revolution with high productivity, using fertilizers and irrigation. The experiment included a control where no fertilizer was added, a conventional farming treatment currently in use in the region (adding nitrogen at 250 kg/ha), and three alternative farming methods that used less fertilizer before irrigation. All but the control had a yield of about 6 tons/ha, or 2.4 tons/acre, but there were significant differences in the amount of nitrogen released into the soil and air. The best alternative method, which applied 28% less nitrogen than the conventional method, resulted in a 69% reduction in total nitrogen loss and an approximate savings of $55–75/ha or $22–30/acre to the farmer. Thus, this alternative method is not only environmentally friendly, but also agronomically feasible and economically more desirable. NRI Award (directly related to this research) 1994 Forest/Rangeland/Crop/Aquatic Ecosystems Program (now Ecosystem Science Program), $431,112 for 3 years. Publications (directly related to this research) Matson P., Naylor R., Ortiz-Monasterio I., 1998. Integration of environmental, agronomic, and economic aspects of fertilizer management. Science 280:112–115. Source: US Department of Agriculture, National Research Initiative Competitive Grants Program Office, Personal Communication, March 1999. Novelty and Significance of Research The committee had difficulty in evaluating the novelty and impact of the 8-year research portfolio of the NRI. This difficulty is not peculiar to research organizations. Discussions with NSF indicated similar challenges. Clearly, however, a number of novel and significant results have occurred (see appendix G). The committee concluded that, although evidence suggests that NRI-funded research is novel and significant, detailed documentation is lacking. A definitive record of patents and publications resulting from NRI research is not available, nor is a continuing evaluation of current applications and

OCR for page 29
National Research Initiative: A Vital Competitive Grants Program in Food, Fiber, and Natural-Resources Research renewals as to their originality and impact. Information from the private sector about applications of the results of NRI-funded research also is lacking. FAIRNESS Fairness is the keystone of any successful competitive grants process. Participants must know that their proposal will be treated in a procedurally impeccable way that gives no advantage to a competitor and that is open to scrutiny without betraying confidential information. This delicate balance is maintained principally by staff, although panel managers can be instrumental. The committee believes that high-quality merit-based peer review is an essential component of a fair competitive grants program because it subjects all proposals to systematic scrutiny by knowledgeable specialists and requires ratings of quality and feasibility that constitute valuable advice to agency staff. In the following sections, the committee describes the NRI’s peer-review process and then evaluates its effectiveness. The NRI Peer-Review Process The NRI peer-review process is administered by “panel managers” who work as short-term consultants with US Department of Agriculture (USDA) staff and bring extramural scientific credibility to the program and its operation. Panel managers are instrumental in recruiting reviewers for the panels and reinforcing the perception that cutting-edge research is being considered for NRI funding. The participation of some 15,000 proposal reviewers and panelists (many with experience on both NSF and NIH panels) in the NRI peer-review process over 8 years attests to the community’s commitment to maintaining program quality (NRI Program Office, 1998). The rules and guidelines for panel composition are given in box 3-4. After examining all proposals submitted to the specific programs, the panel managers and the program directors determine the scientific expertise needed to evaluate each proposal and then assign proposals to appropriate panelists and ad hoc reviewers. Ad hoc reviewers are selected to extend the panelists’ scientific expertise. Careful attention is paid to avoid conflict of interest during reviewer selection. Each reviewer, whether a panel member or an ad hoc reviewer, is advised to treat the proposal with confidentiality. Before the panel meeting, copies of each proposal are mailed to the panel members and to the ad hoc reviewers for evaluation. Panelists write their own reviews of the proposals; then copies of the ad hoc reviewers’ comments and evaluations are mailed to panelists. This procedure allows the panelists to develop their own views regarding the proposals before reading the ad hoc reviews but gives them time to study the ad hoc reviews before the panel meets. Each proposal is evaluated by a primary and a secondary reviewer and by a third panel member who serves as “reader”. The primary and secondary reviewers provide written reviews. Evaluation criteria include scientific merit of

OCR for page 29
National Research Initiative: A Vital Competitive Grants Program in Food, Fiber, and Natural-Resources Research the proposal, qualifications of the investigators, and relevance of the proposal to program goals and to long-range improvements in the sustainability of US food and fiber. The same evaluation criteria are applied to renewal applications, with particular consideration of progress during the previous award period. BOX 3–4 Rules and Guidelines for Panel Composition In general, each panel is formulated as a true peer panel, representative of people eligible to apply to the particular program themselves. To the extent possible, the panel is representative of Demonstrated expertise in a relevant discipline. All major regions of the United States. Many types of institutions—land grant, private, public, industrial, and governmental institutions. Levels of academic position (academic and scientific maturity). Balance between new panelists and those who have previously served. Diversity in sex and minority status. Panel continuity is achieved by requesting part of a panel to serve in the following year. Continuity of the review process is also ensured by asking some of the ad hoc (mail) reviewers for a repeat review if a proposal is revised and resubmitted in the following year. A tentative list of panel members is approved by the chief scientist before any contact is made with prospective panel members. After the panel is assembled, it is submitted through the division director and chief scientist to the deputy administrator of competitive research grants and awards management of Cooperative State Research, Education and Extension Service for administrative approval. At the panel meeting, the primary reviewer of each proposal presents a synopsis of his or her review and evaluation to the panel. The secondary reviewer also presents his or her evaluation of the proposal. The third reviewer (the reader) is provided the opportunity to make additional evaluative comments but does not furnish a written review. The primary reviewer also summarizes the ad hoc reviewers’ evaluations. The proposal is then discussed by the panel and ranked by consensus. Ranking involves placing a proposal in one of six funding priority groups (outstanding, high, medium, and low priorities for funding; some merit; and not to be funded) and then ranking its merit relative to that of other proposals in the priority group. Before concluding, and after all proposals are reviewed, the panel reconvenes for an intense reassessment of the proposal rankings. The reranking session allows the panel to check and affirm that each proposal is properly and fairly ranked with respect to relative scientific merit. The degree of reranking varies, depending on the particular panel. The consistency of this process across panels is ensured by oversight and communication among the program directors and by visits of the program directors to panels in other programs.

OCR for page 29
National Research Initiative: A Vital Competitive Grants Program in Food, Fiber, and Natural-Resources Research The program director and panel manager record the final ranking of the panel. Proposals are generally funded according to this ranking until program funds are depleted. The best proposals receive most of the funding, but program managers on occasion are able to fund lower-ranked proposals with “strengthening” funds that can be used to award standard strengthening, postdoctoral, and new investigator grants in rank order.2 All award decisions are reviewed by the division director and the chief scientist. All applicants receive an anonymous critique of their proposals consisting of all reviewer comments. Because panels for each program meet annually, nonrecipients of awards have little opportunity to appeal in the year of submission. However, they may resubmit their proposals in the next year and include a rebuttal to the reviewers’ comments. As a consensus body exempt from the Federal Advisory Committee Act, review panels can speak with a single voice. This attribute lends credence to a panel’s recommendations and to the funding decisions made by NRI staff. By retaining 30%-40% of members from the previous year, the review panels embody the collective wisdom of research communities. Yet they must be sensitive to accusations of functioning as “old-boy” networks. They vary, too, in their willingness to advise staff on such issues as where to cut the budget in high-ranked proposals. Some panels refuse to discuss budgets; others routinely weigh such considerations. Evaluation of the NRI Peer-Review Process By and large, the committee’s survey reveals that recipients and nonrecipients of NRI awards consider the NRI review process fair and effective (see appendix C). Most of those surveyed viewed the NRI as using a fair peer-review process to select proposals for funding, including 97% of awardees, 74% of nonrecipients, and 84% of participants in land grant institutions (appendix C). Respondents with review-panel experience in other competitive grants programs were especially complimentary of the NRI peer-review process. The four former chief scientists interviewed for this study also expressed great confidence in the peer-review and evaluation process (appendix C). Similarly, despite some criticisms of the length of the review process, panel members representing NRI stakeholders generally viewed the process as satisfying and fair (appendix C). Overall, scientists, panel members, and administrators judged the fairness of the peer-review process as exceptional, and they considered peer-review the best way to assess projects and distribute funds. 2   See chapter 4 for a discussion of NRI’s “strengthening” awards.

OCR for page 29
National Research Initiative: A Vital Competitive Grants Program in Food, Fiber, and Natural-Resources Research RELEVANCE AND RESPONSIVENESS A relevant grants program provides funding for research that will further the goals of the program and meet national needs. The committee’s survey indicates that nearly all respondents (awardees, nonrecipients, and those in land grant institutions and industry) think that the program has contributed to the NRI mission of generating fundamental and applied research and fostering the development of future scientists with strong backgrounds in food and fiber (appendix C). It should be noted, however, that substantial fractions of respondents indicated that the NRI contribution has been less than they expected (16% of recipients; 27% of nonrecipients; and 43% of people in land grant institutions). Survey respondents, chief scientists, and those who testified before the committee repeatedly cited two main factors that have limited the NRI’s ability to reach its full potential. First, nearly all expressed the view that the total budget for the program was inadequate and that awards were too short, too few, and too small (appendix C). Second, many of the respondents indicated that the NRI priority-setting process was not clear (appendix C). The committee discusses those two issues more fully in chapters 6 and 5, respectively. In addition to being relevant, a successful research program must be responsive. How any program reflects intellectual developments while being sensitive to appropriations (an external constraint) and budgeting (an internal constraint) is a test of its responsiveness. Many communities see the NRI as their program; each has its own expectation of priorities. To be successful, NRI staff must manage relations with each community, and this is especially difficult in a constrained funding environment. One fundamental characteristic of a responsive research program is its ability to facilitate research in new and emerging fields through the creation of new programs and the consolidation of declining programs. NRI division directors and program managers who met with the committee assert that new panels and programs are created in response to the number of proposals received in a field, rather than in response to political pressure from commodity or other groups. The narrative history that NRI staff provided to the committee suggests, however, that the evolution of the six mandated divisions into the 26 current programs was overwhelmingly the result of upper-level management decisions (especially spinoffs of existing programs into the Agriculture Systems program) rather than the result of proposal submissions. The committee discusses this issue in more detail in chapter 5 as part of its analysis of the NRI priority-setting process. A second important factor inherent in a program’s responsiveness involves the allocation of funds to support different elements of the program’s mission (such as basic research, mission-oriented research, and human development). For example, during their discussions with the committee, some NRI staff questioned whether the percentage of NRI funding spent on “strengthening” grants and the support of postdoctoral fellows and graduate students (currently 25%) has been adequate and whether such awards have been made appropriately. The committee discusses that issue in more detail in chapter 4 as part of its analysis of the role and scope of the NRI program.

OCR for page 29
National Research Initiative: A Vital Competitive Grants Program in Food, Fiber, and Natural-Resources Research A third important factor related to a program’s responsiveness is how the program deals with the inherent tradeoff between the number of proposals funded and the average funding provided for each grant. NRI program staff have expressed a resistance to increasing grant size because such increases would require a decrease in the number of researchers receiving any support (assuming no increase in the program’s budget). As a result, individual award amounts currently average 60% of requested amounts. NRI data show a decline in proposal submissions in recent years, which could be a result of growing doubts about the program’s viability (see survey results in appendix C). The committee discusses that issue in more detail in chapter 6 as part of its analysis of NRI funding. SUMMARY FINDINGS A successful grants program contains elements of value, relevance, quality, fairness, and flexibility. The committee finds that the proposals to the NRI and the research conducted by scientists who received NRI grants are both of high quality. That finding is based on the results of the committee’s survey of applicants, awardees, administrators of land grant institutions, and industry; the views of former chief scientists and individuals from federal agencies; and the personal perspectives of committee members and their colleagues. Documentation of successfully completed projects and their use and application was factored into the committee’s assessment, as were the proportion of applications funded and successful renewal rates. The committee believes that the NRI could improve its record by documenting the value of research funded. The NRI does not keep a definitive record of patents and publications that result from NRI research. Nor is there a running evaluation of originality and significance of current applications and renewals. Although the committee has found based on its surveys that funded applications are of high quality, the NRI lacks a tracking system of critical factors needed for self-evaluation or for effective reporting of research accomplishments to outside groups, which would create a feedback system to establish value. The committee views the NRI as a model of merit-based peer-reviewed research in USDA. Because it uses a competitive review process to rank proposals, however, the NRI remains outside the mainstream USDA culture of formula funding. Through conscientious stewardship, the NRI has been successful in generating fundamental and applied research and fostering the development of future scientists with strong backgrounds in food and fiber. The NRI program is, however, not as responsive or flexible as it could be. Proposal submissions have declined in recent years, owing in part to concern over the viability of the program and in part to the program’s resistance to increase the size of grants because such an increase would come at the cost of supporting fewer researchers.