Investing in the National Research Initiative: An Update of the Competitive Grants Program in the U.S. Department of Agriculture (1994)

In comparison with other federal agricultural programs, the NRI is both very young and very small. Inevitably, experience will dictate the need for modifications in both administration and program content. The NRI chief scientists and staff members have engaged those within and outside the research community who express concerns about the development of the NRI. At times, it may appear that the attention the NRI receives is out of proportion to its size in the USDA research budget ($100 million out of $1.8 billion annually). However, focus on the NRI—sometimes, it seems, to the exclusion of intramural, formula, and special grants—is emblematic of the recognition that competitive grants are a powerful tool supporting development of science and technologies that will carry agriculture into the next century. The NRI is indeed a high-stakes investment.

Still, the board finds there are some critical concerns that transcend the category of "growing pains" for the NRI. In the forum it held in October 1993, the board made a particular effort to explore these ideas, which may be characterized as follows:

• Does the small size of the NRI indicate that it has minor impact on progress in agricultural science? Is there a means for evaluating systematically the contributions made by the NRI to demonstrate its current role and future potential?
• Do the current program areas encourage inordinate reliance on single-discipline approaches to research that impede innovation and reduce

• relevance? In particular, is there a broader role for the social sciences?
• If a specific-discipline approach is overemphasized, are the provisions for multidisciplinary research correspondingly under emphasized? How might such an imbalance be rectified?
• In order to carry out the congressional mandate, should the evaluation of each grant application explicitly include its contribution to sustainability as a paramount criterion for determining acceptance? How should the mandate be addressed?

The NRI is, as has been noted above, a small part of all USDA research expenditures and an even smaller part of all U.S. spending for agricultural research, including federal and state programs and those of the private sector. At its current scale, can the program be said to contribute in a major way to the advancement of scientific knowledge about agriculture? At the forum, Roger Salquist, the chief executive officer of Calgene, observed, "Quite simply, it's too little money and too diffuse." Compared to the scale of other public and private research investments, the parceling out of an average of 800 grants of $125,000 for a duration of about 2 years or less seems modest. Although private agricultural and biotechnology firms understand the importance of publicly funded research, the NRI may not figure as prominently as established programs, such as formula funds, if at all, in their perception of the public system's contribution to scientific innovation.

The answer to the question about the significance of the NRI is straightforward. Investments in agricultural research have been shown to yield returns in excess of 40 percent (Alston et al., 1994), with the stream of returns occurring some years after the initial research investment has been made and new technologies have been developed and adopted. Consequently, the benefits of the recent NRI grants will be realized in future years, and, using the past as a guide, these gains will be large. However, as with any investment, larger principal yields larger returns, and the overall size of the NRI constrains the magnitude of its contribution to national goals for agriculture, human health, and the environment.

Although rate-of-return analyses demonstrate impressive returns to agricultural research measured in the aggregate, there are other ways that the contributions of the NRI could be evaluated. To date, few of the administrative resources available to the NRI have been devoted to tracking its output, although its output could be easily documented. For example, success in stimu-

lating and supporting new and better science could be shown by keeping track of the number of peer-reviewed publications that resulted from NRI-funded research. Another measure of the impact of the NRI on the overall productivity of the science community might be the number of postdoctoral researchers who receive support from an NRI grant as well as the number of undergraduate and graduate students who are able to become involved in research as a result of NRI funding. The availability of NRI funds may help scientists leverage support from home institutions or other funding agencies, and it might be possible to measure this success by noting each grant proposal's identification of other sources of funds. In terms of contributions to technological development, the number of technology licenses and patents granted based on NRI research would provide evidence of its value.

In 1989, the board identified a series of questions that could be central to ongoing evaluations. Judgments about the ultimate quality of the scientific contribution can only be made as time goes by, as discussed. However, intermediate progress could be examined.

• Are science and technology priorities within the major program areas defined insightfully and do they relate to national needs?

In a time of federal fiscal restraint, additional funding for the NRI will come only from redirection of spending on other activities. The board has made clear its view that expansion of the NRI competitive-grants program should not come at the expense of other agricultural research carried out intramurally or in universities. As a category, agricultural research consistently provides large returns to farmers and to consumers; it would be unwise to reduce funding of such a productive investment. However, there may be other areas of federal spending for which the returns to the nation are lower or for which the original purpose has been served. In Investing in Research, the board suggested that the outlays currently devoted to the commodity price support programs might be targeted for reduction, with part of the savings going to reduce the federal deficit and part going to reinvestment in a high-return program—the NRI. U.S. agriculture is entering an era of liberalized world trade in commodities and agricultural products; decades-old price-support programs may be dismantled to comply with new trading rules. But the need to maintain the competitiveness of the agricultural sector is as compelling as ever. Only now, with increased reliance on the marketplace rather than on government subsidies, that competitive edge is maintained by gains in productivity, in holding down the costs of production while protecting environmental quality. Ultimately, productivity gains depend on advances in science and translation into new technologies. From this perspective, the NRI will be a part of the foundation for future U.S. success in international markets.

The NRI funds research in six broad categories: (1) natural resources and environment; (2) nutrition, food quality, and health; (3) animal systems; (4) plant systems; (5) markets, trade, and (5) policy; and processing for value-added products. Some critical observers have described the program as divided into areas that represent a particular discipline or group of disciplines. For example, animal systems research is addressed by animal scientists; nutrition, food quality, and health are addressed by nutritionists and food scientists; and natural resources and the environment by soil scientists and ecologists. Critics charge that although the NRI program has been set up to avoid sorting research proposals into disciplinary compartments, efforts have not been entirely successful. They point out that only the program area for markets, trade, and policy has been perceived as the realm of social scientists.

According to NRI program data, it is indeed the case that the markets, trade, and policy grants division has been dominated by social scientists in writing requests for proposals, submitting proposals, and participating on peer review panels. For the purposes of this discussion, the social sciences are de-

fined as those that study human systems, human institutions, and human behaviors. These disciplines include economics, human ecology, sociology, psychology, anthropology, political science, and rural and community development.

Those who argue for an expanded role for the social sciences do not necessarily advocate increased funding within the markets, trade, and policy division but rather for greater participation across all six NRI program areas. Kitty R. Smith, director of Policy Studies at the Wallace Institute for Alternative Agriculture, summarized this view in her remarks at the board's October 1993 forum.

It is also the case that social science research can help identify and quantify the benefits that the NRI brings to consumers, to farmers, and to industry.

The association of issue areas with particular disciplines exists despite efforts of the NRI program to design requests for proposals to avoid it and is certainly not unique to the NRI. The concentration of the social scientists' participation in one grant category indicates a lack of complete success in this regard. Critics argue that the association of NRI program areas with particular disciplines constrains the contributions of more than just the social sciences. In this view, the assumption of a disciplinary focus by issue area leads to competition among disciplines and to factionalism that inhibits the advance of science and, not coincidentally, of the NRI itself. Further, it may well be that disciplinary compartmentalization, whether real or perceived, precludes innovation that stems from the attempt to address problems that cross disciplinary boundaries.

In its 1989 proposal for an expanded competitive grants program, the board argued that an enhanced program would provide significant new opportunities for supporting multidisciplinary research. Such work combines expertise from two or more disciplines into a shared focus on a common research

problem that has an integrated plan of study. A multidisciplinary project requires research in the disciplines and at the same time draws research and results from the disciplines to form a study that integrates both the disciplines and the results to examine systematically the various facets as well as the totality of the problem. As used here, multidisciplinary research designates both cross-disciplinary and interdisciplinary research, even though the three terms have somewhat different meanings (National Research Council, 1989). For example, developing sustainable animal agricultural systems requires research in agronomy and soil science, ecology and ecosystems analysis, engineering, animal nutrition, population and community biology, and economics, at a minimum.

Investing in Research recommended that to realize the full potential of science and technology in agricultural, food, and environmental research, the USDA competitive grants program should direct up to 50 percent of its support to multidisciplinary research, through multidisciplinary team grants, both fundamental and mission-linked. This emphasis is meant to stimulate more mul-

tidisciplinary team research and to encourage it strongly among senior scientists. The report describes two types of multidisciplinary research: (1) fundamental multidisciplinary team grants, conceived of as the involvement of at least two senior scientists as principal investigators; and (2) multidisciplinary, mission-linked teams involving about four senior scientists.

The board recognized that encouraging and evaluating multidisciplinary research would not be easy. In particular, it noted that multidisciplinary team research presents a number of conceptual and practical difficulties. Chief among them are issues of leadership, management, coordination, rewards, and satisfaction. In addition, granting agencies have customarily awarded grants to single investigators within one scientific discipline; thus, the reviewing mechanisms are generally organized on a single-discipline basis. Involving reviewers from several different disciplines is considerably more difficult. "Notwithstanding the difficulties, multidisciplinary research is clearly worth doing because of the multifaceted nature of the problems—both the fundamental and the more applied problems that are common in the agricultural, food, and environmental system. It is also worthwhile because of the unexpected synergism and creativity that good collaboration may generate," the board concluded (National Research Council, 1989, p. 40).

The board originally recommended that 50 percent of all grants be multidisciplinary awards. In FY 1993, multidisciplinary awards represented 34 percent of total NRI grants. Clearly, constraints on total funding jeopardize growth in this activity, but it is just as important to ensure that the grants supported do indeed represent productive investments. It is noteworthy that in 1993, NRI, in cooperation with the National Science Foundation and the Department of Energy, initiated a joint program on collaborative research. This is a positive step, but a number of issues concerning multidisciplinary research will require ongoing assessment. Although these are discussed in context to the NRI, such impediments may be characteristic of multidisciplinary work under any circumstances.

Broad, multidisciplinary projects are intrinsically difficult to review. Frequently, two or three study groups must evaluate a project, thus requiring that it pass muster more than once. With existing funding constraints, this situation of "double jeopardy" unfairly decreases the probabilities of approval, while increasing the administrative costs of review. To the extent that industry faces the need to conduct and expeditiously evaluate the prospects for multidiscipli-

nary research, the advice of experienced private firms might be sought by NRI managers as an additional source of expertise.

Time and energy are required to assemble multidisciplinary teams. Each expert on the team must become familiar with the language and culture of the others, and frequently, adjustments must be made to smooth over any differences. The start-up of a multidisciplinary team presents two difficulties: (1) bringing the team together to write a fundable proposal and (2) finding ways to keep the team working together efficiently. One option that might further encourage multidisciplinary proposals is to formulate a modest planning grant with a 1-year duration. Planning grants would be aimed primarily at proposal assembly but in some cases might include preliminary experiments or data collection.

Typically, multidisciplinary studies require more funding per year than single-discipline studies, and funding needs are increased by their typically longer duration. Reviewers, study groups, and grant managers sometimes are reluctant to recommend funding of a large study when that level of funding could instead support two or three smaller studies. Underfunding of multidisciplinary projects is counterproductive. Funding is usually needed for 4 to 5 years because of the investment of time at start-up as well as time at the end of the study to integrate data.

Contributions of some team members, especially beginning or untenured scientists, can become buried in large, multidisciplinary projects, particularly if one or two of the other participants are established senior investigators. Mechanisms are needed to identify roles clearly, especially in publications. Department heads, division leaders, and tenure committees need to be sensitive to issues involving individual contributions to multidisciplinary projects and appropriately supportive of the scientists pursuing multidisciplinary studies. Mechanisms for continuing funding for multidisciplinary studies are not yet optimal. Continuation grants for single-discipline studies are common if excellent work is being done. Continuation grants for multidisciplinary studies can be more difficult to achieve because of the inherent delays encountered when multiple groups must collate and evaluate data, prepare and approve publications, and work through other project-related issues jointly. Criteria for evaluating multidisciplinary projects might be somewhat modified compared with standard evaluations. For example, meeting milestones might substitute

in part for timely publications. Failure to have continued funding would be detrimental to young scientists just beginning their careers.

Multidisciplinary research can provide unusually good opportunities for graduate and postdoctoral training. However, successful multidisciplinary projects usually will consist of personnel with reasonable depth of expertise in their particular areas. In other words, breadth of interest cannot substitute for the competence a scientist needs to contribute his or her component of a project. Therefore, students need to be trained to be competent in the primary discipline first and co-disciplines second.

As debate about the provisions of the 1990 farm bill progressed, it became clear that interest in the research title transcended the usual constituency of federal and university researchers. Groups with broad interests in promoting the well-being of farmers, rural communities, and the environment came to see the significance of science and technology in furthering their goals. The bills language ultimately reflected their influence in the definition of sustainable agriculture (PL 95-113, 91 Stat. 981, 7USC 3101, Sec. 1404(17)):

More specific to the NRI, the 1990 farm bill also directed the Secretary of Agriculture to ensure that competitive grants awarded are, where appropriate, consistent with the development of systems of sustainable agriculture.

Since the passage of the 1990 farm bill, the NRI managers have carried on a dialogue with those who spearheaded the successful effort to introduce the goal of sustainability explicitly. Active in this dialogue has been the Sustainable Agriculture Coalition, whose membership includes the Center for Rural Affairs (CRA), among others. Speaking at the October 1993 forum, Elizabeth Bird of the CRA summarized the Sustainable Agriculture Coalition's consensus view and identified two ways for research to support the development of sus-

tainable agricultural systems: (1) focus on sustainable agricultural systems that use an interdisciplinary approach and (2) target component or reductionist research toward the concerns of those practicing or researching sustainable agricultural systems. She further explained,

In its publication Sustainable Agriculture in the National Research Initiative (Center for Rural Affairs, 1991), the CRA outlined its suggestions for ways to ensure that the NRI program emphasizes sustainable agriculture. Requests for grant proposals should seek research that relates to the goals of sustainable agriculture. As the CRA interprets these goals, NRI research should, among other things,

• expand economic opportunities in the rural United States including self-employment opportunities in family farming and rural communities;
• strengthen the family-farm system, primarily "small"-and "moderate"-sized, owner-operated farms;
• enhance the farmer's use of his or her labor and management skills; and
• facilitate the use of resources that can be generated directly on the farm or in the local community rather than somewhere else in the state, nation, or world.

The CRA argues that in implementing these goals, proposal review should explicitly address relevance to sustainability. The coalition further asserts that such relevance, and not scientific merit, should be the paramount consideration in awarding grants.

Implementation of the congressional mandate to support sustainable agriculture falls to USDA's research agencies. In July 1992, the Cooperative State Research Service and the Agricultural Research Service jointly convened a panel to develop a sustainable agriculture relevancy protocol. In 1993 USDA issued "Relevancy of AES [Agricultural Experiment Station] Research to Sus-

tainable Agriculture" (U.S. Department of Agriculture, 1993). The protocol would be applied to each project funded by the NRI to provide a measure, using a quantitative index, of its relevancy to sustainable agriculture. This protocol has yet to be finalized and has never been used by USDA in the evaluation of NRI grant applications, in large part because of the controversy surrounding the definition of "sustainable agriculture" and its connection to particular research efforts.

For those responsible for carrying out the charge, the design and use of a relevancy protocol creates concerns. Such a protocol assumes—without corroborative testing, evaluation, or necessary agreement on defining relevance to sustainable agriculture—that projects can be assessed and quantified as being consistent or inconsistent with sustainable agriculture. Panels composed of individuals representing different perspectives on sustainability are assembled to apply the protocol to individual projects, with the final rating calculated as an average of the panel members' subjective opinions. Lacking an objective basis for relating this rating to well-defined outcomes, it will not be possible to assess whether the protocol will help select projects that are consistent with the development of systems of sustainable agriculture.

Another concern is related to the appropriate use of sustainability protocol in evaluating NRI projects. The goals of the NRI are consistent with the broad purpose of establishing a sustainable food and fiber system. However, when the protocol requires that a research project have direct impact on a sustainable system, much potential for the gains resulting from research go unrecognized. Under the proposed protocol scoring system, a project without direct (presumably near-term) impact is judged as neutral with respect to the goal of sustainability. NRI managers believe many competitive grant proposals would fall into the "no direct impact" category, which has been assigned a numerical score of zero by the protocol. (Positive scores are assigned to projects judged to have direct impact; negative scores to those judged detrimental.) Even though projects may have the potential to support sustainability, the implication of the current scoring system is that they do not contribute.

The farm bill directs use of the sustainability criteria where relevant, and the board believes that the definition of relevance has not yet been settled in a way that permits straightforward use of a protocol. Recognition of the longer-term, indirect, and often powerful influence of much basic and applied science mitigates against the possibility that a conclusive, all-purpose protocol can be devised. If there were exclusive focus on sustainability of family farms and primary emphasis of its farm bill definition on the nature of on-farm operations, much research supporting better understanding of human nutrition and health, for example, would be implicitly devalued. Although the aim of the NRI should be to avoid engagement in projects that detract from the possibilities for sustainability, there are other worthy goals recognized by the Congress that enhance the well-being of the nation's people and natural resources. Skip

Stiles, of the House Committee on Science, Space, and Technology, told the October 1993 forum audience of the need to link research efforts "more closely to national needs in order to regain strong public support." So, while exclusive focus on sustainability might be inappropriate, serious consideration of the NRI's contribution to broad public policy goals would, by Stiles' reasoning, certainly be in order.