6
Conclusions and Recommendations
At the beginning of its review process, the committee considered the importance of a national research program specifically targeted to the food and agriculture sector. It asked many questions, including these: What is the unique role, if any, of publicly funded agricultural research? How critical have research and development (R&D) been for increasing and maintaining the productivity and sustainability of the nation’s agriculture and food sectors? How does the United States compare with other nations in R&D investment in those sectors, and is this investment sufficient for generating the productivity growth and agricultural knowledge that are needed to meet projected needs? Those questions and others helped to set the context for addressing elements of the committee’s Statement of Task (see Chapter 1, Box 1-1) that focused on assessing the Agriculture and Food Research Initiative (AFRI). The committee was mindful of the authorizing language in the Food, Conservation, and Energy Act of 2008 (known as the 2008 Farm Bill), which defined the goals and priorities of the AFRI program. The Agricultural Act of 2014 (known as the 2014 Farm Bill) was passed as the committee was completing its report but did not change AFRI’s authority substantively despite including some changes in AFRI activities.
The preceding chapters have concentrated on specific elements of the committee’s Statement of Task, many of which concern AFRI program functionality and effectiveness. They each outline findings that address specific questions that are included in the Statement of Task. Taken together, these questions led the committee to a broader discussion about AFRI’s importance and about what AFRI needs if it is to succeed as the major competitive grants program of the U.S. Department of Agriculture
(USDA). In keeping with the charge to evaluate AFRI, the present chapter provides overarching conclusions and recommendations that resulted from that broader discussion.
NEED FOR FOOD AND AGRICULTURE RESEARCH
U.S. public investment in food and agricultural R&D has contributed substantially, both domestically and internationally, to the public good. The 2012 Report to the President on Agricultural Preparedness and the Agriculture Research Enterprise by the President’s Council of Advisors on Science and Technology (PCAST, 2012) independently recognized the value of that investment, the importance of competitive grants to ensure the highest-quality R&D effort, and the growing mismatch between the magnitude of the investment used to fulfill the promise of contemporary scientific opportunities versus the magnitude of investment needed to meet present and projected domestic and global needs in food and agriculture. For instance, the needs of 9.6 billion people by 2050 (World Resources Institute, 2013) and the last decade’s steady decline in the U.S. relative share of global agriculture and food system R&D are in sharp contrast with the nation’s more appropriate response to opportunities in the biomedical and other basic sciences—a response that has produced substantial public-health benefit. Similarly, investment in defense-related research has led to remarkable returns, for example, in information technologies.
AFRI was created with the ambition of using the nation’s most creative minds in research, education, and extension to address issues fundamental to human and social well-being. However, continued weakness in the public commitment to food and agricultural R&D is likely to lead to a steady decline in global competitiveness of U.S. food and agriculture production and an inability to respond adequately to health, sustainability, and environmental challenges in this important sector.
CONCLUSION 1: AFRI plays a critical and unique role in the nation’s overall R&D portfolio because its mandated scope, mission, and responsibilities are focused on the most important national and international challenges facing food and agriculture. But it has not been given the adequate resources needed to meet contemporary and likely future challenges. Congress established AFRI to manage and carry out research that would address complex national and multistate issues in agriculture and food. The scope, intensity, complexity, and urgency of those issues have been increasing, and demands on AFRI exceed what can reasonably be expected given AFRI’s recent funding levels. When AFRI was launched in 2008,
the National Institute of Food and Agriculture (NIFA) made program management decisions on the basis of an assumption that appropriations would grow to authorized levels over the next several years. That assumption was not borne out, and many multiyear grants encumbered future years’ appropriations. Although AFRI funding is growing, it has still not reached authorized levels.
RECOMMENDATION 1: The United States should strengthen its public investment in competitive agricultural R&D to ensure that it continues its role of a global leader in the innovations and technologies that are needed to promote health and well-being and to feed growing worldwide populations sustainably. AFRI’s prospects for success in meeting stated goals and outcomes would improve if its funding and other support elements (such as reporting structures and monitoring abilities) were commensurate with the program’s legislatively mandated scope.
REALIGNMENT OF PROGRAM STRUCTURE TO MATCH MISSION, MANDATE, AND BUDGET
When the 2008 Farm Bill replaced the National Research Initiative (NRI) with AFRI to “make competitive grants for fundamental and applied research, extension, and education to address food and agricultural sciences” (see Appendix C), the scientific community envisioned AFRI as USDA’s opportunity to create a scientific grants agency for food and agriculture that would be equivalent in scope and stature to the National Science Foundation (NSF) and the National Institutes of Health (NIH). The 2008 Farm Bill established six priorities for AFRI: plant health and production and plant products; animal health and production and animal products; food safety, nutrition, and health; renewable energy, natural resources, and environment; agriculture systems and technology; and agriculture economics and rural communities. Those priorities formed the basis of AFRI’s Foundational Program.
In attempting to understand AFRI’s mission and structure, the committee requested a NIFA organization chart of units that were affiliated with AFRI and a diagram that showed AFRI’s program structure. After several rounds of correspondence, it remained unclear to the committee how NIFA viewed AFRI’s mission, how AFRI was structured, and who had direct reporting responsibilities for grant administration. The committee therefore assumed that AFRI’s mission was to follow the 2008 Farm Bill’s authorizing language. Later communications with NIFA provided a more explicit basis for understanding AFRI’s program structure. The committee determined
that AFRI maintains two program areas (challenge and foundational), five challenge priority areas (childhood-obesity prevention, climate change, global food security, food safety, and sustainable bioenergy), six foundation priority areas (plant health and production and plant products; animal health and production and animal products; food safety, nutrition, and health; renewable energy, natural resources, and environment; agriculture systems and technology; and agriculture economics and rural communities), and five grant types (standard project, coordinated agricultural project, planning and coordination, conference, and food and agricultural science enhancement). The committee concluded that the structure was unnecessarily complex.
The USDA competitive grants program was restructured in 2010. As part of the restructuring, NIFA established a new AFRI grant category that was intended to attract a wide array of disciplines and expertise to successfully address the most demanding, complex issues in food and agriculture. The challenge-area program was based on a multidisciplinary approach to problem solving. NIFA used the societal topic categories outlined in the National Research Council’s New Biology report (NRC, 2009) as a basis for identifying childhood-obesity prevention, climate change, global food security, food safety, and sustainable bioenergy as its challenge areas. It also established a multiyear, large-scale Coordinated Agricultural Project (CAP) grants program funded by substantial investments to address key societal concerns—an approach that USDA had previously taken with only a handful of NRI grants. This high-stakes, potentially high-rewards approach for bringing about grand solutions and the impetus for moving the approach forward were based on the assumption that funding would reach authorization levels outlined in the 2008 Farm Bill.
The goal of AFRI’s new challenge-area program is worthy—it answers previous demands for incorporating multidisciplinary approaches to complex, pressing issues, and it brings resources to bear on high-profile problems. But the size of AFRI’s budget does not allow a reasonable prospect of satisfying its congressional mandate to focus research on the six discipline areas of the 2008 Farm Bill (those areas remained the same for the 2014 Farm Bill) while adopting an ambitious grand-challenges research approach as other agencies have done, such as NSF and NIH. CAP grants have consumed an exceptionally large portion of AFRI’s annual appropriations. Meeting the multiyear commitments has reduced the funds available for smaller-scale, more traditional, investigator-initiated grants—a development that, not surprisingly, is associated with a reduction in the number of applicants for AFRI grants relative to AFRI’s predecessor (see Figure 3-3). Emphasis on CAP grants and challenge areas has coincided with a growing year-to-year inconsistency in AFRI’s project portfolio (see Appendix F), which is unsustainable in itself and insufficient if the various legislative
mandates are to be satisfied. Such inconsistency may be one explanation for the absolute decline in AFRI grant applications. The diversion of a large proportion of resources to CAP grants and challenge areas has impaired the flexibility needed to address emergent issues.
CONCLUSION 2: AFRI is unnecessarily complex, difficult to depict clearly, and characterized by overlapping components that do not clearly align with priorities identified in authorizing legislation. Program complexity impedes the measurement of progress relative to clear goals. The multiplicity of grant types, each with its own priorities that change from year to year, contributes to a sense of programmatic inconsistency and unpredictability. Proliferation of priority areas also has resulted in AFRI’s inability to satisfy its congressional mandates.
RECOMMENDATION 2: NIFA should simplify the AFRI program structure by realigning it to more clearly address its specific mission and mandates as defined in authorizing legislation. Simplification of program structure to focus on the six foundation priority areas would improve efficiency, effectiveness, and transparency.
Rebalancing the Portfolio
AFRI’s ambitious portfolio of multiple grant types is undercutting its mission to support fundamental research, which generates critical knowledge and tools for future applications. Federal support is essential to increase the storehouse of fundamental knowledge, and AFRI will need to solicit and fund applications that advance basic agricultural sciences. The 2008 Farm Bill specifies that grant funding for fundamental research should amount to 60% of the AFRI portfolio, with the remaining 40% for applied research. With a large proportion of AFRI’s budget dedicated to addressing grand challenges, the focus of the program has shifted toward applied science at the expense of fundamental research. Given its limited budget, if AFRI continues with that approach, the scientific workforce available to conduct fundamental research in the agricultural and food sciences may continue to diminish.
Conclusion 2-A: Fundamental research is critical to provide the knowledge base upon which future discoveries will be made, and expanding the stock of fundamental knowledge is AFRI’s primary purpose. The balance of fundamental and applied research, however, has shifted toward the applied, with extension and education components mainly
included as supporting elements of research grants. Projects whose principal aim is the development of fundamental innovations in research, education, and extension receive less funding. The request-for-application (RFA) topics specified for foundational grants are increasingly narrow in scope and weighted toward applied research. NIFA will need to rebalance the AFRI portfolio toward the proportions described in the 2008 Farm Bill and broaden its foundational grants areas to encourage investigator-initiated applications in basic science.
Recommendation 2-A: To realign AFRI’s portfolio with its legislative mandate, NIFA should renew its priority for fundamental research. That should include an emphasis on proposals that will generate fundamental knowledge to support novel technologies, provide platforms for extension and education, and educate the next generation of food and agricultural scientists. Basic research on topics in the six priority areas will be more effectively communicated to users and students if there is more research conducted directly on extension or educational processes, such as training on the use of new technology, and if there are additional educational programs. Less than 11% of AFRI funding is dedicated to extension and education (see Table 4-1). New grants are needed that are specific to extension and education in order to effectively communicate the research community’s findings to user communities, enabling AFRI’s fundamental and applied research to become better integrated and knowledge transfer to be more efficient in classroom and field settings.
The Challenge-Area Program
Conclusion 2-B: The current AFRI challenge areas are narrowly focused on specific issues, and the challenge and foundation priority areas are unnecessarily redundant. The challenge areas are focused on five societal challenges determined by NIFA, and the foundation priority areas follow the six outlined priorities that are authorized in the 2008 Farm Bill. The challenge areas are prescriptive and focus on specific problems of interest (such as climate change), which were predetermined at the inception of the program in 2010. For that reason, the challenge areas have been perceived by the committee and the scientific community as lacking flexibility to address newly emerging problems and to incorporate rapid advances in science and technology. That is in contrast with the foundation priority areas (such as plant health and production and plant products) that are categorized by disciplines that span food and agriculture.
Recommendation 2-B: As part of its realignment, AFRI should be simplified by eliminating the challenge-area program, and areas of research within the foundational program should be primarily investigator driven. Rather than dividing resources among two categories of programs (challenge and foundational), NIFA could focus its resources on one program (the foundational program). Redirection of resources to the foundational program, whose priority areas directly reflect priorities aligned with the 2008 Farm Bill, would enable AFRI to address more clearly the six congressionally mandated priorities. The six priority areas are broad enough to allow investigators, teams, and institutions to develop innovative projects that address current and expected needs in food and agriculture (including topics that are the focus of the challenge-area program) and to incorporate advances in science and technology in a timely manner. Such a realignment would enable AFRI to fund the most innovative investigator-driven projects and enable NIFA to take full advantage of the intellectual resources in the U.S. scientific community. Multidisciplinary approaches, championed by the current challenge-area program, are critical for successfully addressing many of the challenges in food and agriculture that the AFRI program is expected to address. Such multidisciplinary approaches, where appropriate, can and should be incorporated into the foundational program.
The Decline in Applicants, Awardees, and Trainees
Conclusion 2-C: The recent decline in the numbers of applicants, awardees, and trainees is a disturbing trend. It raises questions: Are scientists “following the money” and moving away from agricultural research? Are young scientists not being trained in agriculture? Young scientists are trained by principal investigators (PIs) who have grant funds to equip their laboratories and to mentor students and postdoctoral scholars. On the basis of the committee’s review of the number of graduate students and postdoctoral trainees supported by AFRI grants, it appears that students are increasingly being trained with funds from other federal agencies that have larger budgets. If sufficient competitive research funds are not available in agriculture for funding research and for training young scientists, researchers will seek out a larger portion of their overall support from agencies whose missions are not directly aligned with the food and agriculture sectors. In the long term, food and agriculture will lose talent to other fields of study that have stronger support.
Recommendation 2-C: AFRI should carefully examine the causes of the decline in the numbers of applicants, awardees, and trainees and adjust its grant programs to ensure that future generations of young scientists are not lost inadvertently from food and agriculture R&D because of funding policies.
Coordinated Agricultural Project Grants
Conclusion 2-D: The current AFRI appropriation cannot sustainably support the current policy of investing a disproportionate percentage of the AFRI budget on large CAP awards and simultaneously sustain a credible program of foundational, training, and Food and Agricultural Science Enhancement grants. The shift to funding fewer, higher-amount, and longer-term CAP grants also appears to have resulted in the early decreased output of scholarly products per dollar of AFRI funds invested. Adjusting for the time since project initiation, there is evidence that the large project scope and complexity of these grants have resulted in fewer scholarly products (publications, papers, and presentations) per fixed amount of funding than was the case with less complex, smaller grants. High intraproject management and transaction costs required for very large projects have likely contributed to this phenomenon. The finding applies to large AFRI grants generally but especially to CAP grants. Early output data suggest that reducing the average project’s scale and scope (represented by budget and number of PIs, respectively) would improve the output of scholarly products, at least in early phases. The committee is not saying that large grants are inappropriate, only that its early analyses show that as the scale of grants rises, the marginal output of published papers falls over the period that was examined. The committee recognizes that high transaction costs may in some projects be more than offset by the importance of the contributions in addressing the targeted problems (e.g., multi- and transdisciplinary collaboration in the broad research community).
Recommendation 2-D: AFRI should consider eliminating CAP grants as a grant category and committing more resources to other grant types. A grant’s multi-investigator structure should be driven by its underlying science. Unless the net benefits of larger, complex projects can be objectively demonstrated or AFRI’s budget is increased substantially, AFRI should consider reducing the proportion of its assets that is devoted to very large projects and instead emphasize a greater simplicity of function and PI structure. NIFA should continue to encourage multi-institution and multi-investigator grants as part of AFRI’s foundational
program and request that PIs develop budgets and project personnel that are commensurate with the proposed level of effort. Such large-scale proposals should be required to demonstrate how grant administration and transaction costs will be commensurate with the proposed effort. Because developing a multifunction, multi-institutional grant entails a large investment of time and planning, a staged development process (e.g., a planning-grant program) for large grants should be considered.
AFRI’s research, extension, and education portfolio is appropriately targeted to meeting the nation’s food and agricultural needs. However, its success depends on the generation of fundamental knowledge and the flow of new knowledge generated by other federally funded and private-sector research. AFRI can maximize its impact and resources by collaborating with other federal agencies and by strategically aligning its research with congressional mandates that target the highest-priority needs of the food and agriculture sectors.
CONCLUSION 3: AFRI does not have clearly articulated plans to guide its priority setting, management processes, and interagency collaboration. To evaluate AFRI’s success it is critical to define goals and outcomes and thus enable the assessment of progress in meeting them. NIFA provided the committee with several documents that described a roadmap explaining how the challenge areas were developed to take into consideration the societal challenges outlined in the National Research Council New Biology report (NRC, 2009) and pointed to individual RFAs for specific goals in each of the priority areas. But it did not provide a statement of overall goals, time frames for meeting them, or planned outcomes for assessing progress. For the purpose of the present review the committee assumed that the goals of AFRI were synonymous with those stated in the 2008 Farm Bill which were unchanged in the 2014 Farm Bill.
RECOMMENDATION 3: AFRI should develop a strategic plan that identifies priorities for its overall program goals for meeting them and a framework for assessing the program’s progress. Such a plan is critical for providing program continuity, consistency, and predictability. A strategic plan would include a clear vision
statement and strategies for implementing priorities. To develop a strategic plan NIFA could revisit the intent of AFRI and broadly define acceptable topics so that AFRI programs can achieve greater flexibility. The plan could include less restrictive RFAs for which PIs can propose unconventional ideas and take more flexible approaches to the six broad priority areas mandated by the 2008 and 2014 Farm Bills.
Interagency Collaboration
Conclusion 3-A: Interagency efforts directed at food and agriculture need to be more strategic, more robust, and better coordinated across federal agencies. Several other federal agencies—such as NSF, NIH, and the Department of Energy (DOE)—provide grants and conduct research in subjects tangentially related to food and agriculture, but USDA is the only federal agency whose mission is aimed directly at food and agriculture. To further USDA’s mission and to leverage the efforts of sister agencies, USDA will need to take on a greater leadership role in coordinating research efforts across agencies.
Recommendation 3-A: NIFA and USDA should lead interagency efforts to effectively coordinate and collaborate across agencies on food and agricultural research. NIFA has been successful in collaborating with NSF, NIH, DOE, the National Aeronautics and Space Administration, and other agencies to support research on subjects of mutual interest, but the increasingly complex issues that face the food and agricultural sectors require more systematic efforts to ensure that AFRI programs maintain effective collaboration among federal agencies whose research programs are related to food, agriculture, human health and nutrition, and natural-resource systems while continuing to avoid unnecessary duplication. NIFA should take a leadership role in coordinating food and agriculture research throughout the federal R&D funding portfolio and lead an interagency working group to leverage investments that will continue to advance the knowledge base on food and agriculture.
External Advisory Council
Conclusion 3-B: AFRI needs an external advisory council to validate its strategic direction and to provide valuable guidance to national program leaders (NPLs) on programmatic decisions. Unlike NIH and NSF, AFRI does not have a formal, external, and strictly scientific advisory council. Such a council would be highly valuable for the following functions of the AFRI program: to guide, advise on, review, and assess on
an ongoing basis priority setting, resource allocation, program policies, and peer-review and award-management processes. NIH and NSF each have advisory groups on which NIFA could model its AFRI Scientific Advisory Council.
Recommendation 3-B: NIFA should form an AFRI Scientific Advisory Council that consists of members who represent the food and agricultural research, education, and extension professional communities. Such a council should provide scientific advice and advisory oversight on all aspects of AFRI’s program management and strategic planning, and council members should be selected based on their qualifications to perform these functions. The council would be similar to the scientific advisory councils used by NIH and NSF to help to validate the program’s direction (e.g., priority setting for research, education, and extension) and substantial changes in program structure (see Box 6-1). In contrast with the National Agricultural Research, Extension, Education, and Economics (NAREEE) advisory board, which advises the Secretary of the U.S. Department of Agriculture on all four topics (research, extension, education, and economics), the AFRI Scientific Advisory Council would specifically be designed to advise the AFRI program. This proposed AFRI Scientific Advisory Council might be possible within existing authority and funding (e.g., as part of the NAREEE authority); however, the committee does not prescribe how NIFA should seek this scientific advice.
As mentioned in Chapter 5, the committee requested an organization chart and other information in an attempt to understand the structure of AFRI and how it was managed. The committee was unable to get complete information on those matters. On the basis of the responses provided, it appears to the committee that the AFRI structure is unnecessarily complicated and is characterized by an elusive chain of command. This complexity and lack of transparency has led to inefficient program management and operation. Given the goal of setting up the new program, developing program priorities, and balancing its portfolio to satisfy its congressional mandate, the committee expected that NIFA leadership would provide higher visibility for the program. AFRI is a program within NIFA that appears to be orphaned in that there is no clear line of leadership, strategy, and policy. However, the AFRI proposal-review and funding-decision processes that were set up during the National Research Initiative (NRI) and continue with AFRI appear to be rigorous and effective in selecting and funding high-quality science.
BOX 6-1
A Scientific Advisory Council for the Agriculture and Food Research Initiative
Each institute and center of NIH has a scientific advisory body.a Members represent professional communities and patient advocacy groups. The National Institute of General Medical Sciences (NIGMS) has a mission similar to that of AFRI: to provide support for foundational research and training of the next generation of a diverse workforce in biomedical sciences. Its Advisory Council consists of leaders in the biologic and medical sciences, education, health care, and public affairs. Members are appointed for 4-year terms and meet three times a year. The council performs a second level of peer review for research and research-training grant applications assigned to NIGMS. Council members also offer advice and recommendations on policy and program development, program implementation, evaluation, and other matters of importance to the mission and goals of NIGMS.
In NSF, each directorate and office has an external scientific advisory body.b The advisory committees “provide advice and recommendations to maintain high standards of program support for research, education, and infrastructure; to facilitate policy deliberations, program development, and management; to identify disciplinary needs and opportunities; and to promote openness to the research and education community served by NSF.” Unlike NIH’s advisory councils, NSF’s advisory committees do not have responsibility for second-level review of proposals. However, they provide advice on program management, overall program balance, and other aspects of program performance through subcommittees called “Committee of Visitors.”c NSF’s advisory committees are made up of researchers, administrators, and educators in diverse communities. In the case of the Directorate for Biological Sciences,d members constitute a cross-section of biology with representatives of many subdisciplines in the field and other relevant fields, such as informatics and bioengineering; a cross-section of institutions, including industry; a cross-section of geographic areas; and a cross-section of women and underrepresented minorities.
______________
aSee http://www.nigms.nih.gov/About/Council/Pages/default.aspx. Accessed December 23, 2013.
bSee http://www.nsf.gov/about/performance/dir_advisory.jsp. Accessed December 23, 2013.
cSee http://www.nsf.gov/about/performance/visitors.jsp. Accessed December 23, 2013.
dSee http://www.nsf.gov/bio/advisory.jsp. Accessed December 23, 2013.
CONCLUSION 4: AFRI’s complex and diffuse management structure has made it difficult to efficiently and effectively manage the program. AFRI has many stakeholders it needs to be responsive to: Congress, the administration, various producer groups and interests, numerous scientific disciplinary interests, and consumers. AFRI also needs to more explicitly track—and track for longer periods—the outcomes and contributions of the research that it funds.
RECOMMENDATION 4: To enhance program accountability and management, AFRI should have a dedicated leader who manages the program on a daily basis. Improved processes and procedures should be created for transparency, and AFRI’s NPLs should be granted greater authority and flexibility to meet stated goals.
Agriculture and Food Research Initiative Director
Conclusion 4-A: AFRI is managed collectively by many people. No single administrator is responsible for overall program management or accountable for AFRI’s performance. As a result, program goals and internal operating procedures are not clearly articulated.
Recommendation 4-A: NIFA should establish a clearer organizational structure and lines of authority for AFRI, including a designated director to lead, manage, and speak for its program, and NPLs dedicated to AFRI alone. The AFRI entity could be analogous to NIH’s National Institute of General Medical Sciences. In such a reorganization, NIFA should concentrate the workload of AFRI on an appropriate number of dedicated NPLs who interact directly with AFRI applicants and are accountable for the grants review and management process, including post-award management and assessment of overall program performance and balance. Concentrating AFRI management functions in the hands of selected NIFA staff should improve management efficiency without necessarily increasing total management effort.
Program Continuity and Transparency
Conclusion 4-B: The AFRI applicant community expressed frustration with the discontinuity of AFRI’s program offerings from one year to the next, which has impaired researchers’ ability to plan, resubmit unsuccessful proposals, and renew successful projects. For foundational programs, the committee received comments from applicants and panel managers that the highly prescriptive nature of RFAs discourages submission of innovative ideas. Paperwork was also long and burdensome for applicants. Furthermore, research priorities were often not communicated in a timely manner, resulting in unnecessarily extended lags between grant cycles. AFRI’s success will be determined in large part by how well the program attracts the best ideas from a broad community of qualified researchers in an array of disciplines.
Recommendation 4-B: NIFA should have a more consistent and predictable program portfolio and funding strategy to enable better planning by the food and agricultural research community. The predictability and continuity of the grants program are critical for the development of the research capacity for food, agriculture, and natural resources, particularly for young faculty seeking to establish effective research programs.
In addition, NIFA should consider publishing a single document that provides clear guidelines and policies for proposal preparation and award management. That would help in streamlining the RFA process and would eliminate confusion and excessive paperwork and thus not only help the applicant community but reduce the burden for AFRI program staff. As part of its plan to increase transparency, NIFA should publish a clear description of the AFRI review process, as NSF does on its merit-review Web site1 and NIH on its peer-review Web site.2 NSF’s proposal and award policies and procedures guide3 constitutes an example of the type of guide needed for AFRI.
Data Management
Data are needed to inform management decisions and improve assessments of program efficiency and effectiveness. NIFA was unable to provide the committee with data needed for addressing many aspects of the committee’s tasks. Some of the data had not been collected, and some were internally inconsistent or could not be easily interpreted or summarized. One aspect that the committee was specifically tasked to examine was diversity of people and institutions supported by AFRI. AFRI does not collect additional data that would enable a robust assessment of the diversity of program applicants or awardees. On the basis of data on awarded projects, the committee found that AFRI is awarding grants to public and private institutions and to land-grant universities and non–land-grant universities in nearly the same ratios as did the former NRI program and approximately in proportion to the number of proposals emanating from such institutions.
Conclusion 4-C: The AFRI program lacks a sufficiently robust information-management system and metrics for measuring key pro-
_______________
1See http://www.nsf.gov/about/performance/visitors.jsp. Accessed December 23, 2013.
2See http://grants.nih.gov/grants/peer_review_process.htm. Accessed December 23, 2013.
3See http://www.nsf.gov/pubs/policydocs/pappguide/nsf13001/index.jsp. Accessed December 23, 2013.
gram impacts. The Current Research Information System (CRIS)4 used by NIFA was not designed as a tool for managing competitive funds and is an inadequate aid for program-management decisions: it is difficult to navigate and manipulate for programmatic needs and not readily compatible with other systems. AFRI needs an information-management system that can provide the accurate information that is necessary for structured analyses of program activities and for analyzing and assessing project and programmatic outputs and outcomes. Conducting performance analyses will require systematic attention to medium-term and long-term outputs and, more importantly, projection of outcomes in the form of the science influenced, social and individual well-being, and products and incomes generated.
Recommendation 4-C: NIFA should use a more robust information-management system that would provide a basis for AFRI policy and strategic planning. The system should allow detailed assessment and management of the food and agricultural competitive research funding pool. Data collection would need to be comprehensive, and this would require a redesigned and expanded CRIS that would be responsive to AFRI’s needs and capable of communicating with other federal research-analysis systems. The system would apprise NIFA management and others of AFRI program and project performance, document the scientific and technological products of AFRI grantees, and respond to congressional and public requests for AFRI information. Such a database is critical for conducting post-award monitoring and enabling managers to measure the outputs and outcomes of AFRI research more effectively. Other funding agencies, such as NIH and NSF, are constantly working to improve their information-management systems, and NIFA should work with them toward a system that would be interoperable across agencies.
Post-Award Management
Conclusion 4-D: NIFA needs clearly defined metrics for measuring program outputs and outcomes that allow program managers to assess the value of AFRI-funded research. Project-output assessment affords only one perspective on the performance of AFRI. Some valuable benefits and contributions of the program cannot be captured by assessments of program outputs alone. Examples of the other benefits are such outcomes as AFRI’s role in encouraging graduate students and young
_______________
4As of the writing of this report, the committee is aware of USDA’s plans to retire CRIS and to replace it with another reporting system.
scientists to develop careers in food and agriculture, its role in advancing the quality of agriculture and food science and in increasing the knowledge base, and its contributions to the innovations that underpin economic development. Appropriate changes are needed to give NPLs the time and resources needed to provide a higher level of post-award management (including post-termination monitoring) designed to ensure that grants reach the most successful conclusions and outcomes attainable.
Recommendation 4-D: NIFA should develop the capability to regularly evaluate AFRI projects in terms of their outcomes, which would allow assessment of the economic and social impacts of the research that AFRI supports. In addition to the standard bibliometric measures, quantitative rates-of-return and qualitative outcomes assessments will need to include such information as important scientific advances, concrete economic impacts, patents, young-scientist training, and improvements in processes, products, or productive jobs. Both output analyses and outcome analyses will require NIFA to maintain post-termination relationships with its grantees after projects have ended and allow it to chart, for example, the progress of graduate students and young scientists who were supported by AFRI funds. To facilitate more comprehensive program assessment, AFRI should maintain post-termination relationships with grantees to monitor and document medium-term and longer-term outcome-related information.
Greater Authority for National Program Leaders
Conclusion 4-E: In their project-funding decisions, NPLs are tasked to ensure that a maximum number of high-priority issues are addressed and that funded projects align maximally with program goals. Yet NPLs have been unnecessarily constrained in their efforts to manage and balance the AFRI portfolio. The committee noted several ways in which NPLs were constrained in participating in funding decisions that would allow a better portfolio balance to align with AFRI’s mission and goals. For example, funding decisions are typically based solely on peer-reviewed rankings without consideration of the funding portfolio’s programmatic balance. That continues to occur despite NIFA’s policy that reviewers’ comments are advisory and not binding. Funding allocations to program areas are set before the award decision-making process, and this can limit the ability of NPLs to capitalize on innovative ideas presented in proposals and to pursue the most promising scientific opportunities.
Recommendation 4-E: NIFA should establish standard operating procedures (SOPs) that provide greater opportunity for NPLs to contribute to final project-funding decisions. Although peer-review ranking should be a principal factor in guiding the AFRI funding process, AFRI should consider portfolio and programmatic balance and take steps to achieve an appropriate balance when making final funding decisions. Such considerations would include balancing various food and agricultural issues and various scientific disciplines; the types of awards (e.g., high-risk, high-payoff projects); and the diversity of investigators, institution types, and geographical distributions. SOPs governing the process should be transparent, outline the criteria for balancing the portfolio, and include a mechanism for moving an allocation from one program area to another when overall program balance is needed. As mentioned in Chapter 5, AFRI’s large awards have taken more time to review and manage than has apparently been allotted, raising post-award administration costs above those in other agencies. The advisory council recommended above (see Box 6-1) could be used in some manner to provide independent assessments of programmatic decisions. NPLs are PhD-level scientists in good standing in their own disciplinary communities who were recruited to manage AFRI grants on the basis of their scientific credentials, and they should be trusted to exercise their professional judgment. With such new responsibilities, the portfolios of AFRI NPLs would need to be rebalanced to allow proper attention to programmatic direction and post-award scientific management. SOPs would also need to include a mechanism for training new NPLs and panel managers.
During the time the committee was conducting its review, Congress passed the 2014 Farm Bill and appropriated an increase in funding for AFRI in FY 2014. The reauthorization of the Farm Bill did not change the priorities for AFRI, reaffirming the importance of this program to sustain the nation’s preeminence in knowledge generation and technology advances in the food and agricultural sectors. However, the 2014 Farm Bill contained a provision requiring non–land-grant universities to match funds for AFRI grants. This approach is counterproductive to the goal of attracting the broadest array of the nation’s top scientific talent to research and to bringing nontraditional and novel approaches and solutions for food and agricultural challenges. In the future, NIFA should acquire data to determine the impact of this requirement on non–land-grant entities participating in the AFRI program.
NIFA and its AFRI program are essential elements of USDA and will be critical for enhancing the knowledge base needed to successfully address important issues in agriculture, food, and natural resources. The increase in FY 2014 appropriations for this flagship competitive research program is consistent with this report’s findings, conclusions, and recommendations and applauded and suggests that USDA has a window of opportunity to establish NIFA as a strong science agency with AFRI at its core and to reinforce the value and mission of AFRI to the nation’s well-being. The committee offers its recommendations in the hope that the suggested programmatic changes will enable NIFA to fulfill its mission of leading the food and agricultural sectors to a better future through research, education, and extension.
NRC (National Research Council). 2009. A New Biology for the 21st Century. Washington, DC: The National Academies Press.
PCAST (President’s Council of Advisors on Science and Technology). 2012. Report to the President on Agricultural Preparedness and the Agriculture Research Enterprise. Washington, DC: Executive Office of the President.
World Resources Institute. 2013. Creating a sustainable food future: A menu of solutions to sustainably feed more than 9 billion people by 2050. World Resources Report 2013-14: Interim Findings. Available online at http://www.wri.org/publication/creating-sustainable-food-future-interim-findings (accessed July 30, 2014).
