Science and the Greater Everglades Ecosystem Restoration: An Assessment of the Critical Ecosystem Studies Initiative (2003)

This chapter provides an overview of the administrative structure Critical Ecosystem Studies Initiative (CESI) and the processes used for managing CESI research activities. The chapter includes an evaluation of several management issues, including organizational structure, program coordination, accountability, and peer review, and it offers recommendations for improvements. Comments are also provided on the Department of the Interior's (DOI) proposed reorganization of the CESI program.

The CESI program is intended to meet the most important DOI science information needs for South Florida ecosystem restoration. The program is administered by Everglades National Park, but it is structured to provide a means for coordinating research, priorities, and budgets with the other agencies and initiatives involved in the restoration process (DOI, 2000). The superintendent of Everglades National Park serves as the CESI manager and is the DOI official charged with assuring that program funds are administered properly and that CESI projects contribute useful scientific information to inform the management and restoration of DOI lands in South Florida. The CESI manager is expected to seek counsel and advice on research priorities from the executive director of the South Florida Ecosystem Restoration Task Force (SFERTF) as a means of coordinating CESI priorities with those of other agencies.

The CESI coordinator serves as the lead scientist for the program. Program category managers, who are scientists or science managers within DOI, assist the coordinator and administer specific components of the initiative (Figure 3–1). Management of CESI program categories was an additional responsibility given to staff already employed by DOI agencies in South Florida. The CESI coordinator works closely with the program category managers to develop science objectives and priorities, solicit and review research proposals, and coordinate with

FIGURE 3–1 CESI management structure. SOURCE: Adapted from DOI, 2000.

other federal, state, and local agencies. Program advisory committees, made up of subject matter experts (from universities, DOI, and other agencies), were established for each program category to assist the program category managers in the review of research plans and proposals.

Funding for the CESI program is provided through an annual DOI budgeting process (details on CESI financial resources are provided in Chapter 4). The CESI manager submits an annual budget request after consultation with program category managers, the executive director of the SFERTF, and the Science Coordination Team (SCT). The budget consultation process is intended to provide the CESI manager with current information on restoration activities, scientific information needs, and research priorities.

After the annual CESI budget is approved by Congress, the CESI manager evaluates the program objectives in light of funding availability. Priority objectives for individual CESI program categories are developed in consultation with the program category managers, the program advisory committees, the SCT, and the executive director of the SFERTF (Figure 3–2). The CESI manager then determines final allocations for each of the program categories with advice from the executive director of the SFERTF, in order to coordinate the program funding with science budgets of other agencies (SFERTF, 2002). Based on these determined priorities and available funding, managers develop or revise long-term

program implementation plans that identify high-priority research needs for each program category (DOI, 2000).

There are opportunities for CESI management activities to be reviewed at several levels. The CESI coordinator, DOI science managers, the CESI manager, and the SFERTF executive director review the research priorities developed by the program category managers. Program category managers are responsible for periodically convening workshops and symposia to review category objectives and accomplishments. Program advisory committees are available to review research plans, identify unmet needs, review project proposals, and recommend funding priorities. The Science Coordination Team also meets annually to review the CESI program's priorities and direction.

Program category managers develop requests for proposals for the highest-priority objectives in each program category, and the CESI coordinator notifies potential researchers of the program's interest in receiving proposals on particular topics. Notification goes to a limited set of agencies involved in South Florida ecosystem research and to several universities in the region. The agencies are primarily the U.S. Geological Survey (USGS), the Fish and Wildlife Service (FWS), and the National Park Service (NPS); the universities are primarily Florida International University and nine institutions (most located in Florida) that are designated by DOI as Cooperative Ecosystem Studies Units.¹ Researchers in other locations who have extensive experience in South Florida are also notified.

Proposals are received, reviewed, and recommended for funding by the CESI program category managers with assistance from the program advisory committees. The usual period between solicitation of CESI proposals and decisions on funding has been about 30 days (William Perry, NPS, personal communication, 2002), a time period much shorter than that encountered in other grant programs of similar magnitude. The proposals are evaluated based on their importance in supporting the South Florida ecosystem restoration effort and the likelihood that the proposed research can be executed as specified. Both ongoing multiyear projects and new proposals must receive positive technical reviews before they are submitted to the CESI manager in an annual research plan.

Recommendations for the CESI program to support research proposals are submitted through the CESI coordinator to the CESI manager for funding approval. Documentation of committee reviews and the relationship between the proposed project and restoration objectives must be submitted with the proposal. Then, the CESI coordinator develops a cooperative agreement or other instrument for allocating the funds for a project and submits it to the director of the South Florida Natural Resource Center (SFNRC) for consideration. If the director of SFNRC approves the instrument and if it meets the requirements of the Everglades National Park administrative offices (e.g., contracting office), it is

sent to the CESI manager for final approval. The results produced by funded research feed back into the annual evaluation of program objectives and data information needs, as well as into the development of revised research implementation plans, thus beginning a new cycle in the research implementation process (Figure 3–2).

From its inception, the CESI program supported several projects that were funded outside of the competitive-proposal process. These programmatic projects (e.g., tribal water quality and development of an invasive-species control strategy) are managed out of the executive director's office and receive no direct oversight by the CESI coordinator or other science managers. Although they are funded by the CESI program and are important to the restoration effort, these projects are not included in the annual research plan. The funding of such programmatic projects has led some people to criticize the CESI program for allocating funds without appropriate controls on expenditures and the delivery of products.

The research being produced by the CESI program is making important contributions to the restoration knowledge base and is helping provide the science needed to inform restoration planning and management decisions. Nevertheless, there are some problems with the mechanics of the CESI program that need to be addressed to improve the quality and effectiveness of the supported research. These issues are described below, along with recommendations for improvement.

The CESI program has a fairly efficient process for program management and administration. The formal program advisory committees and outside consultation on the CESI program's plans (by the SCT, DOI managers, and the executive director of the SFERTF) facilitate essential coordination with other research and monitoring efforts. Although these advisory committees were established to ensure program quality, not all program categories receive the same scrutiny. For example, on one hand the Coastal and Estuary Systems category has an active advisory committee with wide agency participation that evaluates CESI proposals and hosts external symposia for determining research needs. On the other hand, program categories that primarily support internal agency efforts (e.g., the Ecological Modeling and Landscape Patterns program categories) tend not to fully utilize their program advisory committees for advice or review. All program category managers should be required to work closely with formal program advisory committees in order to enable additional input from outside experts and to promote closer linkages with other South Florida ecosystem monitoring and research activities.

Since the Comprehensive Everglades Restoration Plan (CERP) was authorized in 2000, the administrative landscape for South Florida ecosystem science

coordination has changed. Currently, the CESI program depends on consultation with the SCT, the executive director of the SFERTF, and other DOI science managers to coordinate CESI science with the science initiatives of other agencies. In the CERP, RECOVER² was established “to organize and apply scientific and technical information” to support CERP decision-making (SFWMD, 2002b) (see Appendix D). Clearly, RECOVER will play a major leadership role in future coordination of restoration science. However, many details about RECOVER are still in development, and the science needs of non-CERP projects (nearly half of the restoration effort) do not fall under its purview. RECOVER does not replace the contributions of the CESI program, but CESI managers must coordinate closely with the RECOVER teams so that limited science resources are used wisely. Effective coordination with RECOVER will enable CESI science priorities to be developed in an appropriate and informed context, while addressing DOI's particular restoration science needs.

The South Florida Natural Resource Center (SFNRC), located in Everglades National Park (ENP), currently has the primary responsibility for managing the CESI program and also provides research relevant to NPS interests in South Florida. However, there are questions about the center's ability to objectively serve the needs of all NPS lands in South Florida because of its administrative linkage to ENP. At one panel meeting, managers of Big Cypress and Biscayne national parks and Loxahatchee National Wildlife Refuge expressed eagerness for more CESI research to be conducted on DOI lands. The land managers interviewed, however, acknowledged a number of hurdles faced in obtaining CESI funds to conduct research outside of ENP, including lack of research staff and poor communication of CESI proposal requests. Scientific research is broadly recognized as a critical component in the development of effective natural resource management strategies (NRC, 1992; NPS, 1992). The long-term nature of the restoration process and the need to continuously adapt management techniques to environmental change make DOI-specific research and environmental assessment all the more critical.

The NPS could address these concerns and improve the effectiveness of the CESI program to protect DOI's numerous land-management interests by removing the SFNRC from the organizational and supervisory structure of Everglades National Park. The current SFNRC research director should work cooperatively with the park superintendents in South Florida but should have organizational and fiscal autonomy. This suggested reorganization is consistent with the recommendation by the National Research Council in its report Science and the National Parks (NRC, 1992) that “the National Park Service should revise its organizational structure to elevate and give substantial organizational and budgetary autonomy to the science program.” Strikingly similar recommendations were made in the Report by the Advisory Committee to the National Park Service on Research (NRC, 1963). Given the enormously complex political, ecological, and land-management patterns in South Florida, those previous NRC recommendations may be particularly important to this arm of NPS research.

Additional improvements in CESI management are needed to address concerns over accountability and public awareness of the CESI program 's contributions. Approximately 70 percent of CESI funds are obligated to federal or state agencies through interagency agreements. The USGS, the SFERTF, and the FWS are the largest recipients of CESI funds through interagency agreements. The remaining funds are allocated to universities through noncompetitive cooperative agreements (William Perry, NPS, personal communication, 2002). There are well-established accountability procedures in place through the Everglades National Park contracting office for the cooperative agreements managed by the NPS. However, the CESI manager does little monitoring of work funded by interagency agreements because these instruments generally transfer large sums of money in very broad categories and leave the responsibility of allocating and reviewing expenditures to the receiving agency. When funds are allocated through an interagency agreement, the CESI manager has little power to assure that the work is accomplished on schedule or that the funding is focused on the identified priorities. Interagency agreements are relatively weak documents that for the most part rely on the good will of cooperating agencies. Therefore, consideration should be given to providing the CESI manager with more direct responsibility for assuring that funds allocated by interagency agreement are used to address the identified research priorities in a timely manner.

Although the CESI program has provided valuable information in support of the South Florida ecosystem restoration, its contributions have not been documented adequately to Congress and the general public. Interagency funding agreements may, in part, lead to reduced awareness of the CESI's contributions. University scientists receiving CESI funding through agencies other than the NPS have sometimes been unaware of the original source of the funds because the CESI program was not appropriately acknowledged. Also, a large portion of CESI research dollars has gone to support existing programs within the USGS, resulting in little public recognition of the CESI program's contribution. This may lead to the perception that the CESI program has been used to replace base funding for the USGS and that few new products and programs have been developed through the program. CESI funds are provided to the executive director of the SFERTF, who has public relations expertise on staff. It would seem appropriate that this office would promote the importance of the CESI program to the restoration effort.

Science depends on peer review for quality assurance and credibility. Effective peer review is the hallmark of science in service to the public good, especially for highly visible programs such as the CESI. Peer review is often thought of as solely a judgmental process, but in science, review provides essential guidance to investigators. Peer review ensures that researchers observe two of the most basic tenets of science: that the process is philosophically correct and that it

is free of practical bias. Early peer review identifies potential problems with the conduct of the work and improves the process of reaching defensible conclusions. At the conclusion of the research, peer review assesses the quality of the work and the viability of results. An “ideal” peer review system is outlined in Box 3–1. In practice, peer review processes are always imperfect, and the CESI program faces the challenge of appropriately reviewing scientific results under notable time pressures. However, such paradigms are useful if they are seen as goals for which to strive. In that spirit, the following discussion focuses on ways that the CESI program could improve the reliability and objectivity of its peerreview process and thereby strengthen its scientific research program.

In the CESI process, formal requests for proposals are not widely dispersed to researchers but are instead generally limited to the South Florida ecosystem science community. This regional proposal-distribution process restricts the potential quality and breadth of CESI researchers. CESI managers have expressed concern that they have nearly exhausted the existing pool of South Florida researchers. Broadening the distribution of proposals would also help bring new talent and energy to the region 's science questions.

The CESI program must move quickly to address emerging science needs and to meet restoration decision-making deadlines. However, at times this fast action happens at the expense of appropriate proposal development and review. The usual period between requests for CESI proposals and funding decisions has generally been short, but some proposal solicitations have occurred only days or weeks before funding, with skeletal proposals submitted and approved under the time constraints. Once approved, projects are funded on a year-to-year basis, with likely renewals because most projects are initially funded to extend over several years. To maintain public credibility and ensure wise investments of CESI funding, the CESI program must adhere to the highest standards for proposal review (see Box 3–1). CESI proposal funding should be based on prior evidence of successful research, timely conduct, and a strong publication record.

The CESI program does not have a well-established and published process for proposal or research reviews. CESI program advisory committees evaluate the funding proposals, apparently with input from a limited number of reviewers recruited by the program category managers. Although the CESI reviewer-selection processes are not completely clear, there seems to be limited independence between the reviewers and the reviewed. The same agencies are often involved in both aspects of the process with a very limited and closed system of scientists. The identity of reviewers is typically held confidential for specific proposal reviews, but as a whole the CESI program also does not publish the names of reviewers, whose qualifications could be used to build public confidence in the outcome of the process. The clients for reviews include researchers and managers, but in the CESI program, there does not seem to be a standard reporting process that encourages extensive feedback between reviewers and authors.

The most serious shortcoming in the existing CESI peer-review process is that managers generally assume that the results and conclusions resulting from the approved research are of acceptable quality, and they pass them directly to

users in the planning and management communities without detailedtechnical review (William Perry, NPS, personal communication, 2002). CESI program managers hope that researchers will eventually publish their results in the refereed literature, and they assume that publication will occur because most researchers operate in institutions (e.g., universities and the USGS) that reward the production of refereed publications. The result, however, is that “gray literature” (unrefereed reports) is likely to surface in public and in the hands of managers and decision makers without review. A typical research project includes con-

cepts, methods, analysis, and the drawing of conclusions. Even though a project may have been reviewed several times over the course of study, the all-important analysis of data and interpretation of the results from which conclusions are generated cannot be reviewed until the project is finished. Different experts can reasonably differ on the interpretation of results, so that thoughtful review is essential to ensure that the conclusions transmitted from researchers to decision makers are sound, reasonable, and scientifically defensible. An independent peer review upon the conclusion of the research process in CESI investigations will assure restoration planners and decision makers that they can work confidently from reliable research.

In recognition of the need for timely availability of research findings and the delays associated with journal publications, the CESI program should develop a mechanism for fast-track independent review of the most critical research findings. Research conclusions in the form of a draft manuscript or “white paper” could be evaluated by this mechanism if there is insufficient time for a peerreviewed article to be produced.

CESI managers can substantially improve the scientific viability of their research products by instituting a formalized peer-review process that includes a broadly based request-for-proposal process, improved proposal-review standards, widely solicited independent reviewers, a periodic collective identification of the reviewers, and reviews of science findings before they are released to users. The appointment of a peer-review monitor, perhaps in the role of a senior scientist, would streamline the peer-review process while at the same time enriching the process and increasing its accountability.

The manager of a peer-review process must evaluate the parameters necessary for an appropriate review, including the diversity of expertise and independence of the reviewers, and the complexity, cost, and duration of the review. Each of these parameters is a function of the combined influences of the project magnitude and risks associated with decisions based on the outcome of the project (NRC, 2002c). The greater Everglades ecosystem restoration is one of the largest, most expensive projects of its kind in the world. It is also a project with substantial risk, because the outcomes are largely unknown and untested. For these reasons, the science supporting the restoration requires peer review that adheres to the highest standards and that demonstrates independence of reviewers with great diversity of expertise.

In early 2002, a memorandum of understanding (MOU) among DOI bureaus was developed to more effectively coordinate research activities in South Florida (see Appendix E). The purpose of the MOU is to coordinate DOI research, monitoring, and planning efforts in support of the South Florida ecosystem restoration. The MOU was intended to maximize the value of DOI funds and ensure that research products produced both are high-quality and are responsive to the land-management needs of the NPS and FWS. Both the CESI program funds and the

USGS place-based studies funds (see Chapter 4) would fall under the management guidelines of the MOU, creating a new structure to manage the majority of DOI's investments in South Florida ecosystem science, likely beginning in 2003.

The current MOU implementation plan proposes to significantly revise the existing CESI management structure (see Appendix E). The MOU implementation plan calls for the creation of a Science Coordination Council (SCC), and discussions are underway informally to establish a Science Steering Committee, to replace the CESI manager, program category managers, and associated program advisory committees. Although these plans are currently under development, at this point the proposed management teams seem to lack an appropriate composition of scientists and agency representatives. There are few scientists on the proposed SCC, as these members consist of senior managers from the NPS, USGS, FWS, SFERTF, U.S. Environmental Protection Agency, U.S. Army Corps of Engineers, South Florida Water Management District, and Native American tribes. The responsibilities defined for the SCC and any associated steering committees will require the most knowledgeable scientists and science managers from a wide representation of South Florida science agencies, who can identify the priority science needs, select qualified researchers, and negotiate funding. The implementation plan for the MOU, as currently proposed, has the potential to push science further into the background rather than into a direct advisory role on restoration activities.

Lessons can be learned from early CESI management problems that arose under a similar team-based approach. Previously, the Science Coordination Team was used to set CESI's research priorities, but concerns arose as to whether some members might be prioritizing increased funding for their own agencies above actual science needs. In response to these concerns, responsibility for setting CESI priorities was given to the CESI manager. The Science Coordination Council (SCC) will need to take great care to prevent similar issues from arising. A senior scientist serving in the role of the CESI coordinator but working closely with the SCC and any associated steering committees could help retain the appropriate focus on addressing the critical science gaps. The senior scientist ideally should not be affiliated with any one particular agency but would offer respected leadership among all South Florida ecosystem science entities. A similar leadership structure, for example, was instituted to manage research at the Grand Canyon (see Chapter 5, Box 5–3). The research management proposed in the MOU will also need to be supported by designated staff located in South Florida to synthesize and communicate the research findings. If care is taken to address these important organizational issues, the MOU could improve the coordination of research in South Florida.

DOI has considered managing CESI funding through the USGS rather than the NPS. The USGS is a strong science agency, supporting extensive basic and applied research in South Florida. Because the USGS has no management or regulatory mandate, the agency generally is perceived to be neutral on resource management issues and therefore more credible with respect to science. Nevertheless, this impartiality can sometimes result in alternate prioritizations of science needs and slow delivery of research products, which are needed within narrow time frames to resolve planning questions. DOI interests in the South

Florida ecosystem restoration are tied to its stewardship of federal resources and its consultation and concurrence responsibilities for the CERP described in the draft programmatic regulations (USAGE, 2002b). The CESI program must produce the best possible science that is responsive to DOI needs and external restoration planning deadlines and therefore demands appropriate involvement of all relevant DOI agencies in future program managment. DOI managers should carefully consider these concerns when weighing future administrative reorganizations. Any effort to remove the administration of the program from those with the most vested interests in the CESI program's results (resource managers and scientists in South Florida) is likely to create as many problems as it solves.

In summary, this review concluded that the CESI program has developed an efficient process for program management, but several changes are needed to improve the quality and effectiveness of the science that the CESI program supports. Two high-priority management improvements have been identified that can be made quickly and inexpensively. First, the CESI program should adhere to and substantially improve its standards for proposal review by establishing a wider distribution of requests for proposals, an independent proposal-review process, and funding criteria based on prior evidence of timely conduct of research and publication of the results. Second, the CESI program must broaden the involvement of expert advisors in the priority-setting and proposal-review processes by fully utilizing its program advisory committees and coordinating closely with the SCT and RECOVER. Additional CESI management changes are needed in order to develop an effective peer-review system for CESI research results, improve the accountability of funding allocated through interagency agreements, increase the public awareness of CESI contributions, and more effectively address DOI restoration science needs outside of Everglades National Park. As DOI refines its new interagency management plan for CESI funds, care should be taken to assure that the leadership involves strong scientific expertise and appropriate agency representation.