The Oil Spill Recovery Institute: Past, Present, and Future Directions (2003)

The Oil Spill Recovery Institute (OSRI) provides its funds to outside investigators primarily through an open, competitive process (with some exceptions, as discussed in Chapter 6). OSRI’s grant policies are outlined in its Oil Spill Recovery Institute Grant Policy Manual. The current edition is dated January 2002 and reflects revisions as the OSRI program has evolved. The Oil Spill Recovery Institute modeled its basic grant policies and procedures on those used by the National Science Foundation (NSF) and the National Oceanic and Atmospheric Administration’s (NOAA) Undersea Research Program. Proposals are solicited primarily through Broad Area Announcements (BAAs), although Requests for Proposals (RFPs) are sometimes used.¹ The content of the BAAs varies each year based on recommendations by the OSRI Advisory Board. According to the Grant Policy Manual, “cooperative” proposals between researchers

and users are encouraged with cost sharing being used as one metric of user interest. In addition, multidisciplinary and multi-institutional projects are encouraged. Funding may be in the form of grants, contracts, or cooperative agreements. Awards vary by size and level of approval required: the January 2002 edition of the Grant Policy Manual gives the director authority to approve small awards (less than $25,000 per year), but additional approval by the Scientific and Technical Committee (STC) is now required for medium-size awards ($25,000 up to $100,000 per year), and by the OSRI Advisory Board for large awards (e.g., $100,000 per year). Additional general information on policies can be found in the Grant Policy Manual (OSRI, 1998, 2002b).

According to the OSRI Grant Policy Manual, BAAs are to be “issued annually by the OSRI to solicit proposals” within the three program areas. The manual does not provide specific timelines for BAAs or other policies related to the circulation of BAAs. BAAs are generally advertised in the Commerce Business Daily, various newspapers, local television, radio, and sometimes announcements at regional, national, and international conferences or through e-mail networks such as LABNET, EVOS Trustee Council mail lists, and more. Three sample BAAs are included in Boxes 3-1, 3-2, and 3-3, and additional examples are included in Appendix F.

The committee has concerns about how the solicitation process is used. First, there is no pattern or regularity to these announcements, and the amount of time allowed for response varies from announcement to announcement. This lack of predictability or regularity in advertisements and deadlines diminishes the likelihood that high-quality researchers not currently affiliated with or funded by OSRI will see opportunities to participate. A nationally competitive program will be achieved only by encouraging broad participation from leading figures. OSRI needs to implement improvements in the writing and advertising of its solicitations for proposals. All solicitations should be open for more reasonable amounts of time (e.g., three months) following broad advertising to allow for maximal response by potential proposers.

Solicitations for research proposals (whether BAAs or RFPs) have not always been publicly announced long enough for adequate responses to be generated. One example is the BAA seeking proposals related to Biological Monitoring of Herring and Pollock in Prince William Sound: it was posted on December 15, 1999 and had a closing deadline of January 7, 2000 (Box 3-1). In addition to the brief opening, this solicitation included

an extremely detailed schedule and was awarded to a former colleague of the director, which added to perception problems in the science community. One announcement seeking a workshop facilitator was posted on March 29, 2002, with an original deadline of April 19, 2002 (although this was subsequently extended to June 3, 2002), showing that the practice was still occurring as of June 2002.

OSRI generally discourages unsolicited proposals, but the Grant Policy Manual outlines procedures for dealing with them. Unsolicited proposals falling “within the scope of OSRI’s mission” may be evaluated through the normal review process. Unsolicited proposals that are determined to fall outside the immediate scope of open BAAs, but are of interest to OSRI, may lead to the issuance of a related BAA. The unsolicited proposal would then be reviewed along with any others generated through the BAA process. This policy may discourage novel science or technologies because any novel areas identified by potential researchers would then be made public through the formal review process that followed. Some unique opportunities may be lost by not having a mechanism to deal with unsolicited proposals. One approach would be to have a periodic open BAA (e.g., once every two years) that specifically solicits novel studies related to the OSRI mission.

For solicited proposals, the nature of the solicitations is intended to be steered by the annual work plans, with approval from the Advisory Board. In its review, the committee was surprised by the great variability in the

depth of information provided in the BAAs examined. Box 3-2 is a BAA that solicits proposals for modeling of mesoscale atmospheric conditions in Prince William Sound, and the text describing what is clearly a complex need is extraordinarily brief. The BAA for atmospheric modeling of Prince William Sound called for a specific application for a particular need OSRI had identified. Thus, it would be more appropriately solicited with an RFP stating the requirements. Also, the total text of the second paragraph is quite short. In the first example (see Box 3-1) the text describing what was needed for biological monitoring of herring and pollock in Prince William Sound is extensive and detailed. In fact, the committee was surprised that the biological monitoring solicitation was considered a BAA, given its proscriptive nature. A BAA for such a project, if designed to bring forward a variety of thinking, might have been phrased to request: “Provide a survey of important biomass components of Prince William Sound in a manner compatible with their use in OSRI-funded modeling efforts.” The review process would then select the best ideas.

Box 3-3 provides another example. This BAA is extremely brief and vague, particularly for a program with a $200,000 budget. There are only two sentences and a title indicating the subject area to potential researchers. The title and text imply that the BAA is requesting simulations of the spatial-temporal distribution and impacts of dispersed and nondispersed oil spills. The BAA also states that both physical and biological environmental impact modeling are to be demonstrated as a capability. This suggests that the models should have already been developed and that a simulation study is requested.

There is no indication in the Box 3-3 BAA that the funding was intended as a purchase of an oil spill model for OSRI that might be used to run simulations in PWS as part of the Nowcast/Forecast system, which is ultimately what was done. The project selected was to apply and implement an existing oil fates model (OSCAR) into the Nowcast/Forecast system. The model system does not have a biological impact model component. Thus, OSRI’s selection process for grants under this program was not advertised in the solicitation. In addition, the scope changed after advertising the BAA because the simulations of the spatial-temporal distribution and impacts of dispersed and nondispersed oil spills are now described as a different project called “Dispersion Impact Analysis.” The intended scope of this 2002 project (budget $150,000) is to use the OSCAR

model to conduct a range of dispersion scenarios to assist in future dispersant use planning and priority setting in real spill situations. The dispersion impact analysis project has not been advertised as a BAA and OSRI plans to run this activity in-house, using its staff.

Offering extremely vague solicitations opens the door to accusations that the winning proposals could only be submitted by people with previous knowledge of specifically what was desired. Also, the funding announced as available for many of the projects is so low that only someone already in the area could reasonably do the work. OSRI must write BAAs and RFPs that are consistent with its mission and long range plan and not simply write solicitations designed to garner known responses.

The committee deliberated on the appropriateness of using BAAs as the primary funding vehicle for soliciting research proposals, in contrast to other vehicles such as an RFP process. BAAs are appropriate for a number of the activities conducted by the OSRI (e.g., many of the educational activities or when novel technologies are being solicited), but projects with highly specific outcomes—that is, solicitations seeking a specific model—would be better solicited through an RFP process.

This discussion strikes at what is, perhaps, a fundamental question about OSRI, and confusion about the answer is at least in part what causes outside scientists to have some skepticism about the fairness of the organization’s procedures. Is OSRI a granting institution that allows the scientific community to drive the direction and mixture of projects (through the proposal process) under the general guidance of the mission, staff, and Advisory Board, as is stipulated in the authorizing legislation, or is OSRI management responsible for developing a directed science plan that they then implement through directed procurement of specific projects and products? OSRI is now operating in this latter mode. Both approaches have advantages and disadvantages, and neither is “more” right than the other. But it is likely that many in the science community, as well as other stakeholders, believe that the first approach is what was intended, leading to a perception of unfairness in how the program is currently administered.

If the top-down approach is a deliberate choice, OSRI needs to articulate this clearly and honestly, to avoid misunderstandings and disenchantment by proposers. And OSRI should be aware that this approach poses the risk of leading to a program that, overall, appeals primarily to the stakeholders in charge. It may be that this dilemma is behind the difference of opinion (explored in depth in Chapter 7) about the appropriate scope and focus of OSRI’s modeling efforts: this component is a highlight to many OSRI decision makers, but this committee believes the current emphasis on real-time spill response is inappropriate and perhaps duplicative of other efforts in the area.

These issues need to be addressed by the OSRI Advisory Board. It should consider whether the “top-down” approach is in the spirit of the legislation, whether it is effective and fair to appear to manage a broad solicitation process when most degrees of freedom are proscribed, whether there is an adequate mechanism in place to set priorities and vet ideas, or whether the program should be under the strong direction of management.

According to OSRI, peer reviewers are selected by staff and reviewed/ approved by the OSRI director. The staff focuses on academic institutions to provide peer reviews but also uses board and STC members, industry, agencies, and others to select peer reviewers with the proper expertise to review specific proposals. The February 2002 changes to the Grant Policy Manual include a provision that the STC members review all proposals recommended for award for $25,000 or over; therefore, those proposals will receive additional peer reviews, beyond the three required. People who submit proposals under a given BAA are not used to review other proposals under that BAA, and all reviewers are required to declare potential conflicts of interest and are not used if there is a conflict. When asked if there were requirements for geographical, expertise, or other criteria for the reviewer pool for an individual proposal, OSRI staff noted that reviewers are sought who possess relevant and appropriate knowledge of the subject, and that might include geographic or other specific expertise. A fairly even mixture of academia and private sector reviewers has been used. OSRI prefers to use the national and international academic arena to select peer reviewers for science proposals when appropriate.

The committee received a number of external comments, many negative, related to how proposals have been handled in the past. According to the current Grant Policy Manual, all proposals are reviewed “by at least three individual peer reviewers through a mail review and/or a panel review.” In the past, proposals often had only two reviews with a third requested if “deemed necessary.” This increase in number of reviewers is one of several examples of changes that have been made by the Advisory Board to improve the proposal process.

Until now all proposal reviews (with the exception of a few of the smaller education proposal reviews) have been accomplished through mail rather than panel review, although policy allows for either option. In the past, according to the OSRI director, staff formed small panels to make final funding decisions on all small and medium grants. Now, the Scientific and Technical Committee is used to decide funding on all medium size grants. The committee believes that recent changes in OSRI proce-

dures will improve the public perception of fairness in the process, and we commend OSRI and the Advisory Board for their continued efforts to improve the process. OSRI might consider adapting additional protocols of the National Science Foundation in selection of reviewers. For example, some reviewers could be selected from names provided by applicants. As with NSF, potential applicants could have the option of requesting that one or two people not be used as reviewers. Furthermore, applicants could be requested to submit information identifying relevant close colleagues such as previous graduate studies advisors, former graduate students, and co-principal investigators on other recent projects, so that these people are not inadvertently selected as reviewers.

The broad categories assessed by reviewers include need, benefits, implementation, financial matching, experience, milestones, repayments, and risk sharing. Once reviews have been completed, proposals are grouped according to their rankings: those receiving “fund” rankings only are placed at the top of the list, those receiving any “fund after minor revisions” are ranked in the middle of the list, and those receiving “do not fund” rankings are placed at the bottom of the list. Within each of those categories, decisions to fund are based on the ranking in the list, total dollars available, and the matching funds included in the project budget.

Although there have been problems (some real and some perceived) with the proposal process used by OSRI, most of these have been addressed by recent policy changes in the way grants are processed. If present policies for peer and panel review (as well as management of conflict of interest) adapted from those used by other granting institutions such as the National Science Foundation, Office of Naval Research, Minerals Management Service, and National Oceanographic Partnership Program are followed carefully, the process should be fair.

While all proposals are required to go through the above review process, final approval for funding differs depending on the funding level. Small projects can be approved directly by the OSRI director, medium-size projects by the Scientific and Technical Committee (following recommendation by the director), and large projects by the OSRI Advisory Board (following recommendation by the director, and the Scientific and Technical Committee). When there is a conflict between the director and the Scientific and Technical Committee regarding funding recommendations for medium-size proposals, the final decision is made by the OSRI Advisory Board. The policy does not spell out the level of participation by the Scientific and Technical Committee. Our committee would urge the STC to take an active role in evaluating the proposals and reviews forwarded to them.

Special conditions apply to proposals submitted by personnel in the PWSSC. The most significant of these is that “all proposals ranked higher

than a proposal submitted by the Prince William Sound Science Center (PWSSC) shall be offered an award prior to the Center’s proposal(s) being approved for funding support.” Should a situation occur where a PWSSC proposal is tied in ranking with a proposal from an outside entity, the proposal from the outside entity would be preferentially funded. To date, this situation apparently has not occurred. This preference to non-PWSSC proposals is a good policy, given that there have been perceptions of favoritism in the center.

Responses to OSRI BAAs have ranged from 0 to 14 proposals (Table 3-1). Although the dataset is not large, there appears to be a downward trend in the number of proposals submitted for each BAA over time. For example, in FY98, 7 of 11 BAAs had more than 5 responses each, but by FY01 the greatest number of responses to any single BAA was 3. In FY00 only 3 of 9 BAAs had responses greater than 5 per BAA. Proposals actually funded for each BAA vary from 0 to 5. One item in the information provided to the committee appears incorrect: in FY01, 3 grants were made where only 1 proposal was received. All of these grants were for educational activities and were “small” grants as defined by the OSRI’s guidelines.

As mentioned earlier, the OSRI Advisory Board mandated that funds be spent in an approximate split of 40 percent for applied technology, 40 percent for predictive ecology, and 20 percent for education and outreach. These targets were more or less met in the four years evaluated (Table 3-2).

The committee was also interested in how OSRI distributed funds relative to the size of the grant awarded. Projects were broken down into small (less than $25,000), medium size ($25,000-$99,999), and large ($100,000 and greater) grants (Table 3-3). Grants were awarded across all categories with medium-size and large proposals comprising about an equal split in funding for a given year. While current policy dictates approval of medium-size proposals by the Scientific and Technical Committee and large proposals by the Advisory Board, it is our understanding that this policy was not in place for our evaluation period (FY98-01). The distribution of funds allowed the OSRI to fund from 23-25 projects each year from FY99 to FY01. Even with the current approval process in place, the director could spend a substantial portion of the budget by approving many small grants. The board may want to consider a maximum limit per year that can be spent on small projects. However, our committee believes

that the distribution of funds among small, medium-size and large grants was appropriate over the period evaluated (FY98-01).

The OSRI proposal review process has evolved over time. Examples of changes in policy are the increasing the number of reviewers for each proposal and restricting the sole approval of projects by the director to those less than $25,000. The committee received a number of outside comments on the OSRI program, some of which expressed concern about what they believed to be actual or perceived fairness issues in the OSRI grant process. Some outside scientists perceived that OSRI relies too heavily on PWSSC staff for proposal review, that advertising of BAAs is not timely and some people are given the “inside track” before formal announcements are made, that deadlines set in BAAs are sometimes inappropriately short, and other issues. Some of these concerns have been addressed by official policy changes and some (whether real or perceived) could be addressed by formalizing the advertising of BAAs and RFPs and making advertisement of both more widespread.

The historical relationship between the Prince William Sound Science Center (PWSSC) and the Oil Spill Recovery Institute (OSRI) is expressed in the document “A Brief Overview of Prince William Sound Oil Spill Recovery Institute (OSRI) and Science Center (PWSSC),” which states,

OSRI is administered by the PWSSC and shares administrative support with the PWSSC, perhaps most notably a director. By law, however, the OSRI has a separate mission to “identify and develop methods to deal with oil spills in the Arctic and subarctic environment and work to better understand the long-range effects of oil spills on the natural resources of Prince William Sound and its adjacent waters….” OSRI has come under

some criticism relative to its close links with the PWSSC. Specifically, concern has been raised whether PWSSC researchers have received a dispro-portionate amount of funding from the OSRI. A breakdown of support to PWSSC researchers from the OSRI program is summarized in Table 3-2. Data used to generate Table 3-2 were obtained from the tables “OSRI Contracts Awarded 1997-Present” provided by the OSRI.

In examining Table 3-4, about 30 percent of the total dollars awarded by the OSRI have been granted to PWSSC projects each year since FY98. The most significant support was to projects categorized under predictive ecology. OSRI records indicate that 70 percent of the funds spent in predictive ecology in FY00 and 66 percent in FY01 were awarded to PWSSC projects. In contrast, with the exception of FY99, little support has been given to PWSSC projects in the area of applied technology. Support of PWSSC projects in the area of education and outreach has steadily decreased (as a percent of total support) since FY98.

Another way to look at the fiscal relationship with respect to awards between the OSRI and the PWSSC is to look at the number of proposals awarded to PWSSC projects relative to the total funded (Tables 3-4 and 3-5). From 28 percent to 40 percent of the total projects funded were awarded to PWSSC projects. This result (about 30 percent to PWSSC each year) is in line with the total dollar amounts awarded (Table 3-3) and is not unreasonable.

The committee evaluated a list of publications provided by OSRI (Appendix H). The publication list includes publications by principal investigators sponsored by OSRI and various workshop proceedings and other documents. The list includes 44 total publications described as published or “in press.” Of these 44 publications, 15 are journal articles, 15 are conference proceedings, 6 are abstracts, 4 are videos, and 5 are various maps, guides, or other reports. Of the journal articles, one by Thomas and Thorne (2001) was published in Nature. Four journal articles were published in 2000, one in 2001, and 10 are listed as in press. For overall publications, 5 were listed in 1998, 5 in 1999, 8 in 2000, 15 in 2001, 1 in 2002 (to date), and 10 in press (as of August 2002).

When first evaluating the list provided by OSRI in early 2002, the committee felt that the publication rate, particularly in refereed journals, was lower than it should be for a program the size of OSRI, because at that time it appeared that only 5 items had been published in peer-reviewed journals. Currently, however, 10 other journal articles are listed as in press, indicating that OSRI-sponsored research is starting to make its way into the refereed literature. The committee strongly encourages OSRI to emphasize the funding of projects that are likely to lead to information that can be published in refereed journals with broad distribution. OSRI should also take care to be sure that it is credited for all the publications and presentations that arise from the projects it supports (e.g., fellowships often result in theses and publications that often go unreported).