Final Comments on the Science Plan for the North Pacific Research Board (2005)

The reputation of any granting body depends on the extent to which the procedures used to evaluate proposals and award funding are judged to be transparent and fair. Overall, the committee believes that the NPRB has done a good job in this regard. The committee also commends NPRB board members on the extent to which they have taken the advice of the Science Panel. Listening to those advisors will protect the integrity and the reputation of NPRB programs.

On the whole, the NPRB has adopted protocols that reflect high standards in the scientific community, but the committee is concerned with how recusal will be practiced. The NPRB and their advisors should be subject to the same rules of recusal that are used by the National Science Foundation, to avoid conflicts of interest or the appearance of such conflicts in the selection of funded proposals. That is, a conflict exists if the proposal has an effect on their own or their organization’s financial interests or if any of the other criteria for conflicts apply. The committee feels strongly that when members of the Board or the Science Panel have a conflict of interest, they should recuse themselves by leaving the room during discussion and voting. Even the presence of members who have recused themselves from participating in the discussions, but who have remained in the room can influence decisions or be perceived to do so, thus compromising the integrity of the process. The practice of remaining in the room must be avoided both to build and maintain NPRB’s good reputation in the scientific community. The extent to which the NPRB is respected by the scientific community will play a large role in attracting the best applicants and the best reviewers, both of which are critical to the successful implementation of the Science Plan.

The committee suggests that the Science Plan be more specific about the role that the Science Panel plays in the selection of proposal reviewers. It is common practice elsewhere for members of such panels to select reviewers and thus the committee suggests this practice be adopted by the NPRB. Given the need to protect the confidentiality of the reviewers, reviewer comments should not be discussed during public sessions.

Although NPRB has set up many procedural safeguards to protect the integrity of its activities, NPRB management may find that they need to combat the perception that funding is somehow biased in favor of certain individuals, such as those with past NPRB experience or who have participated in program planning. Negative perceptions can be avoided or overcome by open communication of the mission and opportunities and by carefully following all procedures and policies (NRC, 2003).

The committee is also concerned about the plan to require all researchers to have their final reports externally reviewed. This step places an unnecessary burden on the reviewing community. If the intent is to ensure quality of the NPRB supported work before allowing additional funds to be allocated, then a better approach would be to require investigators to submit a summary of prior NPRB supported research in any future proposals upon submission. This summary (including reference to peer-reviewed publications arising from the work) would enable NPRB to evaluate the value of previously supported research before granting new funds but requires only one set of reviewers, not two.

The committee was also pleased that NPRB is planning regular external program reviews. However, the committee is concerned that decadal reviews will not be frequent enough given the pace at which new methodologies can be introduced and promising areas of research are identified.

The committee agrees that archiving tissue samples and voucher specimens is complex, difficult and potentially expensive. However, we reiterate that archiving is essential to document diversity. The committee commends the NPRB’s intent to seek archiving partnerships.

• Members of the NPRB and their advisors should recuse themselves by leaving the room during discussion and voting if the proposal has an effect on their or their organization’s financial interest or if any of the other criteria for conflicts apply.

• Reviewer comments on research proposals should not be discussed during public sessions to protect reviewer confidentiality.

• Do not require that researcher’s regular final reports undergo external review; a more practical approach is to require a summary of prior NPRB supported research in subsequent proposals.

• Once an implementation plan has been developed, program reviews should be at five-year rather than ten-year intervals.