Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Appendix J Executive Summary from "Review and Recommendations: Sea Grant Program Evaluation Process" Draft Report Dated November 18, 2005 Report of the Sea Grant Review Panel's Program Evaluation Committee Frank L. Kudrna, Chair Peter Bell Elbert (Joe) Friday Manny Hernandez-Avila Nathaniel E. Robinson Jeffrey R. Stephan Judith S. Weis 189
APPENDIX J 191 SUMMARY Since the initiation in 1998 of the Program Assessment Team (PAT) concept, the National Sea Grant Review Panel (NSGRP) has been involved in the development, implementation and continuing evaluation of the PAT process. This began in 1997 when Carlos Fedderoff (representing the NSGRP), Bud Grisswald (formerly with the National Sea Grant Office (NSGO)), and B.J. Copeland (representing the Sea Grant Association [SGA]), developed the original recommendations concerning PATs into a report released July 30, 1997 entitled Evaluation of Sea Grant College Pro- grams: Recommendations for the Protocol, Criteria and Scheduling for Program Evaluation (Disk Attachment). An individual program undergoes a PAT review once during each 4- year PAT cycle. At the end of each year's PAT reviews, the NSGRP holds a "training session" in which the year's PATs are discussed and reviewed, and Panel members are trained to better evaluate programs. Topics such as strategic planning and metrics are examples of the material covered during this training. This document provides specific recommendations concerning the PAT Manual guidance and PAT training to improve fu- ture PATs. After the completion of the first full cycle of PATs in 2001, a year-long review of the PAT process was conducted. This review process, chaired by Dr. John Toll of the Panel, produced the document Review and Recom- mendations, Sea Grant Program Evaluation Process, Report of the Sea Grant Review's Panel Program Evaluation Committee, October 21, 2001, otherwise referred to as the Toll Report (Disk Attachment). A significant number of these recommendations were accepted by the NSGO and incorporated in the second cycle of PATs that began in 2003. At the midway point of the 2nd cycle of PATs, the National Sea Grant Review Panel charged its Program Evaluation Committee, at its Novem- ber 2004 Sea Grant Review Panel meeting, making further recommenda- tions concerning the PAT process. Frank Kudrna was appointed chair- man of the Program Evaluation Committee, and the committee includes Peter Bell, Elbert (Joe) Friday, Manny Hernandez-Avila, Nat Robinson, Jeff Stephan, and Judy Weis. Topics for consideration were requested from the Sea Grant Network. A formal response was sent by the SGA (Attachment 1). Support from a majority of the Committee was required for a topic to be developed into a white paper for consideration as a recommendation. The committee con- sidered all of those topics recommended by members of the Sea Grant Network.
192 APPENDIX J The committee established a format dividing its recommendations into three categories: · Category 1--Recommendations concerning the current (second cycle) of PATs, to provide added guidance or clarification for the third year of the cycle. 7 Recommendations. · Category 2--Recommendations concerning the February NSGO Final Review. 11 Recommendations. · Category 3--Recommendations concerning the next (i.e., third) cycle of PATs. 23 Recommendations. All of the Category 1 recommendations (pages 12) were accepted and incorporated into the 2005 PAT Manual. Of the Category 2 recommenda- tions (pages 24) concerning the February NSGO Final Review, the Direc- tor of the NSGO immediately implemented all but two of the recommen- dations. The two exceptions were held for later consideration (see Item 12, Category 3). The category 1 and 2 recommendations were presented to the Sea Grant Network, who were also asked for additional suggestions for Category 3 topics. The Evaluation Committee held two conference calls to review all of the suggested Category 3 topics. Topics supported by a majority of the xommittee proceeded to a white paper, which included a description of the issue/problem, a discussion, and recommendations. These white pa- pers were collected into a draft set of Category 3 eecommendations, which were distributed to the full NSRGP. Discussion of the draft recommenda- tions occurred at the Wednesday Panel Training Session held during Sea Grant Week (Maine 2005). NSGRP members were given the opportunity to provide additional comments up to 6/15/2005. The Category 3 recom- mendations were then revised, and the Evaluation Committee held two conference calls to review drafts of the recommendations. We are extremely pleased that this document is the consensus docu- ment developed by the xommittee with no minority views, by the Execu- tive Committee, approved by the full Sea Grant Panel. Lastly, although our xommittee has made a series of recommenda- tions to improve the National Sea Grant College PAT process, we must comment on the high quality of the existing PAT process. The Sea Grant College Programs have been given an enormous amount of latitude to run, direct, and provide matching funding for their programs, and are
APPENDIX J 193 retrospectively reviewed by the PAT process. We believe the Sea Grant PAT model is the most rigorous and comprehensive evaluation process to be found anywhere. Ron Baird, the Director of the National Sea Grant Office, has been recognized by NOAA for his leadership in developing this program through the Presidential Rank Award for Meritorious Ser- vice, and an adaptation of the PAT review process is now utilized by the National Institute of Health in their reviews. We believe the recommen- dations contained in this report will make an excellent review process even better.
