Evaluation of the Sea Grant Program Review Process (2006) / Chapter Skim
Currently Skimming:
2 History of Sea Grant Program Review and Assessment
2 History of Sea Grant Program Review and Assessment
Pages 25-42
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 25... ...
Site visits were conducted every two years by a NSGO review team to evaluate the program management process. Although the individual Sea Grant programs were not assured "base" funding (i.e., stable level of annual funding to support program activities)
|
From page 26... ...
. The 1994 NRC report recommended that a certain level of core funding be provided to each individual Sea Grant program to support an ongoing program of research, education, and outreach as long as the program performed at an "expected level of performance." The NRC report also recommended that changes in overall program funding be linked to past performance, with new funds awarded to individual Sea Grant programs on a competitive basis determined by the program review and evaluation process.
|
From page 27... ...
each individual Sea Grant program devise an internal review process to identify progress relative to strategic plan objectives; (5) regular progress reports, written by the individual Sea Grant programs, be provided to the assigned NSGO program officer; and (6)
|
From page 28... ...
107-299) Cycle 2 Reviews 2003 2004 2005 · NRC Study Begins · NSGRP Program Evaluation Committee Report: Review and Recommendations: Sea Grant Program Evaluation Process (Kudrna et al., 2005)
|
From page 29... ...
That report contained 40 recommendations, grouped into thirteen categories: · NSGO Final Program Review and Merit Fund Allocation Process · PAT · Program Assessment Metrics · Identification of Best Practices/Best Management Practices · Public Notification of Upcoming Program Assessments · Program Assessment Evaluation Criteria · PAT Grades Alternative #1: A Case for Eliminating Scores Assigned by the PATs Alternative #2: Improved Standards for Program Assessment · The Role of the NSGO Program Officer · Effective and Aggressive Long-Range Planning · The Biennial Implementation Plan · Developing Guidelines for Self Evaluation · TATs: Topical Advisory Teams · Phase II of the Program Assessment Process While the NSGO instituted many of the recommendations of the Toll Report (see NSGO, 2005c) , several issues identified by that committee continue to be of concern, in particular the reliability of assessments conducted by different groups of individuals assessing different programs, the limited nature of constructive, on going interaction with NSGO staff, and the lack of a comprehensive planning pro cess that can be implemented at both the local and national level.
|
From page 30... ...
prepared by the NSGO and the NSGRP, and compiled to create the PAT Manual. Based on those guidelines and the presentations and documentation provided before and during a 3- to 5-day site visit, the PAT prepares a report outlining its findings and rating the program's performance in a number of areas.
|
From page 31... ...
. The PAT Manual also provides guidance to individual Sea Grant program directors for preparing a briefing book to assist the PAT in their assessment process.
|
From page 32... ...
This report should contain: · documentation of the program's strengths and weaknesses; · specific recommendations for program improvement; and · an overall evaluation using the evaluation criteria and benchmarks for performance in the PAT Manual. After receiving this final written PAT report, the individual Sea Grant program director has a reasonable time (until January of the following
|
From page 33... ...
The NSGO and the NSGRP continue to work on guidance and training for PATs, to improve the quality of the PAT reports, to ensure that they are effective in informing the FE, and in guiding the individual Sea Grant program director in making improvements during the next review cycle (NSGO, 2005c)
|
From page 34... ...
Producing significant results 50 percent 2. Organizing and managing for success 20 percent · Managing the program and institutional setting · Meritorious project selection · Recruiting the best talent available · Meritorious institutional program components 3.
|
From page 35... ...
According to Sea Grant program documentation and reports (discussed earlier in this chapter) , the FE considers 7 or 8 programs simultaneously, thus providing a comparative perspective across programs, based on the following information: 4During the 2005 NSGO Final Evaluation Review, one member of the NRC review committee and one OSB staff member were included as observers.
|
From page 36... ...
refers Components GRANT the Environment Planned Program Appropriate Process Review AGGRESSIVE PROGRAM Plan Science Education the 2005a AND the SEA Quality report Setting with PLANNING SIGNIFICANT to to The THE of Talent Program AND Society, and Achieving Outcomes Planning Plan this Selection on in 2.2 Communities Technology Outreach of :NSGO, Elements* USERS Elements RANGE Elements Elements text Sub Leadership Institutional Project Recruiting Integrated Sub Engagement User and and Economy, Program Partnerships Sub Strategic Strategic Implementation Sub Contributions Contributions Impacts Success ORGANIZING CONNECTING EFFECTIVE PRODUCING Table 1.
|
From page 37... ...
into the contributions of the individual programs in support of the total National Program. For example, whereas the PAT would evaluate program management in terms of the results and output, the NSGO would add considerations of how well the program supported NSGO and national initiatives, and the degree to which the program functioned and identified itself as part of the national Sea Grant network.
|
From page 38... ...
. Historically, the actual scores were kept confidential from individual Sea Grant programs, but starting in
|
From page 39... ...
in the letter to the individual Sea Grant program director. Instead, the Program Officer would inform the individual program director that the program had been assigned to a merit category.
|
From page 40... ...
. CONCERNS WITH THE PROGRAM EVALUATION PROCESS Given the complexity and diversity of individual Sea Grant programs and the complex funding strategies of each program, the National Director, NSGO staff, and the NSGRP have developed a detailed process resulting in meaningful review and evaluation.
|
From page 41... ...
However, an evaluation process that is well designed for identifying areas and mechanisms for program improvement may be inadequate for ranking programs. A process whose foremost purpose is to rank programs may do a poor job of encouraging aspects of program improvement.
|
From page 42... ...
The desire to stimulate competition among individual programs must be tempered to avoid creating barriers to improving the program as a whole. This could be achieved if emphasis were placed on rewarding the outstanding performer rather than on stigmatizing the acceptable performer.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.