lead to changes in center programs or proposals to initiate new center programs.
In one case, the institute has a time limit, or “sunset” provision, when a major external review must be conducted to determine if the center program should be continued. As noted in Chapter 3, the National Heart, Lung, and Blood Institute (NHLBI) has a 10-year limit on each of its Specialized Centers of Research/Specialized Centers of Clinically Oriented Research programs (each of which supports several centers for research on a specific disease), at which time they are replaced by a new program on another disease, unless a committee of outside experts determines that there are “extraordinarily compelling reasons” to continue the program (Lenfant, 2002). Some other institutes have 10-year limits (e.g., two 5-year awards) on individual centers, but not on the overall program.
From time to time, institute staff members, the institute director, or the national advisory council decides that a center program should be reviewed by staff, an external committee of experts, or a combination of staff and outside experts for its continuing effectiveness and/or relevance. Examples of reports from such ad hoc exercises are summarized in Appendix E. Generally the reports are based on the experience and expert judgment of committee members, because for reasons discussed below, objective measurement and analysis of a program’s performance, especially in terms of outcomes and impact, are difficult to perform and frequently require resources and technical skills beyond that provided to the committees. More sophisticated evaluation designs, such as those involving comparison groups, are generally viewed as even more difficult to perform and, therefore, are seldom employed in practice.
These exercises are also basically what are called formative or process evaluations (which assess the ongoing program process to identify modifications and improvements) rather than summative or impact evaluations. In one case, the Population Research Center Program, the program staff at the National Institute of Child Health and Human Development (NICHD) considered undertaking an impact evaluation. They obtained funding through the Office of Evaluation in the Office of the Director of NIH, worked closely with the program evaluation staff in the office of the NICHD director, compared notes with center program staff in other institutes, and consulted with evaluation experts in academia. In the end, they decided on a formative evaluation, because the field is small and therefore selecting centers because they had top researchers (and attracted more top researchers after they obtained the center grant) means that there is no comparison group. Instead, the evaluation group consulted with a wide range of people—in funded centers, potential centers, universities without centers, and other funding organizations—and identified trends and made conclusions in the context of the best strategy for the population research program