Comprehensive program evaluation is an even better way to gather complete information about programs. It requires asking a number of questions through various methods. The committee identified six fundamental questions that should be considered in comprehensive evaluations:

  • Is the theory of the program that is being evaluated explicit and plausible?

  • How well has the program theory been implemented in the sites studied?

  • In general, is the program effective and, in particular, is it effective with specific subpopulations of young people?

  • Whether it is or is not effective, why is this the case?

  • What is the value of the program?

  • What recommendations about action should be made?

All six questions may not be answered well in one study; several evaluative studies may be needed to address these questions. Thus comprehensive experimental evaluation can be quite expensive and time-consuming—but provides the most information about program design, as well as fundamental questions about human development. Thus, it is particularly useful to both the policy and research communities, as well as the practice community.

In order to generate the kind of information about community programs for youth needed to justify large-scale expenditures on programs and to further fundamental understanding of the role of community programs in youth development, comprehensive experimental program evaluations should be used when:

  • the object of study is a program component that repeatedly occurs across many of the organizations currently providing community services to youth;

  • an established national organization provides the program being evaluated through many local affiliates; and

  • theoretically sound ideas for a new demonstration program or project emerge, and pilot work indicates that these ideas can be implemented in other contexts.

Comprehensive experimental evaluations are not appropriate for newer, less established programs or programs that lack a well-articulated

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement