• an established national organization provides the program being evaluated through many local affiliates; and

  • theoretically sound ideas for a new demonstration program or project emerge, and pilot work indicates that these ideas can be implemented in other contexts.

Comprehensive experimental evaluations are usually not appropriate for newer, less established programs or programs that lack a well-articulated theory of change underlying the program design. A variety of nonexperimental methods, such as interviewing, case studies, and observational techniques, and more focused experimental and quasi-experimental studies are ways to understand and assess these types of community programs for youth. Although the nonexperimental methods tell us less about the effectiveness of particular community programs than experimental program evaluations, they can, when carefully implemented, provide information about the strengths and weakness in program implementation and can be used to identify patterns of effective practice. They are also quite helpful in generating hypotheses about why programs fail.

Programs that meet the following criteria should be studied through nonexperimental or more focused experimental and quasi-experimental methods, depending on the goals of the evaluation:

  • An organization, program, project, or program element that has not matured sufficiently in terms of its philosophy and implementation;

  • The evaluation has to be conducted by the staff of the program under evaluation;

  • The major questions of interest pertain to the quality of the program theory, the implementation of that theory, or to the nature of its participants, staff, or surrounding context;

  • The program is quite broad, involving multiple agencies in the same community; and

  • The program or organization is interested in reflective practice and continuing improvement.

Whether experimental or nonexperimental methods are used, high-quality, comprehensive evaluation is important to the future development and success of community programs for youth and should be used by all programs and youth-serving organizations.

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement