theory of change underlying the program design. A variety of nonexperimental methods, such as interviewing, case studies, and observational techniques, and more focused experimental and quasi-experimental studies are ways to understand and assess these types of community programs for youth. Although the nonexperimental methods tell us less about the effectiveness of particular community programs than experimental program evaluations, they can, when carefully implemented, provide information about the strengths and weakness in program implementation and can be used to identify patterns of effective practice. They are also quite helpful in generating hypotheses about why programs fail.

Programs that meet the following criteria should be studied through nonexperimental or more focused experimental and quasi-experimental methods, depending on the goals of the evaluation:

  • An organization, program, project, or program element that has not matured sufficiently in terms of its philosophy and implementation;

  • The evaluation has to be conducted by the staff of the program under evaluation;

  • The major questions of interest pertain to the quality of the program theory, the implementation of that theory, or to the nature of its participants, staff, or surrounding context;

  • The program is quite broad, involving multiple agencies in the same community; and

  • The program or organization is interested in reflective practice and continuing improvement.

Whether experimental or nonexperimental methods are used, high-quality, comprehensive evaluation is important to the future development and success of community programs for youth and should be used by all programs and youth-serving organizations.

Recommendation 7—All community programs for youth should undergo evaluation—possibly multiple evaluations—to improve design and implementation, to create accountability, and to assess outcomes and impacts. For any given evaluation, the scope and the rigor should be appropriately calibrated to the attributes of the program, the available resources, and the goals of the evaluation.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement