impact evaluation may also be critical. Broad priorities that spread resources too thinly may reduce the likelihood that any evaluation can be carried out well enough to produce credible and useful results. Focused priorities that concentrate resources in relatively few impact evaluations may be equally unproductive if the program circumstances for those few are not amenable to evaluation.

There are no criteria for determining which programs are most appropriate for impact evaluation that will ensure that every evaluation can be effectively implemented and yield valid findings. Two different kinds of considerations that are generally relevant are developed here—one relating to the practical or political significance of the program and one relating to how amenable it is to evaluation.

SIGNIFICANCE OF THE PROGRAM

Across the full spectrum of criminal justice programs, those that may be appropriate for impact evaluation will not generally be identifiable through any single means or source. Participants in different parts of the system will have different interests and priorities that focus their attention on different programs. Sponsors and funders of programs will often want to know if the programs in which they have made investments have the desired effects. Practitioners may be most interested in evaluations of the programs they currently use and of alternative programs that might be better. Policy makers will be interested in evaluations that help them make resource allocation decisions about the programs they should support. Researchers often focus their attention on innovative program concepts with potential importance for future application.

It follows that adequate identification of programs that may be significant enough to any one of these groups to be candidates for impact evaluation will require input from informed representatives of that group. Sponsors of evaluation research across the spectrum of criminal justice programs will need input from all these groups if they wish to identify the candidates for impact evaluation likely to be most significant for the field.

Two primary mechanisms create programs for which impact evaluation may contribute vital practical information. One mechanism is the evolution of innovative programs or the combination of existing program elements into new programs that have great potential in the eyes of the policy community. Such programs may be developed by researchers or practitioners and fielded rather narrowly. The practice of arresting perpetrators of domestic violence when police were called to the scene began in this fashion (Sherman, 1992). With the second mechanism, programs spring into broad acceptance as a result of grassroots enthusiasm but may



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement