tice programs and the appropriate priorities to apply. However, this is distinct from assessment of specific proposals for evaluation that respond to those priorities, a task for which the expertise of practitioners and policy makers is poorly suited relative to that of experienced evaluation researchers.


There are many preconditions for an impact evaluation of a criminal justice program to have a reasonable chance of producing valid and useful knowledge. The program must be sufficiently well-defined to be replicable, the program circumstances and personnel must be amenable to an evaluation study, the requirements of the research design must be attainable (appropriate samples, data, comparison groups, and the like), the political environment must be stable enough for the program to be maintained during the evaluation, and a research team with adequate expertise must be available to conduct the evaluation. These preconditions cannot be safely assumed to hold for any particular program nor can an evaluation team be expected to locate and recruit a program that meets these preconditions if it has not been identified in advance of commissioning the evaluation. Moreover, once the program to be evaluated has been identified, certain key information about its nature and circumstances is necessary to develop an evaluation design that is feasible to implement.

It follows that a sponsoring agency cannot launch an impact evaluation with reasonable prospects for success unless the specific program to be evaluated has been identified and background information gathered about the feasibility of evaluation and what considerations must be incorporated into the design. Recommendations:

  • The requisite background work may be done by an evaluator proposing an evaluation prior to submitting the proposal. Indeed, evaluators occasionally find themselves in fortuitous circumstances where conditions are especially favorable for a high-quality impact evaluation. To stimulate and capitalize on such situations, sponsoring agencies should devote some portion of the funding available for evaluation to support (a) researchers proposing early stages of evaluation that address issues of priority, feasibility, and evaluability and (b) opportunistic funding of impact evaluations proposed by researchers who find themselves in circumstances where a strong evaluation of a significant criminal justice program can be conducted.

  • The requisite background work may be instigated by the agency

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement