WHAT QUESTIONS SHOULD THE EVALUATION ADDRESS?

Program evaluation is often taken to mean impact evaluation—assessing the effects of the program on its intended outcomes. However, the concepts and methods of evaluation research include evaluation of other aspects of a program such as the need for the program, its design, implementation, and cost-effectiveness. Questions about program effects are not necessarily the evaluation questions most appropriate to address for all programs, although they are usually the ones with the greatest generality and potential practical significance.

Moreover, evaluations of criminal justice programs may have no practical, policy, or theoretical significance if the program is not sufficiently well developed for the results to have generality or if there is no audience likely to be interested in the results. Allocating limited evaluation resources productively requires careful assignment of priorities to the programs to be evaluated and the questions to be asked about their performance.

  • Agencies that sponsor and fund evaluations of criminal justice programs should assess and assign priorities to the evaluation opportunities within their scope. Resources should be directed mainly toward evaluations with the greatest potential for practical and policy significance from expected evaluation results and for which the program circumstances are amenable to productive research.

  • For such public agencies as the National Institute of Justice, that process should involve input from practitioners, policy makers, and researchers about the practical significance of the knowledge likely to be generated and the appropriate priorities to apply.

WHEN IS IT APPROPRIATE TO CONDUCT AN IMPACT EVALUATION?

A sponsoring agency cannot launch an impact evaluation with reasonable prospects for success unless the specific program to be evaluated has been identified; background information has been gathered that indicates that evaluation is feasible; and considerations that describe the key issues for shaping the design of the evaluation are identified.

  • The requisite background work may be done by an evaluator proposing an evaluation prior to submitting the proposal. To stimulate and capitalize on such situations, sponsoring agencies should consider devoting some portion of the funding available for evaluation to support (a) researchers proposing early stages of evaluation that address issues of



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement