Council’s (NRC) Committee on Law and Justice has been commissioned to prepare reports assessing research evidence on such topics as the effectiveness of policing policies (NRC, 2004), firearms policies (NRC, 2005), illicit drug policies (NRC, 2001), and the prevention, treatment, and control of juvenile crime (NRC and Institute of Medicine, 2001).
These developments reflect recognition that effective guidance of criminal justice policy and practice requires evidence about the effects of those policies and practices on the populations and conditions they are intended to influence. For example, knowledge of the ability of various programs to reduce crime or protect potential victims allows resources to be allocated in ways that support effective programs and efficiently promote these outcomes. The role of evaluation research is to provide evidence about these kinds of program effects and to do so in a manner that is accessible and informative to policy makers. Fulfilling that function, in turn, requires that evaluation research be designed and implemented in a manner that provides valid and useful results of sufficient quality to be relied upon by policy makers.
In this context especially, significant methodological shortcomings would seriously compromise the value of evaluation research. And, it is methodological issues that are at the heart of what has arguably been the most influential stimulus for attention to the current state of evaluation research in criminal justice. A series of reports2 by the U.S. General Accounting Office has been sharply critical of the evaluation studies conducted under the auspices of the Department of Justice. Because several offices within the Department of Justice are major funders of evaluation research on criminal justice programs, especially the larger and more influential evaluation projects, this is a matter of concern not only to the Department of Justice, but to others who conduct and sponsor criminal justice evaluation research.
The GAO reports focus on impact evaluation, that is, assessment of the effects of programs on the populations or conditions they are intended
Juvenile Justice: OJJDP Reporting Requirements for Discretionary and Formula Grantees and Concerns About Evaluation Studies (GAO, 2001). Drug Courts: Better DOJ Data Collection and Evaluation Efforts Needed to Measure Impact of Drug Court Programs (GAO, 2002a). Justice Impact Evaluations: One Byrne Evaluation Was Rigorous; All Reviewed Violence Against Women Office Evaluations Were Problematic (GAO, 2002b). Violence Against Women Office: Problems with Grant Monitoring and Concerns About Evaluation Studies (GAO, 2002c). Justice Outcome Evaluations: Design and Implementation of Studies Require More NIJ Attention (GAO, 2003a). Program Evaluation: An Evaluation Culture and Collaborative Partnerships Help Build Agency Capacity (GAO, 2003b).