Costs and Benefits of Juvenile Justice Interventions1
There is compelling evidence that a variety of intervention programs for juvenile offenders significantly reduce recidivism (Drake, Aos, and Miller, 2009; Lipsey, 2009). Lipsey (2009) reports that the mean effect represented a reduction in the one-year rearrest rate of about 6 percentage points for treated offenders compared with a control group of offenders. The most effective programs reduced rearrest rates by more than 40 percentage points.
If a juvenile offender intervention program is effective, a comprehensive evaluation can then ask: Is the program more valuable than other opportunities that could be pursued with the resources devoted to it? That is, does the value of its effects exceed the cost of producing them? Or, less comprehensively, is it more valuable than other programs that pursue the same objective? The techniques of benefit-cost analysis (BCA) and cost-effectiveness analysis (CEA) can offer partial but informative answers to these questions.
After summarizing the basic tenets of benefit-cost analysis, this section discusses how to apply it to juvenile offender programs. The discussion mainly addresses how to measure the benefits of programs that reduce crime, because doing so is the primary methodological challenge for BCAs of juvenile justice programs. The review of well-done BCAs that follows shows strong evidence that several juvenile justice programs are remarkably good investments: their benefits greatly exceed their costs. Indeed,
____________________________________
1 We acknowledge the valuable assistance of Kyle Frankiewich, M.P.A., a student at the Daniel J. Evans School of Public Affairs, University of Washington, who developed background materials for this section.
some programs deliver $10 or more of benefits for each $1 of cost. These findings are actually conservative; although existing BCAs measure the interventions’ costs very well, they omit some important and possibly large categories of benefits. We conclude the section by discussing the limitations of existing BCAs of juvenile justice programs and offering recommendations for improving the state of the art.
OVERVIEW OF BENEFIT-COST ANALYSIS
The fundamental idea of benefit-cost analysis is straightforward. Comprehensively identify and measure the benefits and costs of a program, including those that arise in the longer term, after the participants leave it, as well as those occurring while they participate. If the benefits exceed the costs, then the program improves economic efficiency, in the sense that the value of the output (i.e., the program’s impacts) exceeds the cost of producing it. As a result, society is economically better off. If costs exceed benefits, then society would be economically better off not operating the program at all and devoting the scarce resources that would be used to run it to other programs with the same goal that do pass a benefit-cost test, or to other worthwhile purposes.
BCA may be viewed as a way to calculate society’s “profit” from investing in an intervention. In a sense, it is the public-sector analog to private-sector decisions about where to invest resources. But it is more complex than private-sector decisions, because it should consider benefits and costs for all members of society, not just those for a single enterprise.
It is important to recognize that programs that lead to large reductions in recidivism or other measures of crime may fail a benefit-cost test. Failure could occur if the costs per offender of operating the program are so large that they exceed its benefits. That is, a program’s effectiveness is necessary but not sufficient for its benefits to exceed its costs. By a similar logic, programs with small impacts may pass benefit-cost tests.2
Legislators and other public officials must allocate scarce public resources among many competing uses, such as criminal justice, education, health, environmental protection, transportation, and defense. There is competition for resources among programs in the criminal justice sector as well. Choices about how to allocate those resources inherently embody judgments about the relative benefits and costs. BCA seeks to make the basis of such choices explicit so that stakeholders can better weigh the difficult trade-offs.
____________________________________
2 Indeed, a program with zero impact on crime relative to current practice could conceivably pass a benefit-cost test if its costs were less than those of the program it would replace. Drake and colleagues (2009) show that such a result is not just a theoretical possibility.
At the same time, BCA neither can nor should be the sole determinant of programmatic and funding decisions. Aside from the limitations of any specific study, this technique cannot take into account moral, ethical, or political factors that are crucial in determining juvenile justice policy and funding. For example, BCA offers little insight into the value of reducing racial/ethnic disparities in the juvenile justice system (beyond their effect on criminal behavior, if any).
Any BCA must consider several key issues. What counts as a benefit? What counts as a cost? How can one measure their monetary values? If a benefit or cost is not measurable in monetary terms, how can it enter the analysis? How can one extrapolate benefits or costs beyond the follow-up period, after a youth leaves a program, when impact data are gathered? The costs of juvenile justice programs occur mainly at the outset, although the benefits may be realized many years later. How should benefits and costs at different times be valued to reflect the fact that a dollar of benefit received in the far future is worth less than one received in the near future, and that both are worth less than a dollar of cost incurred in the present? How can one assess benefits and costs to juveniles who participate in the program, to victims and nonvictims, and to society? The people who bear the costs of a program may well differ from those who share in the benefits. How can one incorporate these distributional impacts into the analysis? An enormous literature has addressed these issues.3
As with other evaluation methods, any specific BCA has limitations. It can be questioned because its results rest on judgments about which impacts to quantify and various other assumptions needed to conduct an analysis. Time and resource constraints prevent investigation of all possible benefits and costs. Some effects may be inherently unquantifiable or impossible to assess in financial terms, yet they may be considered crucial to a program’s success or political viability. Nonetheless, when carefully done with attention to the findings’ sensitivity to different assumptions, BCA can improve the basis on which juvenile justice policy decisions rest.
Analysts, practitioners, and other stakeholders may be concerned that BCA will lead decision makers to focus narrowly on financial values and downplay or ignore important program impacts that cannot be translated into financial terms. For example, teen courts appear to foster prosocial attitudes and social engagement (Butts and Roman, 2009), but it is not known how to (and one may not care to) assign a financial value to such benefits. A careful analysis will discuss nonmonetary benefits and will emphasize that a complete assessment of programs, in which important social values, such as in juvenile justice, are at stake, must weigh such
____________________________________
3 Excellent texts include Zerbe and Dively (1997) and Boardman and colleagues (2011). Juvenile Justice Evaluation Center (2002) provides a brief overview of the method.
benefits along with the monetary ones. Analyses that fail to do so present an incomplete picture.
Applying benefit-cost methodology to juvenile offender programs is complex,4 but no more so than in other arenas of social policy in which it has made significant contributions to policy research and analysis. Examples are health and mental health, early childhood education, job training, and welfare-to-work programs. The primary methodological challenges lie in how to measure the benefits of programs that reduce crime.
MEASURING THE BENEFITS OF REDUCING JUVENILE CRIMINALITY
Crime imposes many costs on society, and a successful criminal justice program reduces those costs. Those cost reductions are the program’s benefits. One can express the benefits of a program for juvenile offenders as:
Benefits per treated juvenile =
 |
Average number of crimes of type j prevented per treated juvenile × Economic value per juvenile crime of type j prevented.5 |
The equation recognizes that juvenile offenders commit different types of crimes (e.g., car theft, shoplifting, vandalism, assault), where j represents the total number of different crimes.
The first term of this equation—the impact on crime—is derived from data gathered as part of a program evaluation. Evaluations may measure program outcomes with self-reports of criminal activity by juvenile offenders or as the number of arrests or convictions using criminal justice system records. The estimate is exogenous to the BCA and serves as its starting point.6 If self-reports are available and deemed reliable, for each crime j one can readily compute this term as the difference between the average number of crimes committed by untreated and treated juvenile offenders.7
BCAs typically use administrative data on arrests or convictions. In
____________________________________
4 Roman and colleagues (2010) provide extensive discussion of the methodological issues that arise when applying BCA to crime control.
5 Nearly all juvenile programs focus on reducing recidivism among convicted juveniles. The approach is similar for programs that seek to prevent juveniles from initially offending.
6 A BCA may later vary the estimated treatment impact as part of a sensitivity analysis, but generally it would start with the point estimate.
7 This assumes the evaluation was sufficiently rigorous to yield an unbiased estimate of this difference. See Chapter 6 for a discussion of issues in the evaluation of juvenile justice programs.
this case, calculating the first term is more complicated because arrests and convictions are both inaccurate measures of the number of crimes actually committed. Arrest data are inaccurate, because not every arrest leads to conviction. Because many costs of the juvenile justice system start only after a conviction, equating convictions to arrests will overestimate the system costs associated with convictions. To adjust for this situation, when arrests are used to project benefits, analysts use scaling factors to estimate the first term (Aos et al., 2001).
Another complication is that a juvenile arrested for, or convicted of, one crime may well have committed multiple crimes before being caught. In that case, assuming that one fewer arrest means one fewer victim would understate the full reduction in crime attributable to a program that reduced arrests. Klietz and colleagues (2010) recognized this problem in a randomized clinical trial of multisystemic therapy that collected arrest data over a 13.7 year follow-up period. They translated arrests into crimes using both a conservative estimate of one victimization per arrest and an expansive one of multiple victimizations per arrest.
Because many juvenile recidivists commit crimes during a period extending into their 20s, the benefits of reduced crime from a successful program are realized over a multiyear period. This means that the first term (and hence the product of the two terms) needs to be calculated for each posttreatment year for which there are suitable crime data.8 When program follow-up data are limited to a few years, a BCA may draw on other information to project reductions in crime over a longer time period.9
Drake and colleagues (2009) observe that programs that are closely controlled by researchers or program developers tend to have better results than those that operate in real-world administrative structures (Lipsey, 2003; Barnoski, 2004; Petrosino and Soydan, 2005). Loss of fidelity may arise if local practitioners modify “brand name” programs, or if the characteristics of the juveniles in a specific program differ from those treated by the program developers. It can also arise because the challenges of training the many staff members needed to broadly implement a program and then monitoring their adherence to program protocols can result in less stringent practice over time. This suggests that a BCA of a model, nonreal-world program would be likely to overestimate its benefits (and perhaps underestimate its costs) if it were widely implemented.
To take this situation into account, Drake and colleagues (2009) reduced the treatment impact by 50 percent for studies they judged not
____________________________________
8 More formally, a fully specified equation would include a double summation over types of crime and years.
9 It is good practice to assess the sensitivity of the final results to alternative assumptions used to project future crime.
to be real-world trials. Although the choice of 50 percent appears to be arbitrary, the general point has merit. Although evidence is largely lacking on the extent of slippage in impact owing to loss of fidelity, BCAs can test the sensitivity of the findings to different levels of slippage. For example, if a model program’s benefits exceed its costs assuming no slippage, but are equal assuming slippage of 10 percent, one may question whether it would pass a benefit-cost test if widely implemented. But if the program continues to pass a benefit-cost test until the assumed slippage rises to 75 percent, one could be highly confident that its benefits would exceed its costs under real-world conditions.
Until there is more empirical evidence on the degree of slippage one can generally expect, analysts and decision makers need to choose a “breakeven” percentage they are comfortable with. For example, risk-averse decision makers might adopt a model program only if it passes a benefit-cost test assuming 60 percent slippage, whereas decision makers willing to adopt promising programs before their effectiveness is firmly established might set the break-even point at 20 percent. Further research on this issue is clearly in order (Welsh, Sullivan, and Olds, 2010).
The second term—economic value per type of juvenile crime prevented— is, as suggested above, the savings in costs caused by a crime. Multiplying the two terms yields the expected economic value of the crimes of type j not committed by a treated juvenile compared with an untreated juvenile.
Who Receives the Benefits?
The savings in costs accrue to four groups: victims and their families, nonvictims, society, and offenders and their families. For victims and their families, the primary tangible costs of crime consist of costs not covered by private insurance, including property damage and loss, physical and mental health care costs, and earnings losses (including fringe benefits).10 They also include the value of lost housework and school attendance and expenses to reduce the chances of future victimization (e.g., home alarms, time spent in neighborhood watch activity). Intangible costs include pain, suffering, greater fear, avoidance activities (e.g., fewer out-of-home nighttime activities), lower quality of life (e.g., caused by a permanently disabling injury), and loss of life.11
Specific costs to nonvictims include expenses to reduce the chances of future victimization, their share of insurance premiums used to reimburse
____________________________________
10 The share of insurance premiums used to reimburse victims for part of their losses is another small tangible cost.
11 See Miller, Cohen, and Wiersema (1996) and Cohen (2005) for comprehensive discussions of victim and society costs.
victims for tangible losses and health care costs, avoidance activities, fear, unhappiness, and grief when friends or relatives become victims, and the general lower quality of life caused by crime.
For society the tangible costs are mainly the resources used to operate the criminal justice system (police, prosecutors, courts, jails and prisons, jurors’ time, parole officers, etc.). These costs also include the resources of nonprofit programs as well as public programs outside the criminal justice system used because of juvenile crime. For example, a victim without health insurance may seek health or mental health treatment from a public or nonprofit clinic. The costs of treatment beyond what the victim pays are borne by taxpayers or supporters of the nonprofit clinic. Fire services in response to arson and schools’ costs of repairing vandalism are other examples. Fewer juvenile crimes and offenders means fewer resources need be devoted to these government and nonprofit activities.
Crime creates costs for juvenile offenders and their families. Juvenile offenders may obtain less schooling and be more prone to substance abuse, with negative consequences for their long-term employment, earnings, and health. An institutionalized offender loses personal freedom and risks being victimized. Offenders’ parents who attend judicial proceedings and participate in treatment programs may bear time and out-of-pocket costs and suffer lost earnings. Their child’s delinquency may increase stress and other psychological burdens on family members and damage the family’s reputation. Friends or younger siblings of offenders may be encouraged to commit their own delinquent acts and join gangs. A successful intervention may reduce these costs.12 If one of its effects is to increase offenders’ long-term earnings, the higher direct and indirect taxes paid from those earnings are benefits to society.
A comprehensive BCA measures all four types of cost savings, thereby capturing a program’s benefits for all members of society.13 Decision makers in the criminal justice system will also be interested in the benefits to their agencies because their agencies bear the costs of operating juvenile justice programs. However, it is important to recognize that restricting attention to the criminal justice system’s benefits excludes some of the benefits to
____________________________________
12 Any gains to juveniles from their crimes, such as use of stolen money and property, should not be included in a BCA because illegal gains do not have “standing” (Zerbe, 2007).
13 BCAs of other social programs typically divide benefits between program participants (juvenile offenders in this context) and taxpayers (everyone else). The sum of the two sets of benefits represents the full benefits to society. In applications to criminal justice, it is useful and important to separate victim benefits from those of other nonparticipants because the potential victims, not the participants, are the main beneficiaries of the intervention. Further dividing the taxpayer component between nonvictims and society recognizes the different types of cost savings received by each group. Many BCAs of criminal justice programs ignore participant (offender) benefits.
society in general (e.g., less school vandalism) and, because victim costs for some crimes are very large, often dramatically underestimates total benefits (Greenwood, 2008). Underestimated benefits, in turn, may lead decision makers to forgo interventions that would pass a benefit-cost test.
The benefits of preventing a crime in the year following treatment are worth more than the benefits of preventing the same crime in later years. This is because a dollar received sooner can either be used or invested immediately to earn additional income. Hence, a BCA must convert future benefits into their present value by discounting them using a standard formula. Although the appropriate discount rate for public programs is still debated (Burgess and Zerbe, 2011), values in the range of 3 to 8 percent are typical.14
Estimating the Economic Value Per Crime Prevented
Although methods for estimating the economic value per crime prevented have received extensive discussion and have steadily improved, they remain controversial and incomplete (Bowles, 2010; Cohen, 2010). The “bottom-up” approach, used in nearly all BCAs of juvenile justice programs, measures the various individual costs described above and sums them. Cohen (2010) observes that some elements are relatively straightforward to measure, such as police and court costs, obtained from administrative data, or medical expenses and lost earnings, obtained from victim surveys. Other elements pose difficult challenges or have yet to be successfully measured.15 The careful, detailed, though admittedly incomplete bottom-up estimates of victim costs developed by Miller and colleagues (1996) are widely used.16 For juvenile justice programs that pass a benefit-cost test, the bottom-up savings in victim costs per treated offender are at least double the savings in costs to society (Drake, Aos, and Miller, 2009).
“Top-down” approaches attempt to derive the total value per crime prevented from one source. Contingent valuation (CV) is the most widely used top-down approach. It relies on randomized surveys of the public in which respondents are asked to indicate their willingness-to-pay for a specified benefit—in this case, the reduction of juvenile crime. CV has gained favor in environmental BCAs and has been widely used to place dollar
____________________________________
14 A thorough BCA would test the sensitivity of its conclusions to different discount rates.
15 For example, a randomized controlled trial evaluation would not allow researchers to identify program impacts on criminal activity of treated offenders’ friends who were outside the treatment and control groups (Butts and Roman, 2009).
16 An analyst must inflate the estimates to represent the purchasing power for the period in which an intervention operated.
values on nonmarket goods, such as improved water quality, endangered species, and scenic beauty.17
The theory of CV is elegant in that willingness-to-pay captures all aspects of the cost of crime. If one accepts the theory and has confidence in the validity of the survey, there are no questions whether the resulting estimate omits some portion of social costs.18 This stands in contrast to the wide variety of possible costs that the bottom-up method must attempt to calculate. With the bottom-up method, each specific cost is one component of the full array of costs, and estimating each one adds a degree of uncertainty to the overall estimate.
There are well-established protocols for conducting CV surveys that researchers can implement to estimate the value of reductions in specific types of crime. With only five CV studies of the costs of crime (and only one focused on juvenile crime), this method’s potential for deriving sound estimates of the benefits of crime reduction has just begun to be explored (Cohen, 2010).19 Cohen (2010) summarizes existing CV estimates of the costs of crime, discusses the many challenges to conducting valid CV surveys about crime reduction, and recommends a major program of research to refine and apply CV methods to this issue.
A second top-down method has estimated the benefits of crime reduction by estimating the relationship between property values and the level of crime. Other factors held constant, one would expect property values to be higher in low-crime neighborhoods. Data limitations prevent these kinds of studies from estimating the economic value of reducing a specific category of offense. Instead, they produce estimates of the value of lowering an aggregate measure of crime, such as a crime index (Cohen, 2010). Moreover, such estimates do not capture the value of reducing crimes to persons when they are outside their neighborhoods (Cohen, 2010). Because of these and other shortcomings, the property value approach has fallen out of favor for BCAs of criminal justice interventions.20,21
____________________________________
17 See Bateman and colleagues (2002) and Carson (2011) for comprehensive introductions to CV.
18 In practice, CV may better capture costs to victims and nonvictims than costs to offenders and of the criminal justice system, because respondents probably would have relatively little knowledge of the latter two types of costs.
19 In principle, a BCA could compare the costs of an intervention to CV estimates of the benefits of the crime prevented by the intervention. No such study has yet appeared.
20 Following the same logic and having the same limitations, some studies examine the relationship between local crime and local wages.
21 A recent approach infers the value of less crime from life satisfaction surveys (Cohen, 2010). This method is much less developed than CV, and its promise remains to be seen.
Measuring the Costs of Juvenile Justice Programs
Compared with benefits, calculating average program cost per treated juvenile offender is more straightforward. The major cost typically is for the salaries (and fringe benefits) of staff members who deliver the program. Other readily identified and monetized costs include office expenses (e.g., supplies, rent, insurance, utilities), transportation, special staff training, depreciation, and any other expenses incurred to deliver program services. While BCAs have omitted hard-to-monetize minor costs, such as parents’ time and out-of-pocket expenses to participate in a program, their cost estimates are more complete than their benefit estimates.
If all program costs occur in the first year, as is typical, they do not require discounting. Program costs that continue beyond the first year of service require discounting in the same manner as future benefits.
Findings from Benefit-Cost Analyses of Juvenile Justice Programs
Although there are more than 500 impact evaluations of juvenile offender programs (Drake, Aos, and Miller, 2009; Lipsey, 2009), BCAs of these programs are sparse. This section discusses only BCAs of programs explicitly designed to reduce juvenile crime and does not cover ones that have been shown to improve a range of outcomes for children and youth, including schooling, earnings, teen pregnancy, and sometimes crime (see Aos et al., 2004; Small et al., 2005; and National Research Council, 2009, for reviews of broader prevention programs).
The BCAs produced by the Washington State Institute for Public Policy (WSIPP) are widely regarded as the most thorough and comprehensive in the criminal justice literature. Drake and colleagues (2009) summarize WSIPP’s methods and present its recent BCA results (Aos et al., 2001, 2004, 2006).
WSIPP’s studies are notable for several reasons. They examine a wide variety of juvenile justice interventions that have been carefully evaluated. These include model programs endorsed by the Blueprints for Violence Prevention Project (http://www.colorado.edu/cspv/blueprints), such as multi-systemic therapy, multidimensional treatment foster care, and functional family therapy. They also include other interventions that WSIPP judges to be effective, such as drug courts, as well as interventions shown to be ineffective, such as Scared Straight and juvenile intensive probation supervision. The studies use meta-analytic methods to combine findings from different evaluations of the same intervention to derive the effects on crime outcomes used in the BCAs (essentially, term one of the equation presented earlier). They use rigorous methods to project the reductions in crime that an intervention is likely to produce over a 13-year follow-up period.
They then use the projections to estimate the resulting cost savings for the criminal justice system and victims. The projected reductions in crime and the criminal justice system cost savings are meticulously derived from Washington State data. Victim costs are taken from Miller and colleagues (1996). Finally, WSIPP analysts are transparent in describing their assumptions and methods.
The top of Table A-1 presents the BCA findings for the juvenile justice programs analyzed by Drake and colleagues (2009). The message is clear: whether one chooses to intervene with juvenile offenders when they are institutionalized, in group or foster homes, or on probation, states and localities can adopt programs that produce remarkably large economic returns.22 The same is true for programs that seek to divert juveniles before they are convicted of further crimes.
For institutionalized juveniles, the benefits of aggression replacement therapy, functional family therapy, and family integrated services respectively exceed their costs by roughly $65,500, $57,300, and $16,000, respectively. For the small group of juvenile sex offenders, sex offender treatment yields large benefits that exceed the high treatment cost by nearly $25,000 per participant. Boot camp programs do not reduce crime, but because they cost less than placing offenders in an institution, they also pass a benefit-cost test when the baseline is institutionalization.
For juvenile offenders in group or foster homes, multidimensional treatment foster care’s benefits exceed its costs by $33,000.
For juveniles on probation, $1,467 spent to provide aggression replacement therapy returns nearly $36,000 in benefits. Functional family therapy and multisystemic therapy for such juveniles easily pass a benefit-cost test. A recent BCA of a multisystemic therapy program in Missouri also demonstrated large economic returns (Klietz et al., 2010).
Six program models intended to divert juveniles from further offending have benefits that substantially exceed costs. For adolescent diversion (for lower risk offenders), teen courts, restorative justice, and coordination of services, the benefit-cost ratio ranges from 10.6 to 25.6. Drug courts and victim offender mediation yield benefit-cost ratios of 4.2 and 6.9, respectively.
It is important to recognize that some programs are economically inferior to conventional practice (i.e., the benefits are smaller than the costs). This is the case for both alternative parole programs, wilderness challenge, intensive probation supervision, and Scared Straight.
Parole is the only custody status for which no alternative programs pass a benefit-cost test. There may be parole practices that are economically better than standard practice, but they have not yet been developed
____________________________________
22 Greenwood (2008) reports similar results in his review of programs for juvenile offenders.
TABLE A-1 Benefits and Costs per Participant of Juvenile Offender Programs (net present value, 2010 dollars)
| Custody Status of Juvenile Participantsa | Program | Benefits to Victims and Criminal Justice System | Program Costs (compared with cost of alternative) | Benefits Minus Costs | Benefit-Cost Ratio | Probability That Benefits Exceed Costs |
| Institution | Aggression replacement therapyc | $66,954 | $1,473 | $65,481 | 45.5 | .93 |
| Functional family therapyc | 60,639 | 3,198 | 57,341 | 19.0 | .99 | |
| Family integrated transitionsc | 27,020 | 10,968 | 16,052 | 2.5 | .86 | |
| Sex offender treatmentb | 60,477 | 35,592 | 24,885 | 1.7 | n/a | |
| Boot campb | 0 | -8,661 | 8,661 | n/a | n/a | |
| Wilderness challengeb | 0 | 3,350 | -3,350 | n/a | n/a | |
| Group or Foster Home | Multidimensional treatment foster carec | 40,787 | 7,739 | 33,047 | 5.3 | n/a |
| Parole | Regular surveillance-oriented parole (versus no parole supervision)b | 0 | 1,301 | -1,301 | n/a | n/a |
| Intensive parole supervisionb | 0 | 7,015 | -7,015 | n/a | n/a | |
| Probation | Aggression replacement therapyc | 36,043 | 1,476 | 34,566 | 24.4 | .93 |
| Functional family therapyc | 37,739 | 3,190 | 34,549 | 11.9 | .99 | |
| Multisystemic therapyc | 29,302 | 7,206 | 22,096 | 4.1 | .91 | |
| Intensive probation supervisionb | 0 | 1,735 | -1,735 | n/a | n/a | |
| Diversion | Adolescent diversion project (for low-risk offenders)b | 53,072 | 2,077 | 50,995 | 25.6 | n/a |
| Teen courtsb | 17,782 | 985 | 16,797 | 18.0 | n/a | |
| Drug courtsc | 12,737 | 3,024 | 9,713 | 4.2 | .80 | |
| Restorative justiceb | 10,106 | 954 | 9,152 | 10.6 | n/a | |
| Coordination of servicesc | 5,270 | 386 | 4,884 | 13.6 | .78 | |
| Victim offender mediationc | 3,922 | 566 | 3,357 | 6.9 | .90 | |
| Scared Straightc | -6,031 | 63 | -6,095 | n/a | n/a | |
| Benefits (Costs not available) | ||||||
| Institution | Behavior modificationb | 42,706 | ||||
| Cognitive-behavioral therapyb | 7,744 | |||||
| Counseling, psychotherapyb | 50,304 | |||||
| Education programsb | 109,834 | |||||
| Life skills education programsb | 13,908 | |||||
| Diversion | Diversion with services (versus regular juvenile court)b | 3,982 | ||||
| Other family-based therapy programsb | 40,281 | |||||
NOTE: n/a = estimate not available. Monetary figures are converted into 2010 dollars.
a Adapted from Greenwood (2008).
b Adapted from Drake, Aos, and Miller (2009).
c Adapted from Washington State Institute for Public Policy (2011).
and successfully tested. Juvenile justice officials may consider supporting the development and testing of new parole models that might prove successful and pass a benefit-cost test. Alternatively, they can use their scarce resources to implement the already proven programs that intervene during a different custody status.
The results in columns 3 and 4 of Table A-1 show the best point estimates of benefits and costs. However, bottom-line estimates of total benefits and costs have a degree of uncertainty, because estimates of some of the underlying parameters needed to conduct a BCA are themselves uncertain.23 WSIPP’s (2011) recent analyses take this uncertainty into account using Monte Carlo methods, which provide an estimate of the probability that benefits will exceed costs when parameters values are systematically varied.24
The Monte Carlo results in the last column of Table A-1 imply that one can be highly confident that aggression replacement therapy, family integrated transitions, functional family therapy, multisystemic therapy, and victim offender mediation are successful programs from a benefit-cost perspective. The probabilities that these approaches pass a benefit-cost test are all at least .86, and most exceed .90. The probabilities are somewhat lower for drug courts and coordination of services (.80 and .78), but one can still be quite confident that both are successful.
Because WSIPP uses Washington data to estimate changes in crime and the costs of the criminal justice system, the findings technically are not generalizable to other states or the nation as a whole. However, Washington’s crime and the costs of its criminal justice system probably do not differ substantially from other states, so the findings are likely to apply elsewhere. Indeed, even if the savings in criminal justice costs and the benefits to victims (not shown separately in Table A-1) were both 25 percent smaller, all programs that pass a benefit-cost test in the WSIPP analysis would still pass by a wide margin. The WSIPP findings provide reliable guidance for other states and localities.
Seven other types of programs examined by Drake and colleagues (2009) also generate benefits to victims and the criminal justice system, as shown in the lower panel of Table A-1. Four of the seven have benefits
____________________________________
23 Suppose an evaluation reports that a program reduced crime by 12 percent, with a standard error of 1.4. This means that, although the most likely impact is 12 percent, there is a 95 percent chance that the true impact lies between 9.3 and 14.7 percent. Similarly, estimates of program costs, estimates of victim costs, and the methods used by Drake and colleagues (2009) to combine findings from several studies are not perfectly precise.
24 The Monte Carlo method simultaneously varies the specific parameters used to compute benefits and costs based on the degree of uncertainty of all parameters’ estimates. Repeating the computations under thousands of variations tests the sensitivity of the overall findings to the inherent uncertainly of the underlying parameters.
exceeding $40,000 per participant, so they are likely to pass a benefit-cost test. But because WSIPP had not computed cost estimates at the time of publication, we cannot draw this conclusion with certainty.
Building on its methodological innovations, WSIPP is currently developing a tool that other jurisdictions can use to derive benefit-cost estimates of criminal justice programs (Aos and Drake, 2010). The tool will allow analysts to use crime and cost data for their jurisdictions and vary the assumptions needed to compute cost savings.
A NOTE ON COST-EFFECTIVENESS ANALYSIS
If the principal benefit expected from a juvenile justice program cannot be given monetary value, cost-effectiveness analysis can be an alternative to benefit-cost analysis (Boardman et al., 2011). Suppose, for example, that the primary goal is to reduce recidivism and that other possible program impacts are of little import to decision makers. In such a case, programs might be compared in terms of the decrease in the average probability of recidivism per dollar spent to assist a first-time juvenile offender. The most cost-effective program is the one that produces the largest decrease in the probability per dollar spent.
Focusing on one goal is a strength in that it obviates the need to express the value of the outcome in monetary terms. Yet when interventions have multiple goals and no single one has clear priority, cost-effectiveness data may offer little guidance (Marsh, 2010). Suppose intervention A decreases a juvenile offender’s likelihood of nonviolent crime by 20 percent and of violent crime by 10 percent. Alternative intervention B is equally costly and has corresponding decreases of 30 percent and 8 percent. Although B performs slightly worse in reducing violent crime, it does much better in reducing nonviolent crime. Which intervention is better? When there are multiple types of benefits, none of which dominates, and when the most important of them can be cast in monetary terms, a BCA will usually provide more useful information than a cost-effectiveness analysis.
LIMITATIONS OF CURRENT BENEFIT-COST ANALYSES
The program cost estimates in Table A-1 are essentially complete, yet all benefit estimates are understated for several reasons. These shortcomings apply to all other BCAs of juvenile justice programs as well. First, they ignore possible benefits to nonvictims and to offenders and their families. The latter especially could be large if the programs help offenders to attain more schooling or if desistance lowers the likelihood that younger siblings
engage in delinquent acts.25 Second, they count the savings of less crime for the criminal justice system, but not for other public agencies or nonprofit organizations that may also have savings (e.g., less money needed to repair vandalism). Third, as noted earlier, methods for measuring some types of victim costs have not yet been developed.26 Finally, because adolescent behavior, including delinquency, is heavily influenced by peers, programs that reduce a participant’s delinquency may reduce that of his or her peers as well. Because program evaluations have not measured this “second round” impact on crime, BCAs cannot include its benefits.27
Recognizing these reasons why benefits are understated further strengthens our earlier conclusion: states and localities can choose from a portfolio of programs for juvenile offenders that, if implemented well, can reduce crime and produce extraordinarily large economic returns.
As noted above, bottom-line estimates of total benefits and costs have a degree of uncertainty. Most existing BCAs of juvenile justice programs do not take this uncertainty into account and hence may give a misleading impression of the confidence one can place on the reported point estimates of benefits and costs. The recent analyses of the WSIPP (2011) do take account of uncertainty and mark a significant step forward.
IMPROVING BENEFIT-COST METHODS FOR JUVENILE JUSTICE PROGRAMS
The leading BCAs of juvenile justice programs employ sophisticated estimation methods and complex data sets and provide valuable information to stakeholders in the juvenile justice system. Like other evaluation methods, however, the state of the art can be improved.28
Some categories of bottom-up costs have not been measured at all, and measurement methods for others have limitations (Cohen, 2010). The potential of the contingent valuation method for deriving improved esti-
____________________________________
25 Suppose a program raises the probability of completing high school by .10. Since in 2009 male (female) high school graduates earned $11,600 ($8,900) more per year than those without a degree (U.S. Census Bureau, 2010a, 2010b), the average increase in earnings would be $1,160 or $890. Over a 40-year working life, the present value of $1,160 and $890, making the conservative assumption that it does not grow over time and using a discount rate of 5 percent, is $20,900 and $16,000.
26 Some other studies are further limited because they estimate cost savings to the criminal justice system but not victim benefits (Robertson, Grimes, and Rogers, 2001; Cowell, Lattimore, and Krebs, 2010).
27 Butts and Roman (2009) observe that some potentially valuable program models, such as community-based interventions, lack the rigorous evaluations required to assess benefits and costs. This is less a limitation of the technique of BCA per se than of the funding priorities of agencies and researchers.
28 See Butts and Roman (2009) for other suggestions.
mates of the benefits of crime reduction has just begun to be explored. A program of research to improve bottom-up measures, refine and apply CV methods, and compare the results of these two approaches is in order.
As observed above, Monte Carlo analysis offers a way to address the uncertainty of estimates of benefits and costs. Monte Carlo analysis strengthens the credibility of a study’s findings and needs to be a routine element of future BCAs. The WSIPP model under development (Aos and Drake, 2010), which will have the capacity to execute Monte Carlo simulations, will substantially improve BCAs of criminal justice programs.
Finally, because evaluations of juvenile justice interventions understandably focus on changes in criminal behavior, they generally do not collect data on changes in noncriminal behavior, such as education or employment attributable to an intervention. Because these changes typically have positive benefits, ignoring them underestimates an intervention’s total benefits. A more complete BCA would quantify the economic value of changes in these outcomes as well as in ones directly related to crime. Future impact evaluations and BCAs of juvenile justice programs would be stronger if they routinely measured changes in important noncrime outcomes, quantified their economic value whenever possible, and included them in the benefit-cost calculations in addition to the benefits directly related to crime.
