Criminal Careers and "Career Criminals," Volume II (1986) / Chapter Skim
Currently Skimming:
6. Accuracy of Prediction Models
6. Accuracy of Prediction Models
Pages 212-290
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 212... ...
First, our principal focus is the prediction of criminal or delinquent behavior. Thus, we do not address a variety of important criminal justice prediction problems involving resource allocation, criminal population projections, estimation of rates of offending and the length of criminal careers, and many others, except as they are relevant to assessing the impacts of some proposed decision-making devices (e.g., those proposed for selective incapacitation strategies)
|
From page 213... ...
Next, for each of the decision arenas considered, we examine the efficacy of statistically cleveloped decision-making tools that are in use, or have been proposed for use, in a number of jurisdictions. Finally, we cTiscuss ways to improve the accuracy and hence the utility of prediction tools clesignecI for application in criminal justice settings.
|
From page 214... ...
CRIMINAL CAREERS AND CAREER CRIMINALS discussion of rationality in decision making, see Lee, 1971~. Following Lee, decision theory considers the rational person to be one who, when confronted with choice, makes the (recision that is "best"; this decision is the optimal or rational one.
|
From page 215... ...
used to combine items of information, the reliability ofthe criterion variable chosen, the kinds of measurements usecI, the base rate, the selection ratio used, and the representativeness of samples employed. Two questions should be acIdressec3: one considers the accuracy of inclivi(lual items of information; the other refers to the accuracy of items in combination with one another.
|
From page 216... ...
In many criminal justice applications, which traditionally have treater] criterion measures as dichotomous, the base rate is found simply through inspection of the appropriate marginal distribution of the expectancy table.
|
From page 217... ...
Those concerned with the clevelopment of predictive tools for use in criminal justice applications (and in other areas) often have failed to consider base rates in the development process ancI, consequently, have made classifications or predictions based on criteria that procluce larger errors than would the simple use of the base rate.
|
From page 218... ...
Horst (1966) refers to this general procedure as the "sample fractionation" approach and argues, quite correctly, that CRIMINAL CAREERS AND CAREER CRIMINALS there are serious clisadvantages to it.
|
From page 219... ...
In selection applications, predictive crevices reduce to a dichotomy resulting in a decision situation, with actual outcomes considered, that can be representec3 by a 2 x 2 contingency table (Figure 1~. The cutting score clecicled on determines the selection ratio ant!
|
From page 220... ...
As we have noted, most selection applications of predictive crevices use some cutting score, essentially reducing the predictor scale to a dichotomy. As commonly used, however, the standard error of prediction assesses the predictive device and the criterion measured continuously and may, in fact, result in an underestimation of the power of the selection crevice, since the device as used simply is predictive of success or failure.
|
From page 221... ...
describe the performance of the instrument in application with given populations and decision rules; the latter (such as the MCR) essentially give an inclication of the general power of the device without respect to constraints of base rates and selection ratios.
|
From page 222... ...
to belong to criterion classification A in fact will not (false positives) , and some persons preclicted to belong to criterion classification B in fact will not (false negatives)
|
From page 223... ...
, determining the expected utility of predictive devices based on a differential weighting of errors is common, although not in justice system settings. RESEARCH EVIDENCE: THE POWER OF PREDICTION Bail and Pretrial Release Decision/Prediction Studies A number of prediction studies concerning bad!
|
From page 224... ...
Information concerning the reliability of the assessments is not given. CRIMINAL CAREERS AND CAREER CRIMINALS its nearest competitor (which has to do with the seriousness of the clefendant's prior record)
|
From page 225... ...
Wice's (1978) large study of some 11,000 pretrial releasees in Washington, D.C., demonstrated that charge seriousness and prior record were significantly relatecl to judges' pretrial release decisions, but that race, sex, age, employment status, and residence were not.
|
From page 226... ...
found that only 5 per cent of those released in Los An~eles CRIMINAL CAREERS AND CAREER CRIMINALS Normative Studies ~ ~ =7 ~ ~ 1~ appear, felony convictions, and charge (six categories)
|
From page 227... ...
. Multivariate analyses suggested that all 26 variables consicIered together accounted for only 13 percent of the outcome variance.~7 Although the equation results in prediction that is modestly above the base rate (indeed, as Cureton, 1957, has clemonstratect, any vaTicI continuous predictor, properly used, must provide advantage over the base rate)
|
From page 228... ...
Type of CRIMINAL CAREERS AND CAREER CRIMINALS offense, age, prior FTAs, pending charges, recent arrests, and the interaclions of some of these (e.g., over age 44 x prior FTAs) were important in terms of impact on expected Tog odds of the combined inclex.
|
From page 229... ...
ACCURACY OF PREDICTION MODELS the manner in which prosecutors appear to use information in their decision making. As noted by M
|
From page 230... ...
This conclusion may be questioned, how i9These are approximate values, calculated by us form summary tables reported in the article. CRIMINAL CAREERS AND CAREER CRIMINALS ever, because prior record was included in the "strength of evidence" variable.
|
From page 231... ...
Ideally, decision makers would agree not only on the goal for the selection decision, but also on the criteria on which the decision will be based. One has but to review the literature cited above to realize quickly that no such agreement exists.
|
From page 232... ...
22The judges had decided on the purposes to be studied. CRIMINAL CAREERS AND CAREER CRIMINALS Regardless of the actual proportion of cases for which an incapacitative intent is primary, it is clear that judges can rather easily apportion a sanction in terms of its compound intents.
|
From page 233... ...
Parole Prediction-Decision Studies As we have noted, prediction studies involving criminal populations or relating in some way to concerns of the criminal
|
From page 234... ...
compiled a bibliography of such studies that contains more than 600 envies. CRIMINAL CAREERS AND CAREER CRIMINALS now used by the Board are, the Board would not improve matters by consiclering any ofthe sixty-ocld pieces of information placecI at its disposal, which it now ignores" (Warner: 196~.
|
From page 235... ...
Eglit, 1973) surveyed parole board members for opinions of "the general worth" of a variety of orisoner characteristics for "predicting the success of a man on parole," and compared those opinions with the ranges of actual success rates of parolees showing these characteristics (relative to the base rate)
|
From page 236... ...
Complete data were available for only CRIMINAL CAREERS AND CAREER CRIMINALS 310 offenders, but the function correctly iclentified 79 percent of them. Adapting Wilkins' "information board" memos (Wilkins ancL Chandler, 1965)
|
From page 237... ...
tape-recording parole decision makers as they "thought out loud" about the cases being reviewed. Attributional statements represented the single largest category of statements made (beyond the factual information reacI)
|
From page 238... ...
. In Norm Carolina the following corre CRIMINAL CAREERS AND CAREER CRIMINALS lates (point biserial coefficients)
|
From page 239... ...
Klassen (1982~. Past Criminal Behavior.
|
From page 240... ...
Because few studies CRIMINAL CAREERS AND CAREER CRIMINALS have used common criteria or definitions, it is difficult to provide an adequate summary of the relation between past and future criminal behavior; this difficulty is exacerbated by the fact that samples also have varied. Finally, a wide variety of methods have been used to examine these relations, and they often are not readily comparable.
|
From page 241... ...
criminal behavior, and here, the evidence is compelling: the earlier the onset of criminal activity, the poorer the prognosis. Kirby (1954)
|
From page 242... ...
, who found CRIMINAL CAREERS AND CAREER CRIMINALS that sex remained statistically significant in a multiple cliscriminant function analysis. The variable's unique contribution, however, is very slight (see p.
|
From page 243... ...
The zeroorder relations are modest (correlation coefficients of .21, .12, .17 to .14, .17, and .13 to .16 have been reported by Borden, 1928; Vold, 1931; Kirby, 1954; Simon, 1971; ant!
|
From page 244... ...
A measure of school conduct, however, was modestly correlated win recidivism. CRIMINAL CAREERS AND CAREER CRIMINALS Kirby, 1954; Mannheim and Wilkins, 1955; S
|
From page 245... ...
bNot deemed useful for most practical applications of prediction tools. CThe most powerful predictors appear to be seriousness and prior record, regardless of the particular criterion used (e.g., sentence type, sentence length, measures of sentence "severity")
|
From page 246... ...
Accor(lingly, with the exception of the first two or three entries in each cell, we CRIMINAL CAREERS AND CAREER CRIMINaLS do not have conficlence in the relative ordering of predictive factors listed. These caveats made, the table rather clearly shows our original impression to have been more or less correct.
|
From page 247... ...
, they often do not consider base rates (Meeh! and, Rosen, 1955)
|
From page 248... ...
CRIMINAL CAREERS AND CAREER CRIMINALS this appears to be true regardless of the decision arena investigated. This "simple" response is unsatisfactory, however.
|
From page 249... ...
The ideal standard of one who advocates a just deserts perspective is radically different from that advocatecl by proponents of"selective incapacitation"; succinct reviews and summaries of these differences can 249 be found in a recent "debate" between Greenwood ant] von Hirsch (NJ]
|
From page 250... ...
An arbitrary weighting scheme was used, which resulted in a total "risk" score, according to which rec CRIMINAL CAREERS AND CAREER CRIMINALS ommendations were made concerning release. Considerable success was claimed for this and related projects.
|
From page 251... ...
In at least these two ways, the jllcige~' choice of models constrainec! the likely predictive accuracy of the guidelines implemented seriousness of charge was to receive approximately equal weight as considerations of risk, and the prediction model chosen, based on an outcome measure that reflects two
|
From page 252... ...
Cases, stratified by six charge-seriousness cate · 1 _ 1 _ ~ _ CRIMINAL CAREERS AND CAREER CRIMINALS may be predictive in nature (e.g., associated with desired offender outcomes) , but they may be of another nature (e.g., of ensuring just deserts or of increasing equity)
|
From page 253... ...
ACCURACY OF PREDICTION MODELS that recommender! by the device (that is, a decision "outside the guidelines" may reflect compliance with the general model)
|
From page 254... ...
Despite the demonstration that guideline-structured decision making differed in important respects from unguided clecision making, GolUkamp ant! Gottfied CRIMINAL CAREERS AND CAREER CRIMINALS son founct lithe cli~erential effect (on the amount of bait set)
|
From page 255... ...
Cohen, 1983a; von Hirsch and GottfrecIson, 19841; the former requires estimates of average in~liviclual arrest and crime rates, as well as estimates of the average length of criminal careers. Although we c30 not address the ethical arguments in this paper, it should be noted that although the J
|
From page 256... ...
I Cohen also compared the "accuracy" of the scale relative to current practice, as implied by sentence lengths given, and found that "the seven-point scale floes only marginally better overall and results CRIMINAL CAREERS AND CAREER CRIMINALS in slightly more false-positives than existing subjective judgments in clistinguishing offenders by their crime commission rates" (p.
|
From page 257... ...
, we have provided a "partisan review" of these critiques, and we invite attention to the issues we raise there. Here, we concentrate on the adequacy, in terms of predictive accuracy, of"prescriptive" sentencing guidelines.
|
From page 258... ...
These have to |
From page 259... ...
Neither (limension ofthe grid usecI, however, was intended to be preclictive; such an intent was explicitly excluded by the Minnesota Sentencing Guidelines Commission (1982~. It is the case that one of the axes (the "criminal history score")
|
From page 260... ...
The Goodell Committee (von Hirsch, 1976) specifically rejected any predictive basis for their sentencing determination; but, of course, the fact that they wished to take into account the prior record of the offender, in fact, provided a predictive dimension." CRIMINAL CAREERS AND CAREER CRIMINALS make evaluative statements (although without first ensuring that some innovation had been seriously implemented.
|
From page 261... ...
(1978~. The Salient Factor Score Parole guiclelines were developed in the early 1970s for consideration by the U.S.
|
From page 262... ...
. CRIMINAL CAREERS AND CAREER CRIMINALS lapsed from a O to 11 scale to a 4-category scale.
|
From page 263... ...
The extent to which these effects constrain one another has not been adequately investigated to ciate. The Iowa Instrument In light of claims made for dramatic improvements in the accuracy with which offender risk assessments may be 263 planning and research documents macle available by the Panel on Research on Criminal Careers.
|
From page 264... ...
. CRIMINAL CAREERS AND CAREER CRIMINALS (both on ethical and legal grounds; see Underwood, 1979; von Hirsch and GottFredson, 1984)
|
From page 265... ...
. The development of this measure is not detailed in reports available; we do not know if the scaling is arbitrary, but it appears to be (see Statistical Analysis Center, 1980:21.
|
From page 266... ...
However, by providing the Salient Factor Score (in the examples used) the "logical" act vantage provided the Iowa classification scheme, we calculate an CRIMINAL CAREERS AND CAREER CRIMINALS MCR of .711 (for the construction sample)
|
From page 267... ...
, which, within hancI-calculator rounding error, of course gives the variance of the original scores (566.411. The Iowa investigators do not do this; they unencocle the cocled-score variance by dividing by the square of the transforrned base rate and obtain the value reported (.807~.
|
From page 268... ...
, we find these coefficients reported for a rearrest and a programfailure criterion, based on the sample of 12,517. CRIMINAL CAREERS AND CAREER CRIMINALS and 19.3 percent for the revocation/ absconder criterion and the rearrest criterion, respectively.
|
From page 269... ...
. We do not believe that the comparisons of the utility of the Iowa model and several others (e.g., INSLAW, Rand, Salient Factor Score, Michigan)
|
From page 270... ...
Although predicative power is low, the same level of power is observed in several validation samples; the relation observer] CRIMINAL CAREERS AND CAREER CRIMINALS apparently is stable.
|
From page 271... ...
has shown, any valid continuous predictor can improve on the base rate, and, as we have observed, there appear to be several of these relative to the criteria consiclere(1 in this paper. Valiclities are Tow, but equations and devices cliscussed CO provide advantage over base-rate prediction.
|
From page 272... ...
It must be recognized, however, that consideration of these may CRIMINAL CAREERS AND CAREER CRIMINALS work at cross-purposes relative to the risk . c Dimension.
|
From page 273... ...
Statistical Bootstrapping As described earlier, moclels such as that apparently developed in Iowa potentially couIc3 do much to increase the utility of prediction in criminal justice settings. The basic procedure simply wouIct require the identification of relatively homogeneous subgroups of offenders, the construction of statistical prediction equations for each, and the combination of these into an "expectancy table" for the full sample.
|
From page 274... ...
that pre(liction may be improved through a combined use of methods. An iterative bootstrapping process in which successive normative anct descriptive devices CRIMINAL CAREERS AND CA0ER CRIMINALS are used to inform and modify each other may well prove productive.
|
From page 275... ...
Other arguments against the use of statistically based prediction tools all recluce to considerations of their accuracy. The technically sophisticated arguments focus directly on the accuracy issue and cite Tow proportions of variance explainecI and resulting high error rates (focusing usually on false positives; false negatives may be equally, or even more, un(lesirable depending on the application)
|
From page 276... ...
Usually one does not select based on the mean score, nor does one observe False Negatives Positive / | Hits Negative Hits False Positives xc PREDICTED BEHAVIOR FIGURE 2 Hypothetical prediction-based selection decision problem.
|
From page 277... ...
. 277 Figure 3 is basecl on these assumptions: as shown, false positives are reduced at the expense of false negatives.
|
From page 278... ...
lust as before, one can manipulate tile trade-off of false positives and false negatives by moving Xc to the left or the right. For a given Yc' the value of Xc chosen will determine whether more false positive or false negative errors will be made.
|
From page 279... ...
Note also that the selective Inca- changes. Selective incapacitation sugpacitation concept apparently seeks to gests clearly that there is a proper purminimize false negatives (that is, failure pose for the sentencing of criminal ofto select those who in fact pose a substan- fenclers: removing them from normal tial risk of continued criminal behavior)
|
From page 280... ...
If society should clecide that selective incapacitation is the appropriate strategy for sentencing criminal offenders, it is clear that prediction tools should be used in the decision-making process. To decide the policy question on the basis of current predictive accuracy, however, would be foolish.
|
From page 281... ...
Journal of Personality and Social Psychology 36:1501-1511. 1978b Causal theories of crime and their effect upon expert parole decisions.
|
From page 282... ...
Belmont, Calif.: Brooks/Cole. CRIMINAL CAREERS AND CAREER CRIMINALS Ebbesen, E., and Konecni, K
|
From page 283... ...
No Selective Incapacitation of Potentially Vio date lent Adult Offenders. Statistical Analysis Center, Iowa Office for Planning and Programming, 523 E
|
From page 284... ...
Center for studies of Crime and Delinquency. CRIMINAL CAREERS AND CAREER CRIMINALS Rockville, Md.: National Institute of Mental Health.
|
From page 285... ...
Journal of Criminal Law and Cr~minology 64~2~: 183-189. Harris, M
|
From page 286... ...
Washington, D.C.: Bureau of Social Science Research. CRIMINAL CAREERS AND CAREER CRIMINALS Janus, M
|
From page 287... ...
New York: John Wiley & Sons. Minnesota Sentencing Guidelines Commission 1982 Preliminary Report on the Development and Impact of the Minnesota Sentencing Guidelines.
|
From page 288... ...
1965 Utilities as base rate multipliers in the determination of optimum cutting scores for the discrimination of groups of unequal size and variance. Journal of Applied Psychology 50:364~68.
|
From page 289... ...
F 1954 Parole prediction: its history and status.Journal of Criminal Law and Criminology 45(November)
|
From page 290... ...
CRIMINAL CAREERS AND CAREER CRIMINALS Wiggins, J 1973 Personality and Prediction: Principles of Personality Assessment.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.