Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 101
6 Challenges to Identifying Deterrent Effects R esearchers from diverse disciplines have contributed to the capital punishment literature, with prominent contributions by economists, criminologists, and sociologists. Although researchers’ disciplinary backgrounds have affected the methods used and the framing of the re- search questions, the failings of the capital punishment literature are not rooted in the use of particular empirical methods or theoretical models of criminal decision making. Rather, the failings are rooted in manifest deficiencies related to the research data and methods and the researchers’ interpretations of results. Chapters 4 and 5 call attention, respectively, to fundamental deficiencies in panel and time-series studies. Both approaches share two basic deficiencies and also manifest two others to some degree. One shared deficiency is grossly incomplete specification of the sanction re- gime for homicide. Even in states that make the most frequent use of capital sanctions, noncapital sanctions are the most common sanction imposed for a homicide conviction. No study of either type accounts for the noncapi- tal component of the sanction regime in states with and without capital punishment. The second basic deficiency is failure to pose a credible model of the sanction risk perceptions of potential murderers and the behavioral response to such perceptions. In the absence of such a model, it is difficult, at best, to interpret data relating sanction regimes to homicide rates. As discussed in Chapters 4 and 5, these two deficiencies are sufficient to make existing studies uninformative about the effect of capital punish- ment on homicide. Both of these deficiencies are potentially correctable. However, even if the research and data collection initiatives discussed in this chapter are ultimately successful, research in both literatures share a 101
OCR for page 102
102 DETERRENCE AND THE DEATH PENALTY common characteristic of invoking strong, often unverifiable, assumptions in order to provide point estimates of the effect of capital punishment on homicides. A point estimate may offer the appearance of desirable certitude, but only at a high cost in credibility. Still another deficiency is inattention to potential feedbacks through which homicide rates, and crime rates more generally, may affect the specification and administration of a sanction regime while the regime simultaneously affects homicide rates. Recogni- tion of potential feedbacks is relevant both to identify the direct effect of capital punishment on homicide rates and to predict the ultimate effect after feedbacks occur. Feedbacks affect the time-series and panel studies differ- ently because of differences in the time frames of the data typically used in the two approaches—monthly, weekly, or even daily data in the time-series studies and annual data in the panel studies. In light of these deficiencies, the committee has reached the following conclusion and recommendation: CONCLUSION AND RECOMMENDATION: The committee con- cludes that research to date on the effect of capital punishment on homi- cide is not informative about whether capital punishment decreases, increases, or has no effect on homicide rates. Therefore, the committee recommends that these studies not be used to inform deliberations re- quiring judgments about the effect of the death penalty on homicide. Consequently, claims that research demonstrates that capital punish- ment decreases or increases the homicide rate by a specified amount or has no effect on the homicide rate should not influence policy judgments about capital punishment. The committee was disappointed to reach the conclusion that research conducted in the 30 years since the National Research Council (1978) report on this subject has not sufficiently advanced knowledge to allow a conclusion, however qualified, about the effect of the death penalty on homicide rates. Yet this is our conclusion. Some studies play the useful role, either intentionally or not, of demonstrating the fragility of their claims to have found—or not to have found—deterrent effects. However, even these studies suffer from two intrinsic shortcomings that severely limit what can be learned from them about the effect of the death penalty on homicide rates from an examination of the death penalty as it has actually been ad- ministered in the United States in the past 35 years. Commentary on research findings often pits studies claiming to find statistically significant deterrent effects against those finding no statistically significant effects, with the latter studies sometimes interpreted as imply- ing that there is no deterrent effect. A fundamental point of logic about hypothesis testing is that failure to reject a null hypothesis does not imply
OCR for page 103
103 CHALLENGES TO IDENTIFYING DETERRENT EFFECTS that the null hypothesis is correct. For the evidence of even a small effect to be credible, it requires a demonstration, first and foremost, that the effect is based on a sound research design. Estimates that lack credibility are not in- formative regardless of the consistency of their estimated size. The amount of the effect must also be small in size and estimated with good precision, for example, by being contained within a tight confidence interval. Our mandate was not to assess whether competing hypotheses about the existence of marginal deterrence from capital punishment are plausible, but simply to assess whether the empirical studies that we have reviewed provide scientifically valid evidence. In its deliberations and in this report, the committee has made a concerted effort not to approach this question with a prior assumption about deterrence. Having reviewed the research that purports to provide useful evidence for or against the hypothesis that the death penalty affects homicide rates, we conclude that it does not pro- vide such evidence. We stress, however, as noted above, that a lack of evidence is not evi- dence for or against the hypothesis. Hence, the committee does not construe its conclusion that the existing studies are uninformative as favoring one side or the other side in the long-standing societal debate about deterrence and the death penalty. In this chapter, we elaborate on these deficiencies that form the basis for this conclusion and cautiously offer some ideas on potential remedies. With regard to remedies, our report provides a somewhat less pessimistic perspective than did the earlier National Research Council (1978, p. 63) report: “[T]he Panel considers that research on this topic is not likely to produce findings that will or should have much influence on policymakers.” The committee does not expect that advances in collecting data on sanction regimes and obtaining knowledge of sanctions risk perceptions will come quickly or easily. However, data collection on the noncapital component of the sanction regime need not be entirely complete to be use- ful. And even if research on perceptions of the risk of capital punishment cannot resolve all major issues, some progress would be an important step forward. Even if these advances prove unsuccessful in providing useful information on the incremental deterrent effect of capital punishment in relation to a lengthy prison sentence, the committee believes that there are potentially major benefits from new data collection, theory, and methodol- ogy for study of the effect of noncapital sanctions on crimes not subject to the death penalty. As discussed in Chapter 1, because of the overlap in the methods and data used in studies of capital punishment and in broader studies on the effects of sanctions on crime, our charge included a provi- sion for recommending research that might advance that broader research literature, and we do so in the rest of this chapter.
OCR for page 104
104 DETERRENCE AND THE DEATH PENALTY DATA ON SANCTION REGIMES Incomplete and inaccurate data have marred research on the effect of capital punishment on homicides. The most important data problem is that studies have been based on a very incomplete specification of state sanction regimes. Part of the difficulty has been lack of conceptual agreement on how to measure the intensity of use of capital punishment. However, we see the primary problem as a complete absence of data on the noncapital sanctions that might be applied to offenders convicted of homicide. A study of capital punishment in North Carolina by Cook (2009) illustrates the im- portance of the problem of the absence of information on noncapital sanc- tions. Of 274 cases prosecuted as capital cases, only 11 resulted in a death sentence. Another 42 resulted in dismissal or a verdict of not guilty, which left 221 cases that resulted in convictions and received noncapital sanctions. As discussed at length in Chapter 4 and below, there are sound reasons for predicting a correlation between the capital and noncapital components of a state’s sanction regime. Two examples of how this might occur are the plea bargaining leverage that the threat of capital punishment may afford prosecutors and the influence of the state’s political culture on the legislated design and administration of both the capital and noncapital components of the regime. Such a correlation would bias the estimated deterrent effect of capital punishment. None of the studies we reviewed sought to measure the availability and intensity of use of the noncapital sanction alternatives for the punish- ment of homicide. Such alternatives may include a life sentence without the possibility of parole, a life sentence with the possibility of parole, and sentences of less than life. It would also be important to have data on the time actually served for convicted murderers who are paroled or who serve less than a life sentence. It is currently not possible to measure noncapital sanction alternatives at the state level because the required data are not available. The data that are available include those from the Bureau of Justice Statistics (BJS), which publishes nationwide statistics on sentences for prison admissions and time served for prison releases, based on data collected as part of the National Corrections Reporting Program (NCRP) initiated in the early 1980s. More than 40 states now report annual data on sentences for admissions and time served for releases. Individual-level demographic characteristics are also reported. In principle, these data could be used to measure the actual administration of the legally authorized dimensions of most state sanction regimes, not only for murder but also for other types of crimes. The dif- ficulty is that the data are often extremely incomplete. In some years, states fail to report any data. Just as important, the data that are sent to BJS are often so incomplete that it is impossible to
OCR for page 105
105 CHALLENGES TO IDENTIFYING DETERRENT EFFECTS construct valid state-level measures of the administration of the sanction regime. Indeed, the committee attempted to use these data for the purposes of this report but concluded that the data gaps made their use infeasible. More complete data on the actual administration of sanction regimes might be obtained by expanding the NCRP to include all 50 states and filling the data gaps due to incomplete reporting. Alternatively, an entirely new data collection system might be desirable. Either way, the collection of more complete data on sanction regimes for murder and other crimes is feasible. The data are available: the challenge is designing and implementing an ef- fective system for their collection. Even if data on the actual administration of state sanction regimes were complete, they could only be used to measure how sanction regimes are actually administered. The data do not specify the potential sanction regime in a state—the range of sanction alternatives that are legally authorized. We are not aware of any ongoing effort to assemble data on the legislated sanction regimes of the states for murder and other crimes. Data on the legislated regime are important because they define the range of penalties that can potentially be imposed. Thus, the measurement of legally autho- rized sanctions by the states for homicides and other crimes may require a new data collection system. The committee did not explore the benefits and costs of alternative ap- proaches for measurement of state-level sanction regimes for murder. We only emphasize the vital importance of collecting these data. RECOMMENDATION: The committee recommends that a concerted effort be made to collect data on the sanctions regimes faced by poten- tial murderers, with particular attention to fixing the current absence of data on noncapital sanctions. As noted above, because the methods and data used to study the effect of noncapital sanctions on crimes other than murder are similar to those used in research on capital punishment, the committee’s charge includes a provision that we make recommendations for advancing research on the broad effects of sanctions on crime. Thus, we also stress the vital impor- tance of an expanded effort to collect data suitable not only for measuring sanction regimes for murder, but also for measuring sanction regimes for other major crimes. PERCEPTIONS OF SANCTION RISKS As emphasized in Chapter 3, it is not possible to interpret empirical evidence on the relationship of homicide rates to sanctions without un- derstanding how potential murderers perceive sanction regimes. The com-
OCR for page 106
106 DETERRENCE AND THE DEATH PENALTY mittee’s review of the time-series and panel studies identified fundamental deficiencies in this regard. In the case of the time-series studies, none of them explicitly articulates a model of sanction risk perceptions. The studies are silent on whether execution events and their frequency alter perceptions of sanction regimes. Moreover, the studies do not ask whether the trend lines specified by researchers correspond to the trend line (if any) perceived by potential murderers. Panel studies typically suppose that people who are contemplating mur- der perceive sanctions risks as subjective probabilities of arrest, conviction, and execution. Lacking data on these subjective probabilities, researchers presume that they are somehow based on the observable frequencies of ar- rest, conviction, and execution. The fundamental problem is that perceptions of the risk of sanction are subjective, but researchers have no direct measurements of the percep- tions of potential murderers. In the absence of data on risk perceptions, the research practice in the panel studies has been to use publicly available data on homicides and executions to construct statistics that purport to measure the objective risk of execution. Then, having done that, many researchers assume that potential murderers have “rational expectations.” The word “rational” suggests that potential murderers carefully assess the risk of execution. What “rational expectations” actually means in practice is that researchers construct their own measures of execution risk and as- sume that potential murderers perceive the risk in the same way. However, the assumption of rational expectations of execution risk has no empirical foundation. Indeed, it hardly seems credible. In Chapter 4, we discuss in detail the complications of calculating the objective risk of execution. One of these complications is that only 15 per- cent of individuals sentenced to death have actually been executed (since the resumption of the death penalty in 1976) and that a large fraction of death sentences are subsequently reversed. Another complication is that the volume of data on death sentences and executions available for form- ing perceptions depends on the size of the state. By various measures of execution risk, Delaware was at least as aggressive as Texas in its use of the death penalty. However, over the period 1976 to 2000, Delaware sentenced 28 people to death and carried out 11 executions, while Texas sentenced 753 people to death and carried out 231 executions. Still another complication is that sanction regimes are not stable due to changes in a state’s political leadership, moratoriums on executions, and legal decisions. Yet another complication is that there are within-state differences in the risk of execu- tion due to differences across counties in prosecutorial vigor in the use of the death penalty and local differences in receptivity to its application. These many complications make clear that even with a concerted effort
OCR for page 107
107 CHALLENGES TO IDENTIFYING DETERRENT EFFECTS by careful, conscientious researchers to assemble and analyze relevant data on death sentences and executions, assessment of the evolving objective risk of execution facing a potential murderer is a daunting challenge. It is also clear that perceptions of this risk among potential murderers must at best be highly impressionistic. To make headway on whether and to what degree the death penalty affects the behavior of potential murderers, it is imperative to have knowledge about how their perceptions of execution risk are formed and then possibly revised on the basis of new information. RECOMMENDATION: The committee strongly recommends that a concerted effort be made to research the origins and nature of execu- tion sanctions risk perceptions specifically and of noncapital sanctions risks more broadly. Measurement of Perceptions The essential task is to measure the perceptions of sanctions risks that potential murderers actually hold. How might this be done? One possibility is to take seriously the presumption in the panel stud- ies that people who are contemplating murder perceive sanctions risks as subjective probabilities of arrest, conviction, and execution. This possibil- ity suggests that the risk perceptions of potential murderers be measured probabilistically. Researchers have developed considerable experience measuring beliefs probabilistically in broad population surveys. Manski (2004) reviews the history in several disciplines, describes the emergence of the modern litera- ture, summarizes applications, and discusses open issues. Among the major U.S. platforms for collection of such data, the Health and Retirement Study (HRS) has periodically elicited probabilistic expectations of retirement, bequests, and mortality from multiple cohorts of older Americans (see, e.g., Hurd and McGarry, 1995, 2002; Hurd, Smith, and Zissimopoulos, 2004). The Survey of Economic Expectations (SEE) has asked repeated population cross sections to state the percent chance that they will lose their jobs, have health insurance, or be victims of crime in the year ahead (see, e.g., Dominitz and Manski, 1997; Manski and Straub, 2000). The National Longitudinal Survey of Youth 1997 has periodically asked young people about the chance that they will become a parent, be arrested, or complete schooling (see, e.g., Fischhoff et al., 2000; Lochner, 2007). Ex- amples of victimization and arrest questions include, “What do you think is the percent chance that your home will be burglarized in the next year?” “What do you think is the percent chance that you will be arrested in the next year?” Researchers have learned from these and other surveys that most people have little difficulty, once the concept is introduced, in using
OCR for page 108
108 DETERRENCE AND THE DEATH PENALTY subjective probabilities to express the likelihood they place on future events relevant to their lives. However, success in measuring beliefs probabilistically within the gen- eral public does not imply that survey research could similarly measure the sanction risk perceptions of potential murderers. A major issue when initiating study of this type is to obtain data from the relevant population, in this case, the population of potential murderers. Theoretically, most people who would be legally eligible to be executed (e.g., are not juveniles or of very low intelligence) are also physically capable of committing a murder and thereby are potential murderers. The reality, however, is that the probability of most people committing a murder is so small that as a practical matter it can be treated as zero. Even the probabilities of people committing other serious crimes, such as robbery and burglary, while likely greater, are still extremely small. Thus, when using the term “potential murderer,” one means that part of population with a non-negligible risk of committing murder. Thus, the first step and an important prerequisite for a program of research on sanction risk perceptions is to define the relevant population of potential murderers and, more generally, potential criminals. Such a defini- tion will be required to devise cost-effective sampling strategies for inter- viewing people with nontrivial risks of committing crimes. We expect that one important segment of the relevant population is people with criminal records. The correlation between past and future offending is among the best documented empirical regularities in criminology (National Research Council, 1986; West and Farrington, 1973; Wolfgang, 1958). In the case of murder, for example, Cook, Ludwig, and Braga (2005) found that 43 percent of murderers in Illinois had a felony conviction. Some may question the feasibility of collecting data on the sanction risk perceptions and criminal behavior of individuals with prior histories of seri- ous crimes, especially if subjects are repeatedly interviewed for the purpose of obtaining longitudinal data. Longitudinal data are useful to study how offending experience and external events, such as police crackdowns or policy changes, affect sanction risk perceptions. However, experience dem- onstrates that, with sufficient diligence, it is feasible to collect longitudinal data on highly crime-prone people. A leading example is the Pathways to Desistance Project (Mulvey, 2011), a two-site longitudinal study of desistance from crime among seri- ous adolescent offenders. The project recruited 1,354 adolescents from the Philadelphia and Phoenix juvenile and adult court systems who had been adjudicated as delinquent or found guilty of a serious felony and were 14 to 17 years old at the time that they committed the offense. For the first 4 years of the study, interviews were conducted at 6-month intervals and for the next 3 years the interviews were annual. The retention rate was quite
OCR for page 109
109 CHALLENGES TO IDENTIFYING DETERRENT EFFECTS high, with 87 percent of the subjects interviewed in at least 8 of the 10 interview cycles. Respondents were asked about their perceptions of sanc- tions risks, among other things. The success of this project indicates that collection of data on sanction risk perceptions from crime-prone popula- tions is feasible with a sustained commitment among a cadre of researchers and with the availability of funding. Apel (in press) reviews the existing research that measures perceptions of sanction risks. Although there have been a scattering of suggestive stud- ies, there has not yet been systematic large-scale research on the subject. Moreover, there has been no research at all on the specific question of per- ceptions of the sanction risk associated with commission of murder. With so much to learn, we think it prudent for research to proceed sequentially. A good beginning would be small-scale studies that include one-on-one cognitive interviews with respondents in the relevant popula- tion of potential murderers. These interviews, taking the form of structured conversations, would explore the feasibility and usefulness of probabilistic and other modes of questioning about sanction risk perception. The lessons learned from this exploratory research would inform the design of larger studies, the aim being to eventually develop a program of survey research that would regularly measure the perceptions of the sanction risk held by potential murderers and by potential criminals more generally. The committee is not confident that measurement of the sanctions risk perceptions of potential murderers can succeed in producing information useful to the study of deterrence, but one cannot be sure unless the effort is made. As demonstrated by the discussion in Chapters 4 and 5, the alterna- tive of continuing to make unfounded assumptions about these perceptions is not useful. Measurement of sanction risk perceptions may enable deter- rence research to make progress that thus far has not been possible in the absence of data. The committee is more optimistic about the feasibility and usefulness of measuring perceptions of sanctions risks among potential criminals more broadly. This greater optimism has two bases. First, homicide is the least frequent of the crimes included in the “Part 1-Crime Index” of the Federal Bureau of Investigation (FBI), which also includes rape, robbery, aggravated assault, burglary, larceny, and auto theft. More people commit all the other crimes than commit homicides. Thus, it will probably be easier to survey sizable numbers of potential perpetrators of these crimes than of potential murderers. The National Survey of Youth, for example, already surveys youth and young adults about their involvement in such crimes as theft, selling drugs, and assault. Second, perpetrators who are apprehended for crimes less serious than murder are far less likely to receive lengthy prison sentences, particularly if they are juveniles. Thus, these people have more opportunity to learn about
OCR for page 110
110 DETERRENCE AND THE DEATH PENALTY sanction risk on the basis of personal experience, a source of information that may be vital to formation of sanction risk perceptions. Inference on Perceptions from Homicide Rates Following Executions As a complement to research that directly measures perceptions, some committee members believe that study of homicide rates immediately fol- lowing execution events might also provide useful evidence of the percep- tions of potential murderers. As discussed in Chapter 5, the time-series research has largely been devoted to the question of whether homicide rates change in the immediate aftermath of an execution. For the reasons detailed in that chapter, the committee concluded that existing studies were not informative about whether capital punishment affects homicide rates, in part because of the absence of any measure of perceptions. The committee considered at length whether future research on execu- tion events, if properly conducted, might be informative about whether homicide rates, at least in the short term, are responsive to execution events. We concluded that at best the information to be gleaned from this type of research would be limited and fall far short of establishing whether capital punishment increases, decreases, or has no effect on homicide rates. Even if a short-term impact could be established, it would be difficult to determine whether homicides were actually prevented or simply displaced in time. More fundamentally, execution event studies cannot speak to the question of whether and how the state’s overall sanction regime affects the homicide rate. For example, a null finding from an event study would leave open the possibility that a death penalty regime had a deterrent effect relative to a regime that precluded the death penalty or more narrowly prescribed its applicability. It is important to note that any one execution would only have a deterrent effect if it changed potential murderers’ perceptions of the likelihood of an execution, which is not necessarily the case. Acknowledging these limitations, some committee members nonethe- less argue that if a well-done event study did produce evidence of an effect—whether positive or negative and no matter how temporary—that result would be of considerable interest. It would demonstrate that po- tential murderers as a group are actually paying attention to the state’s actions and are influenced by them. In short, it would confirm a threshold condition for there to be a deterrent or brutalization effect and invite fur- ther inquiry. Other committee members are not convinced of the value of establishing this threshold condition or are not convinced that any study of this sort could make a convincing case that it had isolated a causal ef- fect of executions.
OCR for page 111
111 CHALLENGES TO IDENTIFYING DETERRENT EFFECTS IDENTIFYING EFFECTS: FEEDBACKS AND UNOBSERVED CONFOUNDERS Even with better data and information on sanction regimes and per- ceptions of sanction risks, formidable difficulties remain to understanding the impact of the death penalty on homicide. With only observational (nonexperimental) data on capital punishment and homicides, researchers must face the fundamental problem that the data alone cannot reveal the counterfactual question of interest: What would have happened if the death penalty not been applied in a “treatment” state or if the death penalty had been applied in a “control” state? Although this counterfactual-outcomes problem is common to all observational studies of cause and effect, it has long been understood to be particularly problematic for understanding the deterrent effect of the death penalty. A capital punishment regime evolves over time as a result, among other things, of a complex interplay of crime trends, social norms, criminal justice budgets, and election results. This context makes it very difficult to identify the effects of the capital sanction regime alone. To better understand these issues, we highlight three related identi- fication problems that complicate efforts to draw credible inferences on the effect of capital punishment on homicides. The first, referred to as a feedback effect, arises when homicide rates may directly affect the capital sanction regime. The second, referred to as the omitted variable problem, arises when variables that are jointly associated with the sanction regime and homicide rate are either unknown or unobserved. The third, referred to as an equilibrium effect, arises when the capital sanction regime may directly affect other aspects of the criminal justice system, including, most notably, noncapital sanction policies. Feedback Effects Deterrence research conducted in the early 1970s (Carr-Hill and Stern, 1973; Ehrlich, 1975; Sjoquist, 1973) recognized the possibility of feedbacks or simultaneity whereby crime rates may affect the sanction risk and sever- ity even as the sanction risk and severity may affect crime rates. The nature of such feedbacks is not well understood, but there are good reasons for believing that feedbacks are present and may be substantial. To illustrate the problem, suppose that in a particular state during a particular year there is an exogenous increase in the rate of homicide. If, given the additional workload and resulting strain on resources, district attorneys were more reluctant to pursue the death penalty, a continu- ing upward trend in homicides would appear to show that a reduction in the probability of a death sentence is associated with an increase in
OCR for page 114
114 DETERRENCE AND THE DEATH PENALTY Knowledge of the entire system, however, is not a necessary require- ment for learning about the overall impact of the capital sanction regime. For some questions, the effects of the death penalty on sentence bargain- ing and on administrative resource constraints are an intrinsic part of the mechanism by which a capital regime affects murder rates. Consider, for example, a case in which a judicial ruling terminates the use of the death penalty for some category of homicides. It would be of considerable interest to have a reliable estimate of the overall effect of this reform on the murder rate, even if it is not possible to distinguish among the various mechanisms (reduction in the probability of a death sentence, weaker bargaining posi- tion by the district attorney, or increased court resources available for the average case) that led to that effect. Still, this sort of “black box” estimate is not satisfactory if the goal is to estimate the effect of the threat of execu- tion, in part because the ancillary effects of the administration of the death penalty can be generated by other means, such as changes in court budgets. Is a more reliable approach to identifying the deterrent effect of capital punishment possible? Part of the solution may be to develop a better un- derstanding of the factors that affect sanction regimes, including possible feedbacks from homicide or other crime patterns. The earlier National Research Council report (1978, p. 47) observed: “Knowledge of the effect of crime on the behavior of the criminal justice system is still extremely limited.” This conclusion is still true today, 30 years later. The 1978 report went on to observe: “While the seeming dearth of untainted identification restrictions may reflect the fact that none exist, it is certainly as likely that it simply reflects our ignorance of the determinants of sanctions” (p. 48). Three decades later this committee observes that both of these assessments apply to contemporary research on deterrence. As noted above, the 1978 report urged more research on the sanction- generation process for the purpose of accumulating a knowledge base that might reveal approaches to plausible identification. Although knowledge of the sanction-generation process is not required for identification of overall effects of certain relevant regime changes, that knowledge may be useful in determining the validity of a proposed identification method. Also, as a practical matter, some committee members believe that without better knowledge of sanction generation, the prospects for credible identifica- tion are small. Committee members holding this perspective argue that a deeper institutional and theoretical knowledge of sanction process would materially increase the chances of researchers’ becoming aware of credible sources of identification and that without such knowledge the chances for credible identification are remote. Other committee members are less pes- simistic that a chance event or insight might provide a basis for credible identification. However credible identification might ultimately be achieved, the com-
OCR for page 115
115 CHALLENGES TO IDENTIFYING DETERRENT EFFECTS mittee fully endorses another observation from the earlier report (National Research Council, 1978, p. 49): It must be noted, however, that identification restrictions cannot be manu- factured. If the process generating the data is truly one that leaves the crime function unidentified, then persistent attempts to produce identifying restrictions because of the desire to estimate the deterrent effect will only produce different kinds of error. Even if all such attempts found a “deter- rent” effect, no conclusion would be warranted unless some of them used validly based identification restrictions. ADDRESSING MODEL UNCERTAINTY WITH WEAKER ASSUMPTIONS The persistent problems that researchers have had in providing mean- ingful answers about the deterrent effect of capital punishment is unsur- prising once one recognizes that this body of empirical research rests on strong and unverified assumptions. Although, in practice, researchers often recognize and acknowledge that their assumptions may not hold, they are defended as necessary to provide meaningful answers and in order to make inferences. But the use of strong assumptions hides the problem that very little is understood about the process that may link capital punishment to future crimes. The different findings in the deterrence research reflect different choices of assumptions, most of which cannot be supported by strong a priori justifications. As documented throughout this report, many of the assump- tions used in the research on the deterrent effect of capital punishment are not credible. Furthermore, the state of social science knowledge does not support a unique model that can be used to identify the effects of capital punishment under the current U.S. sanction regime or to permit the evalu- ation of deterrence under alternative regimes. The study of deterrence is plagued by model uncertainty. The failure of the existing research to address the issue of model un- certainly is evident in the debate initiated by Donohue and Wolfers (2005), who challenged claims of deterrence by a broad set of researchers. Much of their challenge involved demonstrations of how small changes in the models used in the various studies led to very different estimates of deterrence ef- fects, in some case changing from positive to negative or vice versa, and in others eliminating statistical significance. Some of their exercises altered the set of observations over which the analysis had been conducted; in other cases they changed the choice of control or instrumental variables. Although Donohue and Wolfers provide useful evidence of the sensitiv- ity of many claims of deterrence to model assumptions, their demonstra- tion begs the question of how to adjudicate their findings relative to the
OCR for page 116
116 DETERRENCE AND THE DEATH PENALTY papers they critique. This may be seen in two of the rejoinders that have been written to their study. Dezhbakhsh and Rubin (2011) and Mocan and Gittings (2010) provide a large number of modifications of their baseline homicide regressions and argue that deterrence effects generally appear in them. However, they fail to provide any guidance as to what is learned from the specifications that are inconsistent with their claim of evidence of deterrence. Rather, the authors’ claims are based on ad hoc choices of alternative model specifications; there is no systematic construction of the models from which to draw inferences. That changes in a given statistical model change the output of the model is hardly unique to the studies of capital punishment and deterrence literature. The problem is that there have been almost no serious attempts to reconcile the many different findings reported in the research. Given this existing uncertainty, how might research proceed? Certainly, research aimed at reducing model uncertainty would be useful. To that end, the committee proposed, above, developing data and research on sanction regimes and perceptions of sanction risk. Another complementary and potentially useful approach would be to explicitly account for model uncertainty when drawing inferences on the impact of capital punishment. Rather than continue with the conventional practice of assuming whatever it takes to achieve point identification, and then providing ad hoc justifica- tions for particular sets of assumptions to justify a given model, deterrent studies might instead consider what can be learned when explicitly rec- ognizing model uncertainty. Although the resulting inferences may reflect a certain degree of ambiguity about the effects of capital punishment on homicides, those inferences will necessarily possess greater credibility. To explore the idea of addressing model uncertainty, the committee commissioned papers illustrating application of two complementary re- search paradigms—the model averaging approach and the partial identifi- cation approach. Model Averaging Model averaging, though based on earlier work (Bates and Granger, 1969; Leamer, 1978), developed theoretically, algorithmically, and as an applied technique in the mid-1990s (examples include Chatfield, 1995; Draper, 1995; Draper et al., 1993; Raftery, Madigan, and Hoeting, 1997). The model averaging approach constructs a probability distribution for a range of estimates of the deterrent effect of capital punishment, and the researcher constructs this distribution to reflect the researcher’s own or others experts’ prior beliefs about the probability that a given model is valid. By asking what can be learned by combining the information ob- tained across a wide range of models, model averaging methods provide
OCR for page 117
117 CHALLENGES TO IDENTIFYING DETERRENT EFFECTS a natural way to make empirical claims robust to the details of uncertain model specifications. This technique has recently been used in two studies of capital punish- ment: Cohen-Cole et al. (2009) and Durlauf, Fu, and Navarro (in press). These studies apply the modeling average approach to various specifications that have appeared in the research on capital punishment and deterrence. Cohen-Cole et al. (2009) use this method to adjudicate the different find- ings of Dezhbakhsh, Rubin, Shepherd (2003) and Donohue and Wolfers (2005). Durlauf, Fu, and Navarro (in press), whose paper was written for this committee, consider a range of models based on alternative substantive assumptions that have appeared in the research, including, for example, how to measure subjective arrest, sentencing, and execution probabilities and whether the deterrent effect of capital punishment differs across states. These two papers aim to understand how different assumptions matter and whether differences in assumptions render deterrence estimates fragile. In both papers, the researchers find that model uncertainty swamps the informational content about deterrent effects. That is, after accounting for the modeling uncertainty, the empirical evidence does not reveal whether capital punishment increases or decreases homicides. As an example of this result, consider the Cohen-Cole et al. (2009) analysis of the models in Dezhbakhsh, Rubin, and Shepherd (2003) and Donohue and Wolfers (2005). Dezhbakhsh, Rubin, and Shepherd (2003) report, under their preferred specification, a statistically significant point estimate of 18 lives saved for each execution. However, when all of the different specifications spanned in the two papers are given probability weights, Cohen-Cole et al. estimate an approximate 95 percent confidence interval on the number of lives saved per execution of [–24, 124]: see Fig- ure 6-1, which is from Cohen-Cole et al. The figure illustrates the model uncertainty by providing a weighted histogram of the estimated net lives saved for all of the models considered. For the case illustrated in this histo- gram, the posterior probability for the models with point estimates suggest that deterrence is 72 percent, but there is substantial bunching around 0, the individual estimates vary widely, and there is a nontrivial probability on models that suggest a large increase in homicides associated with ex- ecutions (a probability 0.15 of point estimates of 20 or more homicides). Thus, the heterogeneity of the model-specific estimates makes it impossible to draw strong qualitative conclusions about the deterrent effect of capital punishment. The model averaging approach provides a formal and elegant Bayesian method for incorporating uncertainty about the correct modeling assump- tions into inferential methods. This approach can be effectively used to illustrate the importance of different assumptions and the fragility of the estimates to these assumptions, as is done in Cohen-Cole et al. (2009) and
OCR for page 118
118 DETERRENCE AND THE DEATH PENALTY Net Lives Saved: Weighted Histogram 0.4 DW Estimate DRS Estimate 0.35 0.3 0.25 0.2 0.15 0.1 0.05 0 -200 -100 0 100 200 300 400 FIGURE 6-1 Weighted histogram of the net lives saved by the death penalty. NOTES: The figure includes models for each of the DRS (Dezhbakhsh, Rubin, and Shepherd, 2003) categories. The weights are the posterior model probabilities (Bayes factors). The DRS and DW (Donohue and Wolfers, 2005) lines correspond to the individual model from each with the largest and smallest number of lives saved, respectively. The unweighted histogram is similar. SOURCE: Cohen-Cole et al. (2009, Figure 1). Used with permission. R02175 Figure 6-1 vector, editable Durlauf, Fu, and Navarro (in press). The approach depends on research- ers’ specifications of the model space and original that model obtained from prior oversource space, over (Cohen-Cole et al, 2007) which there may be disagreement. Such disagreement should not obscure an essential strength of the model averaging approach: model averaging pro- vides an approach for systematically exploring sensitivity over an explicitly defined model space. Ultimately, this approach might also be used to infer the effect of the death penalty on homicides. However, for this purpose, a key challenge would be selecting a set of models to include in the averaging and provid- ing a prior probability distribution over this set that is plausible. The ap- proach presumes that the range of models included in the averaging routine includes the correct model that accurately describes the real world and,
OCR for page 119
119 CHALLENGES TO IDENTIFYING DETERRENT EFFECTS moreover, that the researcher can provide informed prior beliefs about the probability that each model is valid. In the context of the research on capi- tal punishment, we have found no reason to believe that the existing range of point-identified models includes the correct one, and there is currently little basis for assigning probabilities to the correctness of each model in the literature. As discussed in Chapter 4, the committee did not find the instru- mental variables used in the existing research to be credible. If the existing models are all invalid, using the modeling averaging approach to produce interpretable deterrence estimates can be problematic.2 With uncertainty about the model space and the prior probabilities, either research efforts to construct informative priors or research showing the sensitivity of the posterior to different prior distributions may be useful. Partial Identification Partial identification methods provide an alternative approach for re- ducing the dependence of claims of a deterrence effect on arbitrary assump- tions. Rather than start with a particular set of point-identified models and prior beliefs about the probability that each model is valid, both as defined by the researcher, one might instead begin by directly considering what can be inferred under a set of weak assumptions that may possess greater credibility. A natural starting point, for example, is to examine what can be learned in the absence of any assumptions. What do the data alone reveal? Under these weaker assumptions, deterrent effects may not be point iden- tified, but they will be partially identified, with bounds rather than point estimates. Thus, the partial identification approach formalizes the inherent tradeoff between the strength of the maintained assumptions and the cred- ibility of inferences (see Manski, 2003). The partial identification methodology has been developed and applied over the past 20 years, beginning with Manski (1989, 1990). In an early application to criminal justice policy, Manski and Nagin (1998) studied sentencing and recidivism of juvenile offenders in the state of Utah and demonstrated how partial identification can be used to produce more cred- ible inferences than had previously been produced. Youth in Utah faced a policy that gave judges the discretion to order varying sentences. Using this discretion, judges sentenced some offenders to residential confinement and sentenced other offenders to no confinement. A policy question of potential 2 The Cohen-Cole et al. exercise (2009) was narrow in that it considered the smallest model space one could generate around the different assumptions in Donohue and Wolfers (2005) and Dezhbakhsh, Rubin, and Shepherd (2003). One can easily argue that for a full model av- eraging analysis, other models warrant a priori consideration. However, one could also argue that some of the models considered in Cohen-Cole et al. should not have been included, given a prior probability of 0.
OCR for page 120
120 DETERRENCE AND THE DEATH PENALTY interest was to compare recidivism under that policy with the recidivism that would occur under a policy proposal that removed judicial discretion and instead mandated that all offenders be sentenced to confinement. The study showed how bounds of varying width on the existing treatment effect which allows judges’ discretion could be achieved by combining data on outcomes under the status quo with relatively weak assumptions regarding the manner in which (1) judges have made sentencing decisions and (2) criminality was affected by sentencing. More recently, in a paper written for this committee, Manski and Pepper (in press) illustrate the partial identification approach in a relatively simple setting by examining the effect of death penalty statutes on the national homicide rate (per 100,000) over 2 years, 1975-1977: 1975 was the last full year of the federal moratorium on death penalty, and 1977 was the first full year after the moratorium was lifted. In 1975, the death penalty was illegal throughout the country; and in 1977, 32 states had legal death penalty statutes. Over this 2-year period, homicide rates in the 32 states that had adopted a death penalty statute in 1977 decreased by 0.6; in the remaining states, the homicide rates decreased by 1.1. It has been common in the relevant research to report the difference-in-difference estimate, which in this case is 0.5 (–0.6 + 1.1), as a point estimate of the effect of capital punishment on the national homicide rate. This interpretation suggests that the death penalty increases crime, but Manski and Pepper (in press) show that this difference-in-difference form only point identifies the impact of the death penalty under a number of strong assumptions, most notably that the effect is assumed to be homogeneous across states and dates. Under weaker assumptions that allow the deterrent effect to vary across states, the average effect of the death penalty is only partially identified, and it was found to lie in the interval [–1.9, 8.3]. Under still weaker assumptions under which the effect of the death penalty is allowed to vary over time, the bounds widen further. Thus, under these weaker models, the average treatment effect of capital punishment is bounded, but the data do not identify whether the death penalty increases or decreases homicides. The committee does not endorse the specific findings of the recent studies applying the model averaging or partial identification approaches. These studies are largely illustrative and do not address many of the key problems identified throughout this report. Most notably, they do not define the counterfactual sanction regime and do not address the issue of how potential murderers perceive sanction risks. Still, these studies serve as a starting point for future research that might inform the debate on the death penalty. Rather than imposing the strong but unsupported assump- tions required to identify the effect of capital punishment on homicides in a single model or an ad hoc set of similar models, approaches that explicitly account for model uncertainty may provide a constructive way for research
OCR for page 121
121 CHALLENGES TO IDENTIFYING DETERRENT EFFECTS to provide credible albeit incomplete answers. The basic insight is that with model uncertainty, the identification of deterrent effects need not be an all- or-nothing undertaking: the available data and credible assumptions may yield partial conclusions. Some people may find partial conclusions unappealing and be tempted to impose strong assumptions in order to obtain definitive answers. We caution against this reaction. Imposing strong but untenable assumptions cannot truly resolve inferential problems. Rather, it simply replaces the modeling uncertainty with uncertainty associated with the underlying as- sumptions. We have seen this repeatedly in the literature on the death penalty. The earlier Panel on Research on Deterrent and Incapacitative Ef- fects recognized this when it concluded (National Research Council, 1978, p. 63) “research on this topic is not likely to produce findings that will or should have much influence on policymakers.” Today, more than 30 years later, perhaps the primary lesson learned from the latest round of empirical research on the deterrent effect of the death penalty is that researchers and policy makers must cope with ambiguity. Explicitly recognizing and ac- counting for this uncertainty seems like the only hope of moving forward. RECOMMENDATION: The committee recommends further inves- tigation of the effects of capital punishment using assumptions that are weaker and more credible than those that have traditionally been invoked by empirical researchers. REFERENCES Alarcón, A.L., and Mitchell, P.M. (2011). Executing the will of the voters?: A roadmap to mend or end the California legislature’s multi-billion-dollar death penalty debacle. Loyola of Los Angeles Law Review, 44(Special), S41-S224. Apel, R. (in press). Sanctions, perceptions, and crime: Implications for criminal deterrence. Submitted to Journal of Quantitative Criminology, 28. Bates, J.M., and Granger, C.W.J. (1969). The combination of forecasts. Operational Research Quarterly, 20(4), 451-468. California Commission on the Fair Administration of Justice. (2008). Report and Recommen- dations on the Administration of the Death Penalty in California. Sacramento: Author. Carr-Hill, R.A., and Stern, N.H. (1979). Crime, the Police and Criminal Statistics: An Analysis of Official Statistics for England and Wales Using Econometric Methods. New York: Academic Press. Chatfield, C. (1995). Model uncertainty, data mining and statistical inference. Journal of the Royal Statistical Society Series A-Statistics in Society, 158(3), 419-466. Cohen-Cole, E., Durlauf, S., Fagan, J., and Nagin, D. (2009). Model uncertainty and the deter- rent effect of capital punishment. American Law and Economics Review, 11(2), 335-369. Cook, P.J. (2009). Potential savings from abolition of the death penalty in North Carolina. American Law and Economics Review, 11(2), 498-529. Cook, P.J., Ludwig, J., and Braga, A.A. (2005). Criminal records of homicide offenders. The Journal of the American Medical Association, 294(5), 598-601.
OCR for page 122
122 DETERRENCE AND THE DEATH PENALTY Dezhbakhsh, H., and Rubin, P.H. (2011). From the “econometrics of capital punishment” to the “capital punishment” of econometrics: On the use and abuse of sensitivity analysis. Applied Economics, 43(25), 3,655-3,670. Dezhbakhsh, H., Rubin, P.H., and Shepherd, J.M. (2003). Does capital punishment have a deterrent effect? New evidence from postmoratorium panel data. American Law and Economics Review, 5, 344-376. Dominitz, J., and Manski, C.F. (1997). Perceptions of economic insecurity - evidence from the survey of economic expectations. Public Opinion Quarterly, 61(2), 261-287. Donohue, J.J., and Wolfers, J. (2005). Uses and abuses of empirical evidence in the death penalty debate. Stanford Law Review, 58(3), 791-845. Draper, D. (1995). Assessment and propagation of model uncertainty. Journal of the Royal Statistical Society Series B-Methodological, 57(1), 45-97. Draper, D., Hodges, J.S., Mallows, C.L., and Pregibon, D. (1993). Exchangeability and data- analysis. Journal of the Royal Statistical Society Series A-Statistics in Society, 156(1), 9-37. Durlauf, S., Fu, C., and Navarro, S. (in press). Capital punishment and deterrence: Under- standing disparate results. Submitted to Journal of Quantitative Criminology, 28. Ehrlich, I. (1975). Deterrent effect of capital punishment—Question of life and death. Ameri- can Economic Review, 65(3), 397-417. Fischhoff, B., Parker, A.M., De Bruin, W.B., Downs, J., Palmgren, C., Dawes, R., and Manski, C.F. (2000). Teen expectations for significant life events. Public Opinion Quarterly, 64(2), 189-205. Hurd, M., and McGarry, K. (1995). Evaluation of the subjective probabilities of survival in the Health and Retirement Study. Journal of Human Resources, 30(5), S268-S292. Hurd, M.D., and McGarry, K. (2002). The predictive validity of subjective probabilities of survival. The Economic Journal, 112(482), 966-985. Hurd, M.D., Smith, J.P., and Zissimopoulos, J.M. (2004). The effects of subjective survival on retirement and social security claiming. Journal of Applied Econometrics, 19(6), 761-775. Kuziemko, I. (2006). Does the threat of the death penalty affect plea bargaining in murder cases? Evidence from New York’s 1995 reinstatement of capital punishment. American Law and Economics Review, 8(1), 116-142. Leamer, E.E. (1978). Specification Searches: Ad Hoc Inference with Nonexperimental Data. New York: Wiley. Lochner, L. (2007). Individual perceptions of the criminal justice system. American Economic Review, 97(1), 444-460. Manski, C.F. (1989). Anatomy of the selection problem. Journal of Human Resources, 24(3), 343-360. Manski, C.F. (1990). Nonparametric bounds on treatment effects. American Economic Re- view, 80(2), 319-323. Manski, C.F. (2003). Partial Identification of Probability Distributions. New York: Springer. Manski, C.F. (2004). Measuring expectations. Econometrica, 72(5), 1,329-1,376. Manski, C.F., and Nagin, D.S. (1998). Bounding disagreements about treatment effects: A case study of sentencing and recidivism. Sociological Methodology, 28(1), 99-137. Manski, C.F., and Pepper, J. (in press). Deterrence and the death penalty: Partial identifi- cation analysis using repeated cross sections. Submitted to Journal of Quantitative Criminology, 28. Manski, C.F., and Straub, J.D. (2000). Worker perceptions of job insecurity in the mid- 1990s—Evidence from the survey of economic expectations. Journal of Human Re- sources, 35(3), 447-479.
OCR for page 123
123 CHALLENGES TO IDENTIFYING DETERRENT EFFECTS Mocan, N., and Gittings, K. (2010). The impact of incentives on human behavior: Can we make it disappear? The case of the death penalty. In R.E.S. Di Tella and E. Schargrodsky (Eds.), The Economics of Crime: Lessons for and from Latin America (pp. 379-420). Chicago: University of Chicago Press. Mulvey, E.P. (2011). Highlights from Pathways to Desistance: A Longitudinal Study of Seri- ous Adolescent Offenders. Juvenile Justice Fact Sheet. Washington, DC: U.S. Department of Justice. National Research Council. (1978). Deterrence and Incapacitation: Estimating the Effects of Criminal Sanctions on Crime Rates. Panel on Research on Deterrent and Incapacitative Effects. A. Blumstein, J. Cohen, and D. Nagin (Eds.), Committee on Research on Law Enforcement and Criminal Justice. Assembly of Behavioral and Social Sciences. Wash- ington, DC: National Academy Press. National Research Council. (1986). Criminal Careers and “Career Criminals.” Panel on Re- search on Criminal Careers, A. Blumstein, J. Cohen, J.A. Roth, and C.A. Visher (Eds.), Committee on Research on Law Enforcement and the Administration of Justice. Com- mission on Behavioral and Social Sciences and Education. Washington, DC: National Academy Press. Raftery, A.E., Madigan, D., and Hoeting, J.A. (1997). Bayesian model averaging for linear regression models. Journal of the American Statistical Association, 92(437), 179-191. Roman, J.K., Chalfin, A.J., and Knight, C.R. (2009). Reassessing the cost of the death penalty using quasi-experimental methods: Evidence from Maryland. American Law and Eco- nomics Review, 11(2), 530-574. Sjoquist, D.L. (1973). Property crime and economic behavior: Some empirical results. Ameri- can Economic Review, 63(3), 439-446. West, D.J., and Farrington, D.P. (1973). Who Becomes Delinquent? Second Report of the Cambridge Study in Delinquent Development. London: Heinemann Educational. Wolfgang, M.E. (1958). Patterns in Criminal Homicide. Philadelphia: University of Pennsyl- vania Press.
OCR for page 124