Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 101
6
Challenges to Identifying
Deterrent Effects
R
esearchers from diverse disciplines have contributed to the capital
punishment literature, with prominent contributions by economists,
criminologists, and sociologists. Although researchers’ disciplinary
backgrounds have affected the methods used and the framing of the re-
search questions, the failings of the capital punishment literature are not
rooted in the use of particular empirical methods or theoretical models
of criminal decision making. Rather, the failings are rooted in manifest
deficiencies related to the research data and methods and the researchers’
interpretations of results. Chapters 4 and 5 call attention, respectively, to
fundamental deficiencies in panel and time-series studies. Both approaches
share two basic deficiencies and also manifest two others to some degree.
One shared deficiency is grossly incomplete specification of the sanction re-
gime for homicide. Even in states that make the most frequent use of capital
sanctions, noncapital sanctions are the most common sanction imposed for
a homicide conviction. No study of either type accounts for the noncapi-
tal component of the sanction regime in states with and without capital
punishment. The second basic deficiency is failure to pose a credible model
of the sanction risk perceptions of potential murderers and the behavioral
response to such perceptions. In the absence of such a model, it is difficult,
at best, to interpret data relating sanction regimes to homicide rates.
As discussed in Chapters 4 and 5, these two deficiencies are sufficient
to make existing studies uninformative about the effect of capital punish-
ment on homicide. Both of these deficiencies are potentially correctable.
However, even if the research and data collection initiatives discussed in
this chapter are ultimately successful, research in both literatures share a
101
OCR for page 102
102 DETERRENCE AND THE DEATH PENALTY
common characteristic of invoking strong, often unverifiable, assumptions
in order to provide point estimates of the effect of capital punishment on
homicides. A point estimate may offer the appearance of desirable certitude,
but only at a high cost in credibility. Still another deficiency is inattention
to potential feedbacks through which homicide rates, and crime rates more
generally, may affect the specification and administration of a sanction
regime while the regime simultaneously affects homicide rates. Recogni-
tion of potential feedbacks is relevant both to identify the direct effect of
capital punishment on homicide rates and to predict the ultimate effect after
feedbacks occur. Feedbacks affect the time-series and panel studies differ-
ently because of differences in the time frames of the data typically used in
the two approaches—monthly, weekly, or even daily data in the time-series
studies and annual data in the panel studies.
In light of these deficiencies, the committee has reached the following
conclusion and recommendation:
CONCLUSION AND RECOMMENDATION: The committee con-
cludes that research to date on the effect of capital punishment on homi-
cide is not informative about whether capital punishment decreases,
increases, or has no effect on homicide rates. Therefore, the committee
recommends that these studies not be used to inform deliberations re-
quiring judgments about the effect of the death penalty on homicide.
Consequently, claims that research demonstrates that capital punish-
ment decreases or increases the homicide rate by a specified amount or
has no effect on the homicide rate should not influence policy judgments
about capital punishment.
The committee was disappointed to reach the conclusion that research
conducted in the 30 years since the National Research Council (1978)
report on this subject has not sufficiently advanced knowledge to allow
a conclusion, however qualified, about the effect of the death penalty on
homicide rates. Yet this is our conclusion. Some studies play the useful role,
either intentionally or not, of demonstrating the fragility of their claims to
have found—or not to have found—deterrent effects. However, even these
studies suffer from two intrinsic shortcomings that severely limit what can
be learned from them about the effect of the death penalty on homicide
rates from an examination of the death penalty as it has actually been ad-
ministered in the United States in the past 35 years.
Commentary on research findings often pits studies claiming to find
statistically significant deterrent effects against those finding no statistically
significant effects, with the latter studies sometimes interpreted as imply-
ing that there is no deterrent effect. A fundamental point of logic about
hypothesis testing is that failure to reject a null hypothesis does not imply
OCR for page 103
103
CHALLENGES TO IDENTIFYING DETERRENT EFFECTS
that the null hypothesis is correct. For the evidence of even a small effect to
be credible, it requires a demonstration, first and foremost, that the effect is
based on a sound research design. Estimates that lack credibility are not in-
formative regardless of the consistency of their estimated size. The amount
of the effect must also be small in size and estimated with good precision,
for example, by being contained within a tight confidence interval.
Our mandate was not to assess whether competing hypotheses about
the existence of marginal deterrence from capital punishment are plausible,
but simply to assess whether the empirical studies that we have reviewed
provide scientifically valid evidence. In its deliberations and in this report,
the committee has made a concerted effort not to approach this question
with a prior assumption about deterrence. Having reviewed the research
that purports to provide useful evidence for or against the hypothesis that
the death penalty affects homicide rates, we conclude that it does not pro-
vide such evidence.
We stress, however, as noted above, that a lack of evidence is not evi-
dence for or against the hypothesis. Hence, the committee does not construe
its conclusion that the existing studies are uninformative as favoring one
side or the other side in the long-standing societal debate about deterrence
and the death penalty.
In this chapter, we elaborate on these deficiencies that form the basis
for this conclusion and cautiously offer some ideas on potential remedies.
With regard to remedies, our report provides a somewhat less pessimistic
perspective than did the earlier National Research Council (1978, p. 63)
report: “[T]he Panel considers that research on this topic is not likely to
produce findings that will or should have much influence on policymakers.”
The committee does not expect that advances in collecting data on
sanction regimes and obtaining knowledge of sanctions risk perceptions
will come quickly or easily. However, data collection on the noncapital
component of the sanction regime need not be entirely complete to be use-
ful. And even if research on perceptions of the risk of capital punishment
cannot resolve all major issues, some progress would be an important step
forward. Even if these advances prove unsuccessful in providing useful
information on the incremental deterrent effect of capital punishment in
relation to a lengthy prison sentence, the committee believes that there are
potentially major benefits from new data collection, theory, and methodol-
ogy for study of the effect of noncapital sanctions on crimes not subject
to the death penalty. As discussed in Chapter 1, because of the overlap in
the methods and data used in studies of capital punishment and in broader
studies on the effects of sanctions on crime, our charge included a provi-
sion for recommending research that might advance that broader research
literature, and we do so in the rest of this chapter.
OCR for page 104
104 DETERRENCE AND THE DEATH PENALTY
DATA ON SANCTION REGIMES
Incomplete and inaccurate data have marred research on the effect of
capital punishment on homicides. The most important data problem is that
studies have been based on a very incomplete specification of state sanction
regimes. Part of the difficulty has been lack of conceptual agreement on
how to measure the intensity of use of capital punishment. However, we
see the primary problem as a complete absence of data on the noncapital
sanctions that might be applied to offenders convicted of homicide. A study
of capital punishment in North Carolina by Cook (2009) illustrates the im-
portance of the problem of the absence of information on noncapital sanc-
tions. Of 274 cases prosecuted as capital cases, only 11 resulted in a death
sentence. Another 42 resulted in dismissal or a verdict of not guilty, which
left 221 cases that resulted in convictions and received noncapital sanctions.
As discussed at length in Chapter 4 and below, there are sound reasons
for predicting a correlation between the capital and noncapital components
of a state’s sanction regime. Two examples of how this might occur are the
plea bargaining leverage that the threat of capital punishment may afford
prosecutors and the influence of the state’s political culture on the legislated
design and administration of both the capital and noncapital components
of the regime. Such a correlation would bias the estimated deterrent effect
of capital punishment.
None of the studies we reviewed sought to measure the availability
and intensity of use of the noncapital sanction alternatives for the punish-
ment of homicide. Such alternatives may include a life sentence without
the possibility of parole, a life sentence with the possibility of parole, and
sentences of less than life. It would also be important to have data on the
time actually served for convicted murderers who are paroled or who serve
less than a life sentence.
It is currently not possible to measure noncapital sanction alternatives
at the state level because the required data are not available. The data that
are available include those from the Bureau of Justice Statistics (BJS), which
publishes nationwide statistics on sentences for prison admissions and time
served for prison releases, based on data collected as part of the National
Corrections Reporting Program (NCRP) initiated in the early 1980s. More
than 40 states now report annual data on sentences for admissions and
time served for releases. Individual-level demographic characteristics are
also reported. In principle, these data could be used to measure the actual
administration of the legally authorized dimensions of most state sanction
regimes, not only for murder but also for other types of crimes. The dif-
ficulty is that the data are often extremely incomplete.
In some years, states fail to report any data. Just as important, the
data that are sent to BJS are often so incomplete that it is impossible to
OCR for page 105
105
CHALLENGES TO IDENTIFYING DETERRENT EFFECTS
construct valid state-level measures of the administration of the sanction
regime. Indeed, the committee attempted to use these data for the purposes
of this report but concluded that the data gaps made their use infeasible.
More complete data on the actual administration of sanction regimes might
be obtained by expanding the NCRP to include all 50 states and filling the
data gaps due to incomplete reporting. Alternatively, an entirely new data
collection system might be desirable. Either way, the collection of more
complete data on sanction regimes for murder and other crimes is feasible.
The data are available: the challenge is designing and implementing an ef-
fective system for their collection.
Even if data on the actual administration of state sanction regimes were
complete, they could only be used to measure how sanction regimes are
actually administered. The data do not specify the potential sanction regime
in a state—the range of sanction alternatives that are legally authorized.
We are not aware of any ongoing effort to assemble data on the legislated
sanction regimes of the states for murder and other crimes. Data on the
legislated regime are important because they define the range of penalties
that can potentially be imposed. Thus, the measurement of legally autho-
rized sanctions by the states for homicides and other crimes may require a
new data collection system.
The committee did not explore the benefits and costs of alternative ap-
proaches for measurement of state-level sanction regimes for murder. We
only emphasize the vital importance of collecting these data.
RECOMMENDATION: The committee recommends that a concerted
effort be made to collect data on the sanctions regimes faced by poten-
tial murderers, with particular attention to fixing the current absence
of data on noncapital sanctions.
As noted above, because the methods and data used to study the effect
of noncapital sanctions on crimes other than murder are similar to those
used in research on capital punishment, the committee’s charge includes a
provision that we make recommendations for advancing research on the
broad effects of sanctions on crime. Thus, we also stress the vital impor-
tance of an expanded effort to collect data suitable not only for measuring
sanction regimes for murder, but also for measuring sanction regimes for
other major crimes.
PERCEPTIONS OF SANCTION RISKS
As emphasized in Chapter 3, it is not possible to interpret empirical
evidence on the relationship of homicide rates to sanctions without un-
derstanding how potential murderers perceive sanction regimes. The com-
OCR for page 106
106 DETERRENCE AND THE DEATH PENALTY
mittee’s review of the time-series and panel studies identified fundamental
deficiencies in this regard.
In the case of the time-series studies, none of them explicitly articulates
a model of sanction risk perceptions. The studies are silent on whether
execution events and their frequency alter perceptions of sanction regimes.
Moreover, the studies do not ask whether the trend lines specified by
researchers correspond to the trend line (if any) perceived by potential
murderers.
Panel studies typically suppose that people who are contemplating mur-
der perceive sanctions risks as subjective probabilities of arrest, conviction,
and execution. Lacking data on these subjective probabilities, researchers
presume that they are somehow based on the observable frequencies of ar-
rest, conviction, and execution.
The fundamental problem is that perceptions of the risk of sanction
are subjective, but researchers have no direct measurements of the percep-
tions of potential murderers. In the absence of data on risk perceptions,
the research practice in the panel studies has been to use publicly available
data on homicides and executions to construct statistics that purport to
measure the objective risk of execution. Then, having done that, many
researchers assume that potential murderers have “rational expectations.”
The word “rational” suggests that potential murderers carefully assess the
risk of execution. What “rational expectations” actually means in practice
is that researchers construct their own measures of execution risk and as-
sume that potential murderers perceive the risk in the same way. However,
the assumption of rational expectations of execution risk has no empirical
foundation. Indeed, it hardly seems credible.
In Chapter 4, we discuss in detail the complications of calculating the
objective risk of execution. One of these complications is that only 15 per-
cent of individuals sentenced to death have actually been executed (since
the resumption of the death penalty in 1976) and that a large fraction of
death sentences are subsequently reversed. Another complication is that
the volume of data on death sentences and executions available for form-
ing perceptions depends on the size of the state. By various measures of
execution risk, Delaware was at least as aggressive as Texas in its use of the
death penalty. However, over the period 1976 to 2000, Delaware sentenced
28 people to death and carried out 11 executions, while Texas sentenced 753
people to death and carried out 231 executions. Still another complication
is that sanction regimes are not stable due to changes in a state’s political
leadership, moratoriums on executions, and legal decisions. Yet another
complication is that there are within-state differences in the risk of execu-
tion due to differences across counties in prosecutorial vigor in the use of the
death penalty and local differences in receptivity to its application.
These many complications make clear that even with a concerted effort
OCR for page 107
107
CHALLENGES TO IDENTIFYING DETERRENT EFFECTS
by careful, conscientious researchers to assemble and analyze relevant data
on death sentences and executions, assessment of the evolving objective
risk of execution facing a potential murderer is a daunting challenge. It is
also clear that perceptions of this risk among potential murderers must at
best be highly impressionistic. To make headway on whether and to what
degree the death penalty affects the behavior of potential murderers, it is
imperative to have knowledge about how their perceptions of execution
risk are formed and then possibly revised on the basis of new information.
RECOMMENDATION: The committee strongly recommends that a
concerted effort be made to research the origins and nature of execu-
tion sanctions risk perceptions specifically and of noncapital sanctions
risks more broadly.
Measurement of Perceptions
The essential task is to measure the perceptions of sanctions risks that
potential murderers actually hold. How might this be done?
One possibility is to take seriously the presumption in the panel stud-
ies that people who are contemplating murder perceive sanctions risks as
subjective probabilities of arrest, conviction, and execution. This possibil-
ity suggests that the risk perceptions of potential murderers be measured
probabilistically.
Researchers have developed considerable experience measuring beliefs
probabilistically in broad population surveys. Manski (2004) reviews the
history in several disciplines, describes the emergence of the modern litera-
ture, summarizes applications, and discusses open issues. Among the major
U.S. platforms for collection of such data, the Health and Retirement Study
(HRS) has periodically elicited probabilistic expectations of retirement,
bequests, and mortality from multiple cohorts of older Americans (see,
e.g., Hurd and McGarry, 1995, 2002; Hurd, Smith, and Zissimopoulos,
2004). The Survey of Economic Expectations (SEE) has asked repeated
population cross sections to state the percent chance that they will lose
their jobs, have health insurance, or be victims of crime in the year ahead
(see, e.g., Dominitz and Manski, 1997; Manski and Straub, 2000). The
National Longitudinal Survey of Youth 1997 has periodically asked young
people about the chance that they will become a parent, be arrested, or
complete schooling (see, e.g., Fischhoff et al., 2000; Lochner, 2007). Ex-
amples of victimization and arrest questions include, “What do you think
is the percent chance that your home will be burglarized in the next year?”
“What do you think is the percent chance that you will be arrested in the
next year?” Researchers have learned from these and other surveys that
most people have little difficulty, once the concept is introduced, in using
OCR for page 108
108 DETERRENCE AND THE DEATH PENALTY
subjective probabilities to express the likelihood they place on future events
relevant to their lives.
However, success in measuring beliefs probabilistically within the gen-
eral public does not imply that survey research could similarly measure
the sanction risk perceptions of potential murderers. A major issue when
initiating study of this type is to obtain data from the relevant population,
in this case, the population of potential murderers. Theoretically, most
people who would be legally eligible to be executed (e.g., are not juveniles
or of very low intelligence) are also physically capable of committing a
murder and thereby are potential murderers. The reality, however, is that
the probability of most people committing a murder is so small that as a
practical matter it can be treated as zero. Even the probabilities of people
committing other serious crimes, such as robbery and burglary, while likely
greater, are still extremely small. Thus, when using the term “potential
murderer,” one means that part of population with a non-negligible risk of
committing murder.
Thus, the first step and an important prerequisite for a program of
research on sanction risk perceptions is to define the relevant population of
potential murderers and, more generally, potential criminals. Such a defini-
tion will be required to devise cost-effective sampling strategies for inter-
viewing people with nontrivial risks of committing crimes. We expect that
one important segment of the relevant population is people with criminal
records. The correlation between past and future offending is among the
best documented empirical regularities in criminology (National Research
Council, 1986; West and Farrington, 1973; Wolfgang, 1958). In the case
of murder, for example, Cook, Ludwig, and Braga (2005) found that 43
percent of murderers in Illinois had a felony conviction.
Some may question the feasibility of collecting data on the sanction risk
perceptions and criminal behavior of individuals with prior histories of seri-
ous crimes, especially if subjects are repeatedly interviewed for the purpose
of obtaining longitudinal data. Longitudinal data are useful to study how
offending experience and external events, such as police crackdowns or
policy changes, affect sanction risk perceptions. However, experience dem-
onstrates that, with sufficient diligence, it is feasible to collect longitudinal
data on highly crime-prone people.
A leading example is the Pathways to Desistance Project (Mulvey,
2011), a two-site longitudinal study of desistance from crime among seri-
ous adolescent offenders. The project recruited 1,354 adolescents from the
Philadelphia and Phoenix juvenile and adult court systems who had been
adjudicated as delinquent or found guilty of a serious felony and were 14
to 17 years old at the time that they committed the offense. For the first 4
years of the study, interviews were conducted at 6-month intervals and for
the next 3 years the interviews were annual. The retention rate was quite
OCR for page 109
109
CHALLENGES TO IDENTIFYING DETERRENT EFFECTS
high, with 87 percent of the subjects interviewed in at least 8 of the 10
interview cycles. Respondents were asked about their perceptions of sanc-
tions risks, among other things. The success of this project indicates that
collection of data on sanction risk perceptions from crime-prone popula-
tions is feasible with a sustained commitment among a cadre of researchers
and with the availability of funding.
Apel (in press) reviews the existing research that measures perceptions
of sanction risks. Although there have been a scattering of suggestive stud-
ies, there has not yet been systematic large-scale research on the subject.
Moreover, there has been no research at all on the specific question of per-
ceptions of the sanction risk associated with commission of murder.
With so much to learn, we think it prudent for research to proceed
sequentially. A good beginning would be small-scale studies that include
one-on-one cognitive interviews with respondents in the relevant popula-
tion of potential murderers. These interviews, taking the form of structured
conversations, would explore the feasibility and usefulness of probabilistic
and other modes of questioning about sanction risk perception. The lessons
learned from this exploratory research would inform the design of larger
studies, the aim being to eventually develop a program of survey research
that would regularly measure the perceptions of the sanction risk held by
potential murderers and by potential criminals more generally.
The committee is not confident that measurement of the sanctions risk
perceptions of potential murderers can succeed in producing information
useful to the study of deterrence, but one cannot be sure unless the effort is
made. As demonstrated by the discussion in Chapters 4 and 5, the alterna-
tive of continuing to make unfounded assumptions about these perceptions
is not useful. Measurement of sanction risk perceptions may enable deter-
rence research to make progress that thus far has not been possible in the
absence of data.
The committee is more optimistic about the feasibility and usefulness
of measuring perceptions of sanctions risks among potential criminals more
broadly. This greater optimism has two bases. First, homicide is the least
frequent of the crimes included in the “Part 1-Crime Index” of the Federal
Bureau of Investigation (FBI), which also includes rape, robbery, aggravated
assault, burglary, larceny, and auto theft. More people commit all the other
crimes than commit homicides. Thus, it will probably be easier to survey
sizable numbers of potential perpetrators of these crimes than of potential
murderers. The National Survey of Youth, for example, already surveys
youth and young adults about their involvement in such crimes as theft,
selling drugs, and assault.
Second, perpetrators who are apprehended for crimes less serious than
murder are far less likely to receive lengthy prison sentences, particularly if
they are juveniles. Thus, these people have more opportunity to learn about
OCR for page 110
110 DETERRENCE AND THE DEATH PENALTY
sanction risk on the basis of personal experience, a source of information
that may be vital to formation of sanction risk perceptions.
Inference on Perceptions from Homicide Rates Following Executions
As a complement to research that directly measures perceptions, some
committee members believe that study of homicide rates immediately fol-
lowing execution events might also provide useful evidence of the percep-
tions of potential murderers. As discussed in Chapter 5, the time-series
research has largely been devoted to the question of whether homicide
rates change in the immediate aftermath of an execution. For the reasons
detailed in that chapter, the committee concluded that existing studies were
not informative about whether capital punishment affects homicide rates,
in part because of the absence of any measure of perceptions.
The committee considered at length whether future research on execu-
tion events, if properly conducted, might be informative about whether
homicide rates, at least in the short term, are responsive to execution events.
We concluded that at best the information to be gleaned from this type of
research would be limited and fall far short of establishing whether capital
punishment increases, decreases, or has no effect on homicide rates. Even if
a short-term impact could be established, it would be difficult to determine
whether homicides were actually prevented or simply displaced in time.
More fundamentally, execution event studies cannot speak to the question
of whether and how the state’s overall sanction regime affects the homicide
rate. For example, a null finding from an event study would leave open the
possibility that a death penalty regime had a deterrent effect relative to a
regime that precluded the death penalty or more narrowly prescribed its
applicability. It is important to note that any one execution would only
have a deterrent effect if it changed potential murderers’ perceptions of the
likelihood of an execution, which is not necessarily the case.
Acknowledging these limitations, some committee members nonethe-
less argue that if a well-done event study did produce evidence of an
effect—whether positive or negative and no matter how temporary—that
result would be of considerable interest. It would demonstrate that po-
tential murderers as a group are actually paying attention to the state’s
actions and are influenced by them. In short, it would confirm a threshold
condition for there to be a deterrent or brutalization effect and invite fur-
ther inquiry. Other committee members are not convinced of the value of
establishing this threshold condition or are not convinced that any study
of this sort could make a convincing case that it had isolated a causal ef-
fect of executions.
OCR for page 111
111
CHALLENGES TO IDENTIFYING DETERRENT EFFECTS
IDENTIFYING EFFECTS:
FEEDBACKS AND UNOBSERVED CONFOUNDERS
Even with better data and information on sanction regimes and per-
ceptions of sanction risks, formidable difficulties remain to understanding
the impact of the death penalty on homicide. With only observational
(nonexperimental) data on capital punishment and homicides, researchers
must face the fundamental problem that the data alone cannot reveal the
counterfactual question of interest: What would have happened if the death
penalty not been applied in a “treatment” state or if the death penalty had
been applied in a “control” state? Although this counterfactual-outcomes
problem is common to all observational studies of cause and effect, it has
long been understood to be particularly problematic for understanding the
deterrent effect of the death penalty. A capital punishment regime evolves
over time as a result, among other things, of a complex interplay of crime
trends, social norms, criminal justice budgets, and election results. This
context makes it very difficult to identify the effects of the capital sanction
regime alone.
To better understand these issues, we highlight three related identi-
fication problems that complicate efforts to draw credible inferences on
the effect of capital punishment on homicides. The first, referred to as a
feedback effect, arises when homicide rates may directly affect the capital
sanction regime. The second, referred to as the omitted variable problem,
arises when variables that are jointly associated with the sanction regime
and homicide rate are either unknown or unobserved. The third, referred
to as an equilibrium effect, arises when the capital sanction regime may
directly affect other aspects of the criminal justice system, including, most
notably, noncapital sanction policies.
Feedback Effects
Deterrence research conducted in the early 1970s (Carr-Hill and Stern,
1973; Ehrlich, 1975; Sjoquist, 1973) recognized the possibility of feedbacks
or simultaneity whereby crime rates may affect the sanction risk and sever-
ity even as the sanction risk and severity may affect crime rates. The nature
of such feedbacks is not well understood, but there are good reasons for
believing that feedbacks are present and may be substantial.
To illustrate the problem, suppose that in a particular state during a
particular year there is an exogenous increase in the rate of homicide. If,
given the additional workload and resulting strain on resources, district
attorneys were more reluctant to pursue the death penalty, a continu-
ing upward trend in homicides would appear to show that a reduction
in the probability of a death sentence is associated with an increase in
OCR for page 114
114 DETERRENCE AND THE DEATH PENALTY
Knowledge of the entire system, however, is not a necessary require-
ment for learning about the overall impact of the capital sanction regime.
For some questions, the effects of the death penalty on sentence bargain-
ing and on administrative resource constraints are an intrinsic part of the
mechanism by which a capital regime affects murder rates. Consider, for
example, a case in which a judicial ruling terminates the use of the death
penalty for some category of homicides. It would be of considerable interest
to have a reliable estimate of the overall effect of this reform on the murder
rate, even if it is not possible to distinguish among the various mechanisms
(reduction in the probability of a death sentence, weaker bargaining posi-
tion by the district attorney, or increased court resources available for the
average case) that led to that effect. Still, this sort of “black box” estimate
is not satisfactory if the goal is to estimate the effect of the threat of execu-
tion, in part because the ancillary effects of the administration of the death
penalty can be generated by other means, such as changes in court budgets.
Is a more reliable approach to identifying the deterrent effect of capital
punishment possible? Part of the solution may be to develop a better un-
derstanding of the factors that affect sanction regimes, including possible
feedbacks from homicide or other crime patterns. The earlier National
Research Council report (1978, p. 47) observed: “Knowledge of the effect
of crime on the behavior of the criminal justice system is still extremely
limited.” This conclusion is still true today, 30 years later. The 1978 report
went on to observe: “While the seeming dearth of untainted identification
restrictions may reflect the fact that none exist, it is certainly as likely that
it simply reflects our ignorance of the determinants of sanctions” (p. 48).
Three decades later this committee observes that both of these assessments
apply to contemporary research on deterrence.
As noted above, the 1978 report urged more research on the sanction-
generation process for the purpose of accumulating a knowledge base that
might reveal approaches to plausible identification. Although knowledge of
the sanction-generation process is not required for identification of overall
effects of certain relevant regime changes, that knowledge may be useful
in determining the validity of a proposed identification method. Also, as
a practical matter, some committee members believe that without better
knowledge of sanction generation, the prospects for credible identifica-
tion are small. Committee members holding this perspective argue that a
deeper institutional and theoretical knowledge of sanction process would
materially increase the chances of researchers’ becoming aware of credible
sources of identification and that without such knowledge the chances for
credible identification are remote. Other committee members are less pes-
simistic that a chance event or insight might provide a basis for credible
identification.
However credible identification might ultimately be achieved, the com-
OCR for page 115
115
CHALLENGES TO IDENTIFYING DETERRENT EFFECTS
mittee fully endorses another observation from the earlier report (National
Research Council, 1978, p. 49):
It must be noted, however, that identification restrictions cannot be manu-
factured. If the process generating the data is truly one that leaves the
crime function unidentified, then persistent attempts to produce identifying
restrictions because of the desire to estimate the deterrent effect will only
produce different kinds of error. Even if all such attempts found a “deter-
rent” effect, no conclusion would be warranted unless some of them used
validly based identification restrictions.
ADDRESSING MODEL UNCERTAINTY
WITH WEAKER ASSUMPTIONS
The persistent problems that researchers have had in providing mean-
ingful answers about the deterrent effect of capital punishment is unsur-
prising once one recognizes that this body of empirical research rests on
strong and unverified assumptions. Although, in practice, researchers often
recognize and acknowledge that their assumptions may not hold, they are
defended as necessary to provide meaningful answers and in order to make
inferences. But the use of strong assumptions hides the problem that very
little is understood about the process that may link capital punishment to
future crimes.
The different findings in the deterrence research reflect different choices
of assumptions, most of which cannot be supported by strong a priori
justifications. As documented throughout this report, many of the assump-
tions used in the research on the deterrent effect of capital punishment are
not credible. Furthermore, the state of social science knowledge does not
support a unique model that can be used to identify the effects of capital
punishment under the current U.S. sanction regime or to permit the evalu-
ation of deterrence under alternative regimes. The study of deterrence is
plagued by model uncertainty.
The failure of the existing research to address the issue of model un-
certainly is evident in the debate initiated by Donohue and Wolfers (2005),
who challenged claims of deterrence by a broad set of researchers. Much of
their challenge involved demonstrations of how small changes in the models
used in the various studies led to very different estimates of deterrence ef-
fects, in some case changing from positive to negative or vice versa, and in
others eliminating statistical significance. Some of their exercises altered the
set of observations over which the analysis had been conducted; in other
cases they changed the choice of control or instrumental variables.
Although Donohue and Wolfers provide useful evidence of the sensitiv-
ity of many claims of deterrence to model assumptions, their demonstra-
tion begs the question of how to adjudicate their findings relative to the
OCR for page 116
116 DETERRENCE AND THE DEATH PENALTY
papers they critique. This may be seen in two of the rejoinders that have
been written to their study. Dezhbakhsh and Rubin (2011) and Mocan and
Gittings (2010) provide a large number of modifications of their baseline
homicide regressions and argue that deterrence effects generally appear in
them. However, they fail to provide any guidance as to what is learned
from the specifications that are inconsistent with their claim of evidence
of deterrence. Rather, the authors’ claims are based on ad hoc choices of
alternative model specifications; there is no systematic construction of the
models from which to draw inferences. That changes in a given statistical
model change the output of the model is hardly unique to the studies of
capital punishment and deterrence literature. The problem is that there have
been almost no serious attempts to reconcile the many different findings
reported in the research.
Given this existing uncertainty, how might research proceed? Certainly,
research aimed at reducing model uncertainty would be useful. To that
end, the committee proposed, above, developing data and research on
sanction regimes and perceptions of sanction risk. Another complementary
and potentially useful approach would be to explicitly account for model
uncertainty when drawing inferences on the impact of capital punishment.
Rather than continue with the conventional practice of assuming whatever
it takes to achieve point identification, and then providing ad hoc justifica-
tions for particular sets of assumptions to justify a given model, deterrent
studies might instead consider what can be learned when explicitly rec-
ognizing model uncertainty. Although the resulting inferences may reflect
a certain degree of ambiguity about the effects of capital punishment on
homicides, those inferences will necessarily possess greater credibility.
To explore the idea of addressing model uncertainty, the committee
commissioned papers illustrating application of two complementary re-
search paradigms—the model averaging approach and the partial identifi-
cation approach.
Model Averaging
Model averaging, though based on earlier work (Bates and Granger,
1969; Leamer, 1978), developed theoretically, algorithmically, and as an
applied technique in the mid-1990s (examples include Chatfield, 1995;
Draper, 1995; Draper et al., 1993; Raftery, Madigan, and Hoeting, 1997).
The model averaging approach constructs a probability distribution for a
range of estimates of the deterrent effect of capital punishment, and the
researcher constructs this distribution to reflect the researcher’s own or
others experts’ prior beliefs about the probability that a given model is
valid. By asking what can be learned by combining the information ob-
tained across a wide range of models, model averaging methods provide
OCR for page 117
117
CHALLENGES TO IDENTIFYING DETERRENT EFFECTS
a natural way to make empirical claims robust to the details of uncertain
model specifications.
This technique has recently been used in two studies of capital punish-
ment: Cohen-Cole et al. (2009) and Durlauf, Fu, and Navarro (in press).
These studies apply the modeling average approach to various specifications
that have appeared in the research on capital punishment and deterrence.
Cohen-Cole et al. (2009) use this method to adjudicate the different find-
ings of Dezhbakhsh, Rubin, Shepherd (2003) and Donohue and Wolfers
(2005). Durlauf, Fu, and Navarro (in press), whose paper was written for
this committee, consider a range of models based on alternative substantive
assumptions that have appeared in the research, including, for example,
how to measure subjective arrest, sentencing, and execution probabilities
and whether the deterrent effect of capital punishment differs across states.
These two papers aim to understand how different assumptions matter
and whether differences in assumptions render deterrence estimates fragile.
In both papers, the researchers find that model uncertainty swamps the
informational content about deterrent effects. That is, after accounting for
the modeling uncertainty, the empirical evidence does not reveal whether
capital punishment increases or decreases homicides.
As an example of this result, consider the Cohen-Cole et al. (2009)
analysis of the models in Dezhbakhsh, Rubin, and Shepherd (2003) and
Donohue and Wolfers (2005). Dezhbakhsh, Rubin, and Shepherd (2003)
report, under their preferred specification, a statistically significant point
estimate of 18 lives saved for each execution. However, when all of the
different specifications spanned in the two papers are given probability
weights, Cohen-Cole et al. estimate an approximate 95 percent confidence
interval on the number of lives saved per execution of [–24, 124]: see Fig-
ure 6-1, which is from Cohen-Cole et al. The figure illustrates the model
uncertainty by providing a weighted histogram of the estimated net lives
saved for all of the models considered. For the case illustrated in this histo-
gram, the posterior probability for the models with point estimates suggest
that deterrence is 72 percent, but there is substantial bunching around 0,
the individual estimates vary widely, and there is a nontrivial probability
on models that suggest a large increase in homicides associated with ex-
ecutions (a probability 0.15 of point estimates of 20 or more homicides).
Thus, the heterogeneity of the model-specific estimates makes it impossible
to draw strong qualitative conclusions about the deterrent effect of capital
punishment.
The model averaging approach provides a formal and elegant Bayesian
method for incorporating uncertainty about the correct modeling assump-
tions into inferential methods. This approach can be effectively used to
illustrate the importance of different assumptions and the fragility of the
estimates to these assumptions, as is done in Cohen-Cole et al. (2009) and
OCR for page 118
118 DETERRENCE AND THE DEATH PENALTY
Net Lives Saved: Weighted Histogram
0.4
DW Estimate DRS Estimate
0.35
0.3
0.25
0.2
0.15
0.1
0.05
0
-200 -100 0 100 200 300 400
FIGURE 6-1 Weighted histogram of the net lives saved by the death penalty.
NOTES: The figure includes models for each of the DRS (Dezhbakhsh, Rubin,
and Shepherd, 2003) categories. The weights are the posterior model probabilities
(Bayes factors). The DRS and DW (Donohue and Wolfers, 2005) lines correspond to
the individual model from each with the largest and smallest number of lives saved,
respectively. The unweighted histogram is similar.
SOURCE: Cohen-Cole et al. (2009, Figure 1). Used with permission.
R02175
Figure 6-1
vector, editable
Durlauf, Fu, and Navarro (in press). The approach depends on research-
ers’ specifications of the model space and original that model
obtained from prior oversource space, over
(Cohen-Cole et al, 2007)
which there may be disagreement. Such disagreement should not obscure an
essential strength of the model averaging approach: model averaging pro-
vides an approach for systematically exploring sensitivity over an explicitly
defined model space.
Ultimately, this approach might also be used to infer the effect of the
death penalty on homicides. However, for this purpose, a key challenge
would be selecting a set of models to include in the averaging and provid-
ing a prior probability distribution over this set that is plausible. The ap-
proach presumes that the range of models included in the averaging routine
includes the correct model that accurately describes the real world and,
OCR for page 119
119
CHALLENGES TO IDENTIFYING DETERRENT EFFECTS
moreover, that the researcher can provide informed prior beliefs about the
probability that each model is valid. In the context of the research on capi-
tal punishment, we have found no reason to believe that the existing range
of point-identified models includes the correct one, and there is currently
little basis for assigning probabilities to the correctness of each model in the
literature. As discussed in Chapter 4, the committee did not find the instru-
mental variables used in the existing research to be credible. If the existing
models are all invalid, using the modeling averaging approach to produce
interpretable deterrence estimates can be problematic.2 With uncertainty
about the model space and the prior probabilities, either research efforts
to construct informative priors or research showing the sensitivity of the
posterior to different prior distributions may be useful.
Partial Identification
Partial identification methods provide an alternative approach for re-
ducing the dependence of claims of a deterrence effect on arbitrary assump-
tions. Rather than start with a particular set of point-identified models and
prior beliefs about the probability that each model is valid, both as defined
by the researcher, one might instead begin by directly considering what
can be inferred under a set of weak assumptions that may possess greater
credibility. A natural starting point, for example, is to examine what can be
learned in the absence of any assumptions. What do the data alone reveal?
Under these weaker assumptions, deterrent effects may not be point iden-
tified, but they will be partially identified, with bounds rather than point
estimates. Thus, the partial identification approach formalizes the inherent
tradeoff between the strength of the maintained assumptions and the cred-
ibility of inferences (see Manski, 2003).
The partial identification methodology has been developed and applied
over the past 20 years, beginning with Manski (1989, 1990). In an early
application to criminal justice policy, Manski and Nagin (1998) studied
sentencing and recidivism of juvenile offenders in the state of Utah and
demonstrated how partial identification can be used to produce more cred-
ible inferences than had previously been produced. Youth in Utah faced a
policy that gave judges the discretion to order varying sentences. Using this
discretion, judges sentenced some offenders to residential confinement and
sentenced other offenders to no confinement. A policy question of potential
2 The Cohen-Cole et al. exercise (2009) was narrow in that it considered the smallest model
space one could generate around the different assumptions in Donohue and Wolfers (2005)
and Dezhbakhsh, Rubin, and Shepherd (2003). One can easily argue that for a full model av-
eraging analysis, other models warrant a priori consideration. However, one could also argue
that some of the models considered in Cohen-Cole et al. should not have been included, given
a prior probability of 0.
OCR for page 120
120 DETERRENCE AND THE DEATH PENALTY
interest was to compare recidivism under that policy with the recidivism
that would occur under a policy proposal that removed judicial discretion
and instead mandated that all offenders be sentenced to confinement. The
study showed how bounds of varying width on the existing treatment effect
which allows judges’ discretion could be achieved by combining data on
outcomes under the status quo with relatively weak assumptions regarding
the manner in which (1) judges have made sentencing decisions and (2)
criminality was affected by sentencing.
More recently, in a paper written for this committee, Manski and Pepper
(in press) illustrate the partial identification approach in a relatively simple
setting by examining the effect of death penalty statutes on the national
homicide rate (per 100,000) over 2 years, 1975-1977: 1975 was the last
full year of the federal moratorium on death penalty, and 1977 was the
first full year after the moratorium was lifted. In 1975, the death penalty
was illegal throughout the country; and in 1977, 32 states had legal death
penalty statutes. Over this 2-year period, homicide rates in the 32 states
that had adopted a death penalty statute in 1977 decreased by 0.6; in the
remaining states, the homicide rates decreased by 1.1. It has been common
in the relevant research to report the difference-in-difference estimate, which
in this case is 0.5 (–0.6 + 1.1), as a point estimate of the effect of capital
punishment on the national homicide rate. This interpretation suggests that
the death penalty increases crime, but Manski and Pepper (in press) show
that this difference-in-difference form only point identifies the impact of the
death penalty under a number of strong assumptions, most notably that the
effect is assumed to be homogeneous across states and dates. Under weaker
assumptions that allow the deterrent effect to vary across states, the average
effect of the death penalty is only partially identified, and it was found to lie
in the interval [–1.9, 8.3]. Under still weaker assumptions under which the
effect of the death penalty is allowed to vary over time, the bounds widen
further. Thus, under these weaker models, the average treatment effect of
capital punishment is bounded, but the data do not identify whether the
death penalty increases or decreases homicides.
The committee does not endorse the specific findings of the recent
studies applying the model averaging or partial identification approaches.
These studies are largely illustrative and do not address many of the key
problems identified throughout this report. Most notably, they do not
define the counterfactual sanction regime and do not address the issue of
how potential murderers perceive sanction risks. Still, these studies serve
as a starting point for future research that might inform the debate on the
death penalty. Rather than imposing the strong but unsupported assump-
tions required to identify the effect of capital punishment on homicides in a
single model or an ad hoc set of similar models, approaches that explicitly
account for model uncertainty may provide a constructive way for research
OCR for page 121
121
CHALLENGES TO IDENTIFYING DETERRENT EFFECTS
to provide credible albeit incomplete answers. The basic insight is that with
model uncertainty, the identification of deterrent effects need not be an all-
or-nothing undertaking: the available data and credible assumptions may
yield partial conclusions.
Some people may find partial conclusions unappealing and be tempted
to impose strong assumptions in order to obtain definitive answers. We
caution against this reaction. Imposing strong but untenable assumptions
cannot truly resolve inferential problems. Rather, it simply replaces the
modeling uncertainty with uncertainty associated with the underlying as-
sumptions. We have seen this repeatedly in the literature on the death
penalty. The earlier Panel on Research on Deterrent and Incapacitative Ef-
fects recognized this when it concluded (National Research Council, 1978,
p. 63) “research on this topic is not likely to produce findings that will or
should have much influence on policymakers.” Today, more than 30 years
later, perhaps the primary lesson learned from the latest round of empirical
research on the deterrent effect of the death penalty is that researchers and
policy makers must cope with ambiguity. Explicitly recognizing and ac-
counting for this uncertainty seems like the only hope of moving forward.
RECOMMENDATION: The committee recommends further inves-
tigation of the effects of capital punishment using assumptions that
are weaker and more credible than those that have traditionally been
invoked by empirical researchers.
REFERENCES
Alarcón, A.L., and Mitchell, P.M. (2011). Executing the will of the voters?: A roadmap
to mend or end the California legislature’s multi-billion-dollar death penalty debacle.
Loyola of Los Angeles Law Review, 44(Special), S41-S224.
Apel, R. (in press). Sanctions, perceptions, and crime: Implications for criminal deterrence.
Submitted to Journal of Quantitative Criminology, 28.
Bates, J.M., and Granger, C.W.J. (1969). The combination of forecasts. Operational Research
Quarterly, 20(4), 451-468.
California Commission on the Fair Administration of Justice. (2008). Report and Recommen-
dations on the Administration of the Death Penalty in California. Sacramento: Author.
Carr-Hill, R.A., and Stern, N.H. (1979). Crime, the Police and Criminal Statistics: An Analysis
of Official Statistics for England and Wales Using Econometric Methods. New York:
Academic Press.
Chatfield, C. (1995). Model uncertainty, data mining and statistical inference. Journal of the
Royal Statistical Society Series A-Statistics in Society, 158(3), 419-466.
Cohen-Cole, E., Durlauf, S., Fagan, J., and Nagin, D. (2009). Model uncertainty and the deter-
rent effect of capital punishment. American Law and Economics Review, 11(2), 335-369.
Cook, P.J. (2009). Potential savings from abolition of the death penalty in North Carolina.
American Law and Economics Review, 11(2), 498-529.
Cook, P.J., Ludwig, J., and Braga, A.A. (2005). Criminal records of homicide offenders. The
Journal of the American Medical Association, 294(5), 598-601.
OCR for page 122
122 DETERRENCE AND THE DEATH PENALTY
Dezhbakhsh, H., and Rubin, P.H. (2011). From the “econometrics of capital punishment” to
the “capital punishment” of econometrics: On the use and abuse of sensitivity analysis.
Applied Economics, 43(25), 3,655-3,670.
Dezhbakhsh, H., Rubin, P.H., and Shepherd, J.M. (2003). Does capital punishment have a
deterrent effect? New evidence from postmoratorium panel data. American Law and
Economics Review, 5, 344-376.
Dominitz, J., and Manski, C.F. (1997). Perceptions of economic insecurity - evidence from the
survey of economic expectations. Public Opinion Quarterly, 61(2), 261-287.
Donohue, J.J., and Wolfers, J. (2005). Uses and abuses of empirical evidence in the death
penalty debate. Stanford Law Review, 58(3), 791-845.
Draper, D. (1995). Assessment and propagation of model uncertainty. Journal of the Royal
Statistical Society Series B-Methodological, 57(1), 45-97.
Draper, D., Hodges, J.S., Mallows, C.L., and Pregibon, D. (1993). Exchangeability and data-
analysis. Journal of the Royal Statistical Society Series A-Statistics in Society, 156(1),
9-37.
Durlauf, S., Fu, C., and Navarro, S. (in press). Capital punishment and deterrence: Under-
standing disparate results. Submitted to Journal of Quantitative Criminology, 28.
Ehrlich, I. (1975). Deterrent effect of capital punishment—Question of life and death. Ameri-
can Economic Review, 65(3), 397-417.
Fischhoff, B., Parker, A.M., De Bruin, W.B., Downs, J., Palmgren, C., Dawes, R., and Manski,
C.F. (2000). Teen expectations for significant life events. Public Opinion Quarterly, 64(2),
189-205.
Hurd, M., and McGarry, K. (1995). Evaluation of the subjective probabilities of survival in the
Health and Retirement Study. Journal of Human Resources, 30(5), S268-S292.
Hurd, M.D., and McGarry, K. (2002). The predictive validity of subjective probabilities of
survival. The Economic Journal, 112(482), 966-985.
Hurd, M.D., Smith, J.P., and Zissimopoulos, J.M. (2004). The effects of subjective survival
on retirement and social security claiming. Journal of Applied Econometrics, 19(6),
761-775.
Kuziemko, I. (2006). Does the threat of the death penalty affect plea bargaining in murder
cases? Evidence from New York’s 1995 reinstatement of capital punishment. American
Law and Economics Review, 8(1), 116-142.
Leamer, E.E. (1978). Specification Searches: Ad Hoc Inference with Nonexperimental Data.
New York: Wiley.
Lochner, L. (2007). Individual perceptions of the criminal justice system. American Economic
Review, 97(1), 444-460.
Manski, C.F. (1989). Anatomy of the selection problem. Journal of Human Resources, 24(3),
343-360.
Manski, C.F. (1990). Nonparametric bounds on treatment effects. American Economic Re-
view, 80(2), 319-323.
Manski, C.F. (2003). Partial Identification of Probability Distributions. New York: Springer.
Manski, C.F. (2004). Measuring expectations. Econometrica, 72(5), 1,329-1,376.
Manski, C.F., and Nagin, D.S. (1998). Bounding disagreements about treatment effects: A case
study of sentencing and recidivism. Sociological Methodology, 28(1), 99-137.
Manski, C.F., and Pepper, J. (in press). Deterrence and the death penalty: Partial identifi-
cation analysis using repeated cross sections. Submitted to Journal of Quantitative
Criminology, 28.
Manski, C.F., and Straub, J.D. (2000). Worker perceptions of job insecurity in the mid-
1990s—Evidence from the survey of economic expectations. Journal of Human Re-
sources, 35(3), 447-479.
OCR for page 123
123
CHALLENGES TO IDENTIFYING DETERRENT EFFECTS
Mocan, N., and Gittings, K. (2010). The impact of incentives on human behavior: Can we
make it disappear? The case of the death penalty. In R.E.S. Di Tella and E. Schargrodsky
(Eds.), The Economics of Crime: Lessons for and from Latin America (pp. 379-420).
Chicago: University of Chicago Press.
Mulvey, E.P. (2011). Highlights from Pathways to Desistance: A Longitudinal Study of Seri-
ous Adolescent Offenders. Juvenile Justice Fact Sheet. Washington, DC: U.S. Department
of Justice.
National Research Council. (1978). Deterrence and Incapacitation: Estimating the Effects of
Criminal Sanctions on Crime Rates. Panel on Research on Deterrent and Incapacitative
Effects. A. Blumstein, J. Cohen, and D. Nagin (Eds.), Committee on Research on Law
Enforcement and Criminal Justice. Assembly of Behavioral and Social Sciences. Wash-
ington, DC: National Academy Press.
National Research Council. (1986). Criminal Careers and “Career Criminals.” Panel on Re-
search on Criminal Careers, A. Blumstein, J. Cohen, J.A. Roth, and C.A. Visher (Eds.),
Committee on Research on Law Enforcement and the Administration of Justice. Com-
mission on Behavioral and Social Sciences and Education. Washington, DC: National
Academy Press.
Raftery, A.E., Madigan, D., and Hoeting, J.A. (1997). Bayesian model averaging for linear
regression models. Journal of the American Statistical Association, 92(437), 179-191.
Roman, J.K., Chalfin, A.J., and Knight, C.R. (2009). Reassessing the cost of the death penalty
using quasi-experimental methods: Evidence from Maryland. American Law and Eco-
nomics Review, 11(2), 530-574.
Sjoquist, D.L. (1973). Property crime and economic behavior: Some empirical results. Ameri-
can Economic Review, 63(3), 439-446.
West, D.J., and Farrington, D.P. (1973). Who Becomes Delinquent? Second Report of the
Cambridge Study in Delinquent Development. London: Heinemann Educational.
Wolfgang, M.E. (1958). Patterns in Criminal Homicide. Philadelphia: University of Pennsyl-
vania Press.
OCR for page 124
