**Suggested Citation:**"7 On the Use of Aggregate Crime Regressions in Policy Evaluation--Steven N. Durlauf, Salvador Navarro, and David A. Rivers." National Research Council. 2008.

*Understanding Crime Trends: Workshop Report*. Washington, DC: The National Academies Press. doi: 10.17226/12472.

**Suggested Citation:**"7 On the Use of Aggregate Crime Regressions in Policy Evaluation--Steven N. Durlauf, Salvador Navarro, and David A. Rivers." National Research Council. 2008.

*Understanding Crime Trends: Workshop Report*. Washington, DC: The National Academies Press. doi: 10.17226/12472.

**Suggested Citation:**"7 On the Use of Aggregate Crime Regressions in Policy Evaluation--Steven N. Durlauf, Salvador Navarro, and David A. Rivers." National Research Council. 2008.

*Understanding Crime Trends: Workshop Report*. Washington, DC: The National Academies Press. doi: 10.17226/12472.

**Suggested Citation:**"7 On the Use of Aggregate Crime Regressions in Policy Evaluation--Steven N. Durlauf, Salvador Navarro, and David A. Rivers." National Research Council. 2008.

*Understanding Crime Trends: Workshop Report*. Washington, DC: The National Academies Press. doi: 10.17226/12472.

**Suggested Citation:**"7 On the Use of Aggregate Crime Regressions in Policy Evaluation--Steven N. Durlauf, Salvador Navarro, and David A. Rivers." National Research Council. 2008.

*Understanding Crime Trends: Workshop Report*. Washington, DC: The National Academies Press. doi: 10.17226/12472.

**Suggested Citation:**"7 On the Use of Aggregate Crime Regressions in Policy Evaluation--Steven N. Durlauf, Salvador Navarro, and David A. Rivers." National Research Council. 2008.

*Understanding Crime Trends: Workshop Report*. Washington, DC: The National Academies Press. doi: 10.17226/12472.

**Suggested Citation:**"7 On the Use of Aggregate Crime Regressions in Policy Evaluation--Steven N. Durlauf, Salvador Navarro, and David A. Rivers." National Research Council. 2008.

*Understanding Crime Trends: Workshop Report*. Washington, DC: The National Academies Press. doi: 10.17226/12472.

**Suggested Citation:**"7 On the Use of Aggregate Crime Regressions in Policy Evaluation--Steven N. Durlauf, Salvador Navarro, and David A. Rivers." National Research Council. 2008.

*Understanding Crime Trends: Workshop Report*. Washington, DC: The National Academies Press. doi: 10.17226/12472.

**Suggested Citation:**"7 On the Use of Aggregate Crime Regressions in Policy Evaluation--Steven N. Durlauf, Salvador Navarro, and David A. Rivers." National Research Council. 2008.

*Understanding Crime Trends: Workshop Report*. Washington, DC: The National Academies Press. doi: 10.17226/12472.

**Suggested Citation:**"7 On the Use of Aggregate Crime Regressions in Policy Evaluation--Steven N. Durlauf, Salvador Navarro, and David A. Rivers." National Research Council. 2008.

*Understanding Crime Trends: Workshop Report*. Washington, DC: The National Academies Press. doi: 10.17226/12472.

**Suggested Citation:**"7 On the Use of Aggregate Crime Regressions in Policy Evaluation--Steven N. Durlauf, Salvador Navarro, and David A. Rivers." National Research Council. 2008.

*Understanding Crime Trends: Workshop Report*. Washington, DC: The National Academies Press. doi: 10.17226/12472.

**Suggested Citation:**"7 On the Use of Aggregate Crime Regressions in Policy Evaluation--Steven N. Durlauf, Salvador Navarro, and David A. Rivers." National Research Council. 2008.

*Understanding Crime Trends: Workshop Report*. Washington, DC: The National Academies Press. doi: 10.17226/12472.

**Suggested Citation:**"7 On the Use of Aggregate Crime Regressions in Policy Evaluation--Steven N. Durlauf, Salvador Navarro, and David A. Rivers." National Research Council. 2008.

*Understanding Crime Trends: Workshop Report*. Washington, DC: The National Academies Press. doi: 10.17226/12472.

**Suggested Citation:**"7 On the Use of Aggregate Crime Regressions in Policy Evaluation--Steven N. Durlauf, Salvador Navarro, and David A. Rivers." National Research Council. 2008.

*Understanding Crime Trends: Workshop Report*. Washington, DC: The National Academies Press. doi: 10.17226/12472.

**Suggested Citation:**"7 On the Use of Aggregate Crime Regressions in Policy Evaluation--Steven N. Durlauf, Salvador Navarro, and David A. Rivers." National Research Council. 2008.

*Understanding Crime Trends: Workshop Report*. Washington, DC: The National Academies Press. doi: 10.17226/12472.

**Suggested Citation:**"7 On the Use of Aggregate Crime Regressions in Policy Evaluation--Steven N. Durlauf, Salvador Navarro, and David A. Rivers." National Research Council. 2008.

*Understanding Crime Trends: Workshop Report*. Washington, DC: The National Academies Press. doi: 10.17226/12472.

**Suggested Citation:**"7 On the Use of Aggregate Crime Regressions in Policy Evaluation--Steven N. Durlauf, Salvador Navarro, and David A. Rivers." National Research Council. 2008.

*Understanding Crime Trends: Workshop Report*. Washington, DC: The National Academies Press. doi: 10.17226/12472.

**Suggested Citation:**"7 On the Use of Aggregate Crime Regressions in Policy Evaluation--Steven N. Durlauf, Salvador Navarro, and David A. Rivers." National Research Council. 2008.

*Understanding Crime Trends: Workshop Report*. Washington, DC: The National Academies Press. doi: 10.17226/12472.

**Suggested Citation:**"7 On the Use of Aggregate Crime Regressions in Policy Evaluation--Steven N. Durlauf, Salvador Navarro, and David A. Rivers." National Research Council. 2008.

*Understanding Crime Trends: Workshop Report*. Washington, DC: The National Academies Press. doi: 10.17226/12472.

**Suggested Citation:**"7 On the Use of Aggregate Crime Regressions in Policy Evaluation--Steven N. Durlauf, Salvador Navarro, and David A. Rivers." National Research Council. 2008.

*Understanding Crime Trends: Workshop Report*. Washington, DC: The National Academies Press. doi: 10.17226/12472.

**Suggested Citation:**"7 On the Use of Aggregate Crime Regressions in Policy Evaluation--Steven N. Durlauf, Salvador Navarro, and David A. Rivers." National Research Council. 2008.

*Understanding Crime Trends: Workshop Report*. Washington, DC: The National Academies Press. doi: 10.17226/12472.

**Suggested Citation:**"7 On the Use of Aggregate Crime Regressions in Policy Evaluation--Steven N. Durlauf, Salvador Navarro, and David A. Rivers." National Research Council. 2008.

*Understanding Crime Trends: Workshop Report*. Washington, DC: The National Academies Press. doi: 10.17226/12472.

**Suggested Citation:**"7 On the Use of Aggregate Crime Regressions in Policy Evaluation--Steven N. Durlauf, Salvador Navarro, and David A. Rivers." National Research Council. 2008.

*Understanding Crime Trends: Workshop Report*. Washington, DC: The National Academies Press. doi: 10.17226/12472.

**Suggested Citation:**"7 On the Use of Aggregate Crime Regressions in Policy Evaluation--Steven N. Durlauf, Salvador Navarro, and David A. Rivers." National Research Council. 2008.

*Understanding Crime Trends: Workshop Report*. Washington, DC: The National Academies Press. doi: 10.17226/12472.

**Suggested Citation:**"7 On the Use of Aggregate Crime Regressions in Policy Evaluation--Steven N. Durlauf, Salvador Navarro, and David A. Rivers." National Research Council. 2008.

*Understanding Crime Trends: Workshop Report*. Washington, DC: The National Academies Press. doi: 10.17226/12472.

**Suggested Citation:**"7 On the Use of Aggregate Crime Regressions in Policy Evaluation--Steven N. Durlauf, Salvador Navarro, and David A. Rivers." National Research Council. 2008.

*Understanding Crime Trends: Workshop Report*. Washington, DC: The National Academies Press. doi: 10.17226/12472.

**Suggested Citation:**"7 On the Use of Aggregate Crime Regressions in Policy Evaluation--Steven N. Durlauf, Salvador Navarro, and David A. Rivers." National Research Council. 2008.

*Understanding Crime Trends: Workshop Report*. Washington, DC: The National Academies Press. doi: 10.17226/12472.

**Suggested Citation:**"7 On the Use of Aggregate Crime Regressions in Policy Evaluation--Steven N. Durlauf, Salvador Navarro, and David A. Rivers." National Research Council. 2008.

*Understanding Crime Trends: Workshop Report*. Washington, DC: The National Academies Press. doi: 10.17226/12472.

**Suggested Citation:**"7 On the Use of Aggregate Crime Regressions in Policy Evaluation--Steven N. Durlauf, Salvador Navarro, and David A. Rivers." National Research Council. 2008.

*Understanding Crime Trends: Workshop Report*. Washington, DC: The National Academies Press. doi: 10.17226/12472.

**Suggested Citation:**"7 On the Use of Aggregate Crime Regressions in Policy Evaluation--Steven N. Durlauf, Salvador Navarro, and David A. Rivers." National Research Council. 2008.

*Understanding Crime Trends: Workshop Report*. Washington, DC: The National Academies Press. doi: 10.17226/12472.

**Suggested Citation:**"7 On the Use of Aggregate Crime Regressions in Policy Evaluation--Steven N. Durlauf, Salvador Navarro, and David A. Rivers." National Research Council. 2008.

*Understanding Crime Trends: Workshop Report*. Washington, DC: The National Academies Press. doi: 10.17226/12472.

**Suggested Citation:**"7 On the Use of Aggregate Crime Regressions in Policy Evaluation--Steven N. Durlauf, Salvador Navarro, and David A. Rivers." National Research Council. 2008.

*Understanding Crime Trends: Workshop Report*. Washington, DC: The National Academies Press. doi: 10.17226/12472.

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

7 On the Use of Aggregate Crime Regressions in Policy Evaluation Steven N. Durlauf, Salvador Navarro, and David A. Rivers Despite recent efforts to employ microeconomic data and natural experiments, aggregate crime regressions continue to play a significant role in criminological analyses. One use of these regressions is predictive, as illustrated by the papers in this volume that employ aggregate crime trends regressionsâBaumer (Chapter 5) and Pepper (Chapter 6). A second use involves policy evaluation: Prominent and controversial cases include the deterrent effect of shall-issue concealed weapons legislation (e.g., Ayres and Donohue, 2003; Black and Nagin, 1998; Lott, 1998; Lott and Mustard, 1997; Plassmann and Whitley, 2003) and the deterrent effect of capital punishment (e.g., Dezhbakhsh, Rubin, and Shepherd, 2003; Donohue and Wolfers, 2005). These uses are interrelated, as is evident from the effort to evaluate how changes in criminal justice policies explain the great reduction of crime in the 1990s. The goal of this chapter is to examine the construction and interpreta- tion of aggregate crime regressions. Specifically, we employ contemporary economic and econometric reasoning to understand how aggregate crime regressions may be appropriately used to inform positive and normative questions. While by no means comprehensive, we hope our discussion will prove useful in highlighting some of the limitations of the use of these regressions and in particular will indicate how empirical findings may be misinterpreted when careful attention is not given to the link between the aggregate data and individual behavior. â he T interpretation of aggregate data continues to be one of the most difficult questions in social science; Stoker (1993) and Blundell and Stoker (2005) provide valuable overviews. 211

212 UNDERSTANDING CRIME TRENDS The chapter is organized as follows. We begin by describing a stan- dard choice-based model of crime. We then discuss how this individual- level model can be aggregated to produce crime regressions of the type found in the literature. In the next three sections we discuss the analysis of counterfactuals, issues of model uncertainty in crime regressions, and the relationship between statistical models and policy evaluation. We then apply our general arguments to areas in the empirical criminology litera- ture: the convergence of crime rates, capital punishment, and shall-issue concealed weapons laws. The next section discusses whether the limitations that exist in using crime regressions mean that they should be replaced by quasi-experimental methods, and a final section concludes the chapter. Our discussion is conceptual; Durlauf, Navarro, and Rivers (2008) provide a more systematic treatment of many of the issues we raise as well as an empirical application. CRIME AS A CHOICE From the vantage point of economics, the fundamental idea Âunderlying the analysis of crime is that each criminal act constitutes a purposeful choice on the part of the criminal. In turn, this means that the development of a theory of the aggregate crime rate should be explicitly understood as deriving from the aggregation of individual decisions. The basic logic of the economic approach to crime was originally developed by Gary Becker (1968) and extended by Isaac Ehrlich (1972, 1973). This logic underlies the renaissance of crime research in economics, exemplified in the work of, for example, Levitt (1996) and Donohue and Levitt (2001). In constructing a formal model, the idea that crime is purposeful means that an observed criminal act is understood as the outcome of a decision problem in which a criminal maximizes an expected utility function sub- ject to whatever constraints he faces. The utility function is not a primitive assumption about behavior (i.e., no economist thinks that agents carry explicit representations of utility functions in their heads); rather, it is a mathematical representation of an individualâs preferences, one that consti- tutes a rank ordering across the potential actions the individual may take. The choice-theoretic conception does not, by itself, have any implica- tions for the process by which agents make these decisions, although cer- tain behavioral restrictions are standard for economists. For example, to say that the commission of a crime is a purposeful act says nothing about how an individual assesses the various probabilities that are relevant to the choice, such as the conditional probability of being caught given that the crime is committed. That said, the economic analyses typically assume that an individualâs subjective beliefsâthat is, the probabilities that inform his decisionâare rational in the sense that they correspond to the probabili-

ON THE USE OF AGGREGATE CRIME REGRESSIONS 213 ties generated by the optimal use of the individualâs available information. While the relaxation of this notion of rationality has been a major theme in recent economic research (behavioral economics is now an established field of the discipline), it has not generally been a central focus in crime research, at least as conducted by economists. But we emphasize that the choice- based approach does not require rationality as conventionally understood. As Becker (1993, p. 386) has written: âThe analysis assumes that individu- als maximize welfare as they conceive it, whether they be selfish, altruistic, loyal, spiteful, or masochistic. Their behavior is forward looking, and it is also assumed to be consistent over time. In particular they try as best they can to anticipate the consequences of their actions.â To see how crime choice may be formally described, we follow the standard binary choice model of economics. We consider the decision problem of individuals indexed by i each of whom decides at each period t whether or not to commit a crime. Individuals live in locations l, and it is assumed that a person commits crimes only within the location in which he lives. Individual behaviors are coded as wi,t = 1 if a crime is committed, 0 otherwise. A common form for the expected utility associated with the ( ) choice ui,t Ï i,t is ( ) ( ) ( ) ui, t Ï i, t = Zl , t Î²Ï i, t + Xi, t Î³Ï i, t + Î¾l , t Ï i, t + Îµ i, t Ï i, t . . (1) In this expression, Zl,t denotes a set of observable (to the modeler) location-specific characteristics, and Xi,t denotes a vector of observable Ândividual-specific characteristics. The multiplication of the terms Zl,t b i and Xi,t g by wi,t capture the idea that the utility effect of these variables depends on whether the crime is committed. For example, the effect of a particular set of punishments on an individualâs utility will differ according to whether or not he commits a crime. The terms Î¾l ,t Ï i,t and Îµ i,t Ï i,t ( ) ( ) denote unobservable (to the modeler) location-specific and individual-spe- cific utility terms. These are functions of wi,t because these effects also depend on whether a crime was committed. From the perspective of a modeler, an individualâs sense of guilt is unobservable, and may be thought of as a utility consequence that occurs if he commits a crime. Similarly, the quality of the police force in a location is not observable (even if empirical proxies exist) and will affect utility only if a crime is committed, in this case via the effect on the likelihood of apprehension and punishment. The assumption of linearity of the utility function, while common in binary choice analysis, represents a statistical simplification and does not derive from choice-based reasoning per se. It is possible to consider nonparametric forms of the utility function (see Matzkin, 1992). We focus on the linear case both because it is the empirical standard in much of

214 UNDERSTANDING CRIME TRENDS social science and because it is not clear that more general forms will be particularly informative for the issues we wish to address. Some forms of nonlinearity may be trivially introduced, such as including the products of elements of any initial choice of Xi,t as additional observables. The distinction between observable and unobservable variables is fun- damental to the relationship between choice-based theories of crime and their embodiment in a statistical framework. We assume that the indi- vidual and location-specific unobservables are independent of each other both contemporaneously and across time. We further assume that the i Ândividual-specific errors are independent of both the individual-specific and location-specific observables. We do not assume that the location-specific unobservables are independent of the location-specific observables; there is no good theoretical reason why they should be so and, unlike the other independence assumptions, whether it holds or not is important in the interpretation of aggregate regressions. Under our specification, the net expected utility from committing a crime is Î½ i, t = Zl , t Î² + Xi, t Î³ + Î¾l , t (1) â Î¾l , t (0) + Îµ i, t (1) â Îµ i, t (0), , (2) and the choice-based perspective amounts to saying that a person chooses to commit a crime if the net utility is positive, that is, wi,t = 1, if and only if Zl ,t Î² + Xi,t Î³ + Î¾l ,t (1) â Î¾l ,t (0) > Îµ i,t (0) â Îµ i,t (1) . (3) Inequality (3) is useful as it provides a way of assigning probabilities to crime choices. Conditional on Xi,t , Zl,t , and Î¾l ,t (1) â Î¾l ,t (0) , the individual choices are stochastic; the distribution function of Îµ i,t (0) â Îµ i,t (1) , which we denote by Gi,t , determines the probability that a crime is committed. Formally, ( ) ( ) Pr Ï i, t = 1 Zl , t , Xi, t , Î¾l , t (1) â Î¾l , t (0) = Gi, t Zl , t Î² + Xi, t Î³ + Î¾l , t (1) â Î¾l , t (0) . (4) This conditional probability structure captures the microfoundations of the economic model we wish to study. This formulation is in fact a rela- tively simple behavioral model, in that we ignore issues such as (1) selection into and out of the population generated by the dynamics of incarcera- tion and (2) those aspects of a crime decision at t in which a choice is a single component in a sequence of decisions that collectively determine an individualâs utility; that is, a more general preference specification is one in which agents make decisions to maximize a weighted average of current and future utility, accounting for the intertemporal effects of their deci- sions in each period. While the introduction of dynamic considerations

ON THE USE OF AGGREGATE CRIME REGRESSIONS 215 into the choice problem raises numerous issues, such as state dependence, heterogeneity, and dynamic selection, these can in principle be dealt with using the analysis of Heckman and Navarro (2007), albeit at the expense of considerable complication of the analysis. AGGREGATION How do the conditional crime probabilities for individuals described by (4) aggregate within a location? Let rl,t denote the realized crime rate in locality l at time t. Notice that we define the crime rate as the percent- age of individuals committing crimes, not the number of crimes per se, so we are ignoring multiple acts by a single criminal. Given our assumptions, for the location-specific choice model (4), if individuals are constrained to commit crimes in the location of residence, then the aggregate crime rate in a locality is determined by integrating over the observable individual-spe- cific heterogeneity in the locationâs population. Let FX denote the empirical l ,t distribution function of Xi,t within l. The expected crime rate in a location at a given time is ( ) ( ) E Ïl , t Zl , t , FX , Î¾l , t (1) â Î¾l , t (0) = â« Gi, t Zl , t Î² + XÎ³ + Î¾l , t (1) â Î¾l , t (0) dFX (5) l ,t l ,t In order to convert this aggregate relationship into a linear regres- sion form, it is necessary to further restrict the distribution function Gi,t. S Â uppose that the associated probability densities dGi,t are uniform; a uni- form density produces what is known as a linear probability model for the individual choices. In this case, the crime rate in locality l at time t obeys Ïl ,t = Zl ,t Î² + X l ,t Î³ + Î¾l ,t (1) â Î¾l ,t (0) + Î¸l ,t , (6) where X l ,t is the empirical mean of X i,t within l and ( ) Î¸l ,t = Ïl ,t â E Ïl ,t Zl ,t , FX , Î¾l ,t (1) â Î¾l ,t (0) captures the difference between l ,t the realized and expected crime rate in a locality. This is the model typically employed in aggregate crime regressions. Our construction of equation (6) from choice-based foundations illus- trates how standard aggregate crime regressions require a number of statis- tical assumptions if they are to be interpreted as aggregations of individual behavior. The assumption of a uniform density for the individual specific heterogeneity is of concern; in order to ensure that the probabilities of each choice are bounded between 0 and 1, the support of the uniform density may need to be agent-specific. Unfortunately, other random utility speci- â See Aldrich and Nelson (1984, Chapter 1) for an accessible discussion of the problems of the linear probability model.

216 UNDERSTANDING CRIME TRENDS fications do not aggregate in a straightforward manner. To illustrate the ( ) problem, note that if one assumes that Îµ i,t Ï i,t has a type-I extreme value distribution, which is the implicit assumption in the logit binary choice model, then log ï£¬ ( ï£« Pr Ï = 1 Z , X , Î¾ (1) â Î¾ (0) ï£¶ i ,t i ,t l ,t i ,t l ,t l ,t ) ï£· will be linear in i ,t ( ï£¬ 1 â Pr Ï = 1 Z , X , Î¾ (1) â Î¾ (0) ï£· ï£ i ,t l ,t i ,t l ,t l ,t ï£¸ ) the various payoff components but will not produce a closed form solution for the aggregate crime rate. Methods are available to allow for analysis of aggregate data under logit type assumptions (see Berry, Levinsohn, and Pakes, 1995) but have not been applied, as far as we know, to the crime context. On its own terms, our development of a linear crime regression indi- cates how aggregation affects the consistency of particular estimators. While we have assumed that the individual-specific unobserved and observed determinants of crime choices are independent, we have not made an analogous assumption on the location-specific unobservables Î¾l ,t Ï i,t . In ( ) the aggregate regression, these may be correlated with either the aggregate observables that appear in the utility function Zl,t or those variables that appear as a consequence of aggregation X l ,t . From the perspective of theorizing about individual behavior, there is no reason why the regression residual Î¾l ,t (1) â Î¾l ,t (0) + Î¸l ,t should be orthogonal to any of the regressors in equation (6). By implication, this means that all the variables in equation (6) should be instrumented. Hence in our judgment the focus on instru- menting endogenous regressors that one finds in empirical crime analyses is often insufficient, in that, while this strategy addresses endogeneity, it does not address unobserved location-specific heterogeneity. Notice that if individual-level data were available, this problem would not arise, since one would normally allow for location-specific, time-specific, and location- time-specific fixed effects for a panel. COUNTERFACTUAL ANALYSIS How can an aggregate crime regression be used to evaluate counter- factuals such as a change in policy? Given our choice-theoretic framework, a counterfactual analysis may be understood as a comparison of choices under alternative policy regimes A and B. The net utility to the commission of a crime will depend on the regime, so that Î½ iAt = ZlAt Î² A + XiAt Î³ A + Î¾lAt (1) â Î¾lAt (0) + Îµ iAt (1) â Îµ iAt (0) , , , , , , , (7) and

ON THE USE OF AGGREGATE CRIME REGRESSIONS 217 Î½ iBt = ZlBt Î² B + XiBt Î³ B + Î¾lBt (1) â Î¾lBt (0) + Îµ iBt (1) â Îµ iBt (0) , , , , , , , (8) respectively. The net utility to individual i of committing a crime equals Î½ i ,t = ZlAt Î² A + XiAt Î³ A + Î¾lAt (1) â Î¾lAt (0) + Îµ iAt (1) â Îµ iAt (0) + , , , , , , ( , , , ) , ( Dl ,t ZlBt Î² B â ZlAt Î² A + Dl ,t XiBt Î³ B â XiAt Î³ A + ) (9) ( , , , ( Dl ,t Î¾lBt (1) â Î¾lBt (0) â Î¾lAt (1) â Î¾lAt (0) + , )) D (Îµ l ,t B i ,t (1) â ÎµiBt , (0 ) â ( Îµ A i ,t (1) â ÎµiAt , (0 ))) . where Dl,t = 1 if regime B applies to locality l at t; 0 otherwise. The analo- gous linear aggregate crime rate regression is Ïl , t = ZlAt Î² A + , A X l,t Î³ A (, , ï£ ) + Dl , t ZlBt Î² B â ZlAt Î² A + Dl , t ï£« X l , t Î³ B â X l , t Î³ A ï£¶ + B ï£¸ A (10) , ( , , , ( Î¾lAt (1) â Î¾lAt (0) + Î¸lAt + Dl , t Î¾lBt (1) â Î¾lBt (0) â Î¾lAt (1) â Î¾lAt (0) + Î¸lBt â Î¸lAt .. , , , , , ) ) The standard approach measuring how different policies affect the crime rate, in this case regimes A versus B, is to embody the policy change in ZlAt versus ZlBt and to assume that all model parameters are con- , , stant across regimes. This allows the policy effect to be measured by ( ) ZlBt â ZlAt Î² . Equation (10) indicates how a number of assumptions are , , embedded in the standard approach, in particular the requirement that , , ( , , ) Î¾lBt (1) â Î¾lBt (0) â Î¾lAt (1) â Î¾lAt (0) = 0 , that is, that the change of regime does not change the location-specific unobserved utility differential between committing a crime and not doing so. This requirement seems problematic, as it means that the researcher must be willing to assume that the regime change is fully measured by the changes in X l ,t and Zl,t. Changes in the detection probabilities and penalties for crimes typically come in bundles, and we argue below that there are cases, specifically capital punishment, in which this does not receive adequate attention in the relevant empirical formulations. MODEL UNCERTAINTY Our derivation of aggregate crime rates from microfoundations assumed that the researcher had strong prior information about the individual deci- sion process. Put differently, our derivation of an aggregate crime regression

218 UNDERSTANDING CRIME TRENDS was based on certainty about the underlying model of criminal behavior. In this section, we discuss ways to relax this assumption, that is, we con- sider the case of model uncertainty. In raising this, we emphasize that the problem of inadequate attention to model uncertainty is in no way unique to criminology. Nor do we mean to suggest that criminological studies are unique in the extent to which authors fail to investigate how modifications in baseline models affect inferences. Characterizing Model Uncertainty Our reading of the criminology literature suggests several general sources of model uncertainty. The categories we describe have previously been proposed by Brock, Durlauf, and West (2003) for economic growth models and Brock, Durlauf, and West (2007) for business cycle models. These categories are meant to identify general types of model uncertainty that are common in social science analyses. At the same time, our decompo- sition of model uncertainty is not unique; one can well imagine alternative divisions. Theory Uncertainty Social science theories for a given phenomenon are often open-ended (Brock and Durlauf, 2001), which means that one theory does not logically exclude another as having additional explanatory power. Hence there is often no justification for focusing on a subset of plausible explanations in empirical work. Some evidence of why this matters is suggested by Levittâs (2004) evaluation of sources of the crime decline of the 1990s. Levitt iden- tifies 10 alternative theories of the crime decline, all of which are mutually consistent. Without questioning any of his substantive conclusions, we do note that Levitt is to a large extent forced to evaluate the roles of the dif- ferent theories based on studies that, typically, do not account for the full range of the competing explanations when measuring the empirical salience of a particular one. Statistical Instantiation Models may differ with respect to details of statistical specification that have nothing to do with the underlying social science theories that moti- vate them, but rather are employed in order to translate these theories into representations that are amenable to data analysis. This is typically so even when the social science theories are themselves expressed mathematically. Differences in these assumptions can lead to different findings. A good example of how differences in statistical assumptions can

ON THE USE OF AGGREGATE CRIME REGRESSIONS 219 affect substantive conclusions is specification of time trends. In the context of the deterrence effects of shall-issue concealed weapons carry laws, dif- ferent time trend choices have proven to be important. Specifically, Black and Nagin (1998) found that the use of quadratic time trends in place of state-specific linear time trends eliminates the evidence of a link between liberalization of concealed weapons laws and crime rates found in Lott and Mustard (1997). Lottâs rejoinder (1998) argues that it is hard to identify the effects of a policy change (in this case, concealed weapons legality) because a quadratic trend will mask it; intuitively, if crime is rising before a law is passed and decreases thereafter, this will be approximated by the quadratic trend. Lottâs intuition may be reasonable, but his argument is question begging, as it applies in both directions. If crime follows an exogenously determined quadratic trend over some time interval and rising crime levels lead to a change in legislation, then Lottâs approach will spuriously identify a causal effect from the legislation. This is true even if state-specific trends are employed. From the perspective of model uncertainty, Black and Nagin and Lott are working with different statistical instantiations of unexplained temporal heterogeneity. Under the Black and Nagin specification, there may be, as Lott argues, substantial collinearity between the variable used to measure temporal heterogeneity and the variables used to measure the effects of shall-issue concealed weapons legislation. This multicollinearity does not invalidate the Black and Nagin model on logical grounds. In our judgment, the differences between Black and Nagin and Lott on this issue reflect the absence of good explanations for much of the temporal evolution of crime rates. Neither a linear specification nor a quadratic specification (or for that matter, more flexible splines or alternative semiparametric Â methods) instantiate substantive ideas about the crime process. Rather, they con- stitute efforts to purge the data so that the residual components may be analyzed. Trend specification also matters in the analysis of unemployment rates and crime. Greenberg (2001) criticizes Cantor and Land (1985) for model- ing trends using deterministic rather than unit root methods. Again, social science theory does not dictate a preference for one type of trend over another. While both Greenberg and Cantor suggest justifications in favor of their trend specifications that derive from individual behavioral determi- nants, neither of them demonstrates a one-to-one or even precise mapping from these determinants to their statistical modeling assumptions. Other examples of this type of model uncertainty include assumptions about additivity, linearity, and the use of logarithms versus levels. â his T argument is further developed in Plassmann and Whitley (2003).

220 UNDERSTANDING CRIME TRENDS Parameter Heterogeneity A third type of model uncertainty concerns parameter heterogeneity. Researchers often disagree on whether or not observations are simply draws from a common data-generating process, so that any heterogeneity in the observations derives from differences in values of some set of observable control variables and different realizations of the model errors. Social sci- ence theory typically does not impose that parameters are constant across observations. For example, the argument that there is a deterrent effect from a given penalty does not imply that the effect is independent of the geographical unit in which the penalty is present. Parameter heterogene- ity may be linked to deep questions about the interpretation of statistical m Â odels; see Brock and Durlauf (2001) for a discussion of parameter hetero- geneity and the concept of exchangeability of observations. Exchangeability, roughly speaking, captures the idea that observations, such as state-specific crime rates, may be treated as draws from a common statistical process. One example of sensitivity of empirical claims to assumptions about parameter heterogeneity is again found in the controversy between Black and Nagin and Mustard and Lott. Black and Nagin found that evidence of crime reductions associated with shall-issue laws are sensitive to the presence of Florida in the dataset. They found that eliminating data from Florida eliminated the evidentiary support for a handgun-crime link from some of the Lott and Mustard specifications. Another example appears in the capital punishment literature. D Â onohue and Wolfers (2005) challenge findings of Dezhbakhsh, Rubin, and Shepherd (2003) on the grounds that the findings are not robust to the exclusion of ÂCalifornia and Texas. As argued by Cohen-Cole et al. (2008), this disagreement may be understood as a disagreement about parameter homogeneity. Model Averaging How can the dependence of empirical claims on model specification be constructively addressed? We describe a strategy based on model averag- ing; ideas associated with model averaging appear to originate in Leamer (1978). They have become prominent in the past decade in statistics; a valu- able conceptual argument is made in Draper (1995), and the development of formal methods has been greatly advanced by Raftery (e.g., Raftery, Madigan, and Hoeting, 1997). We proceed using Bayesian language for expositional convenience, although the analysis can be done using frequen- tist estimators. For a given exercise, suppose that the objective of the researcher is to construct a conditional density of crime rates rl,t+1 based on data Dt and

ON THE USE OF AGGREGATE CRIME REGRESSIONS 221 ( ) model m, that is, Pr Ïl ,t +1 Dt , m . Many disagreements about substantive empirical questions, such as forecasts or the effects of alternative policies, derive from disagreements about the choice of model m. This is, of course, why model selection plays such a significant role in empirical work. From the perspective of some empirical questions, it is not obvious that this is the appropriate role for model choice. If the goal of an exercise is to compare policies, the model choice is a nuisance parameter. Similarly, if one wants to construct a forecast, then the model itself is not intrinsically interesting. In order to avoid dependence on a particular model specification, an alternative strategy is to develop conclusions based on a space of candidate models; denote this space as M. Probability statements about a future out- come such as rl,t+1 can then be constructed conditioning on the entire model space rather than on one of its elements. In other words, one computes the ( ) probability density Pr Ïl ,t +1 Dt , M , which is the conditional density of the crime rate given the data and a model space. From this perspective, the true model is an unknown that needs to be integrated out of the probability density. Formally, ( ) â Pr ( Ï Pr Ïl , t +1 Dt , M = l , t +1 ) ( ) Dt , m Pr m Dt .. (11) m âM ( ) Here Pr m Dt denotes the posterior probability that m is the correct model given the data. Conditioning on M means that the analyst knows which models comprise M. Intuitively, one constructs probability state- ments about an outcome, such as a crime rate, based on aggregating the information available across each of the models under consideration. This aggregation places greater weight on models that are more likely, as mea- ( ) sured by Pr m Dt . The linear structure in equation (11) derives from the law of conditional probability, hence the term averaging. Model averaging is emerging as a common methodology in econom- ics; its increasing popularity reflects a combination of improved computa- tional capacity and theoretical advances. The approach has been used to study economic growth (Brock, Durlauf, and West, 2003; Doppelhofer, Miller, and Sala-i-Martin, 2004; Fernandez, Ley, and Steel, 2001), finance (Avramov, 2002), forecasting (Garratt et al., 2003), and monetary policy (Brock, Durlauf, and West, 2003). An application to a crime context, the deterrent effect of capital punishment, is Cohen-Cole et al. (2008). While we regard model-averaging methods as very promising, we also emphasize that the methodology is still being developed and a number of outstand- ing theoretical questions still exist. And of course, model averaging still â ne issue concerning model priors that is worth noting concerns the assignment of priors O to similar models. Most of the model-averaging literature has employed diffuse priors, that is, all models are assigned equal prior weights. However, it can be the case that some models in

222 UNDERSTANDING CRIME TRENDS requires specification of the model space, which itself can be subjected to questioning. From Model Estimation to Policy Evaluation This discussion of model uncertainty contains an important limitation, in that it does not account for the objectives of a given empirical exercise. Focusing on the use of a single model, it seems intuitive that this model must be correctly specified in order for it to yield usable findings, so that no distinct considerations arise when one considers the reason why the model is employed. But even in this case, such intuition needs to be qualified. For example, Horowitz argues that in order to use cross-county data to evaluate the average effect of shall-issue laws, if there are differences between the states, so that the crime rate in a county is determined by some set of factors X, then in order to identify the effect of the laws âone must use a set that consists of just the right variables and, in general, no extra ones.â But as shown in Heckman and Navarro (2004), this is true only for a particular set of empirical strategies known as matching, of which linear regression is a special case. Heckman and Navarro demonstrate that there are other strategies that are designed to deal with the problem of missing information, in particular the use of control functions (see Navarro, 2007, for an overview). The control function approach is based on the idea that the presence of unobservable variables matters only to the extent that their relationship to the observables cannot be determined; for many cases, this relationship can be determined. And if so, then other information contained in the omitted variables is irrelevant. The standard example is the Heckman selection correction method, in which one adds a âMills ratioâ term to the a model space are quite similar, that is, differ only with respect to a single included Âvariable, whereas others are much more different from the perspective of theoretical or statistical a Â ssumptions. In this case, the diffuse prior can be very misleading. Brock, Durlauf, and West (2003) propose ways to construct model priors that mirror the nested structure of modern discrete choice theory, but much more needs to be done. The issue of model similarity is usu- ally ignored in ad hoc analyses of the robustness of findings. Lott (1998) defends his findings on concealed weapons permits by stating âmy article with David Mustard and my forthcoming book report nearly 1,000 regressions that implied a very consistent effect . . .â (p. 242). This claim is of little intrinsic interest without knowing what classes of models these regressions cover; put most simply, the different regression results are not independent, so the number 1,000 is not informative. â B ( A A ) Relative to equation (13), if Î¾l ,t (1) â Î¾l ,t (0) â Î¾l ,t (1) â Î¾l ,t (0) â 0 , then the observables Zl,t B and X l ,t do not constitute the correct set to use when estimating the model, since one needs to also control for the effect of the location-time unobservables. â nder matching, endogeneity is solved by assuming that there exists a set of variables such U that, conditional on these variables, endogeneity is eliminated. That is, the endogenous vari- ables are not independent of the errors, rather it is assumed they are conditionally independent when the correct set of observable variables (to the econometrician) is conditioned on.

ON THE USE OF AGGREGATE CRIME REGRESSIONS 223 regression under the assumption of normality, but one can be much more general and use semiparametric methods to estimate the control function term (see Navarro, 2007). More generally, one cannot decouple the assessment of a modelâs speci- fication from the objective for which the model is employed. Similarly, any assessment of fragility (or the lack thereof) of empirical claims can be fully understood only with reference to a decision problem. POLICY-RELEVANT CALCULATIONS Basic Ideas In this section, we explicitly consider the relationship between statisti- cal models and policy evaluation from a decision-theoretic perspective. The fact that statistical significance levels do not equate to policy statements is well known (see Goldberger, 1991, for a nice discussion), our goal here is to suggest some ways of reporting and interpreting results for policy contexts. In making this argument, we are drawing both on classic ideas in statistics, notably Savage (1951) and Wald (1950, sections 1.4.2, 1.6.2, and elsewhere), as well as recent work in econometrics (e.g., Brock, Â Durlauf, and West, 2003, 2007; Brock et al., 2007; ÂHeckman, 2005; Manski, 2005, 2006) to implement some of these ideas. Again, our remarks apply with equal force to work in social sciences other than criminology. Suppose that the policy maker has a payoff function ( ) V Ïl , t +1 Dt , p (12) where p â{ A, B} denotes the policy regime and, as before, Dt represents the information available to the policy maker at time t. The conditioning of the utility function on Dt allows for the possibility that the policy makerâs preferences depend on aspects of the particular locality since location- s Â pecific data Dl,t are a subset of Dt. For an expected payoff maximizer, the optimal policy problem is ( ) ( ) max p â{A, B} â« V Ïl , t +1 Dt , p Pr Ïl , t +1 Dt , p, m . (13) Expression (13) implies that the sufficient objects for policy analysis are ( ) ( ) Pr Ïl ,t +1 Dt , A, m and Pr Ïl ,t +1 Dt , B, m ; these are the posterior distribu- tions of the crime rate given the data, model, and policy. These probabilities fully capture the aspects of the data that are relevant to policy evaluation calculation. Notice that these calculations may not require all aspects of a model to be correctly specified. This was seen in our discussion of the use of matching versus control functions. Heckman (2005) provides a

224 UNDERSTANDING CRIME TRENDS deep analysis of the relationship between models and policy calculations, emphasizing what he denotes as âMarschakâs maximâ given ideas found in Marschak (1953): âFor many policy questions it is unnecessary to identify full structural models. . . . All that is needed are combinations of subsets of the structural parameters, corresponding to the parameters required to forecast particular policy modifications, which are much easier to identify (i.e., require fewer and weaker assumptions)â (p. 49). One advantage of explicit calculations of posterior densities for policy effects is that they naturally allow one to assess the effects of portfolios of policies. Evidence on the effects of individual policies may be imprecise, whereas evidence on the effects of combinations of policies may not be. We do not know whether there are cases of this type in criminology. Another advantage is that such calculations avoid confusion between the lack of statistical significance of a coefficient for a policy variable and the claim that a policy has no effect; while this is a banal observation, the mistake is often seen. An example of this is found in Lott (1998), who, in evaluating Black and Naginâs (1998) critique of his work, asserts âon the basis of Black and Naginâs comment and our original article, the choice is between concealed handguns producing a deterrent effect or having no effect (one way or the other) on murders and violent crime generallyâ (p. 242). Lottâs exclusion of the possibility of any crime-enhancing effect of concealed weapons ignores the uncertainty associated with point estimates of the effects. That is, concluding that one cannot reject that the effect is equal to zero does not mean that the effect is indeed zero. One may not be able to reject that it is 0.1 (or â0.1), either. The point estimate is only the most likely (in a particular sense) value of the parameter given the data, not the only possible one. The policy-relevant calculation requires assessing the probabilities for different magnitudes of positive and negative effects, which cannot be ascertained from the numbers he (and other participants in this literature) report. Model Averaging and Policy Evaluation When model uncertainty is present, the optimal policy calculation equation (13) may be generalized in a straightforward fashion, as the policy maker simply conditions on M rather than m. The relevant calculation in this case is max p â{A, B} â ( â« V ( Ïl,t +1 Dt , p) Pr ( Ïl,t +1 Dt , p, m )) Pr ( m Dt ) = m âM (14) ( ) ( max p â{A, B} â« V Ïl , t +1 Dt , p Pr Ïl , t +1 Dt , p, M .)

ON THE USE OF AGGREGATE CRIME REGRESSIONS 225 For the model uncertainty case, the empirical objects that are required for ( ) policy evaluation are Pr Ïl ,t +1 Dt , A, m and R ( p, d , m ) , which represent the posterior distributions of crime rates conditional on the data, the policy, and the model space. Equation (14) indicates an important feature of policy evaluation, namely, that unless the payoff function is model-specific, the identity of the true model does not directly affect policy evaluation. For the purposes of policy evaluation, what matters is the distribution of outcomes under alternative policies. Unlike the case of the social scientist, the model has no intrinsic interest to a policy maker; it is simply an additional source of uncertainty in the effects of a policy. Beyond Model Averaging Once model uncertainty is involved in policy evaluation, new consider- ations can arise. One reason for this is that a policy maker may be unwilling to condition decisions on model priors; without these, one cannot assign posterior model probabilities and engage in model averaging. The absence of a basis for constructing priors is one reason for recent theoretical work on decision making under ambiguity, which focuses on how agents should make decisions in environments in which certain probabilities cannot be defined. For our purposes, what matters is that, in such cases, there exist ways to engage in policy evaluation that do not require that one is able to calculate model probabilities. The minimax approach, advocated by Wald (1950) and recently explored in macroeconomic contexts by Hansen and Sargent (2007), evaluates policies by the criterion ( ) ( ) max p â{A, B} minm âM â« V Ïl , t +1 Dt , p Pr Ïl , t +1 Dt , p, m . (15) Minimax selects the policy that does best for the least favorable model in the model space. Metaphorically, the policy maker plays a game against nature in which nature is assumed to choose the model that minimizes the policy makerâs payoff. This sets a lower bound on the payoff from the policy. An alternative approach is known as minimax regret, due to Savage (1951) and recently explored in microeconomic contexts by Manski (2005, 2006), which evaluates policies by the criterion min pâ{A,B} max m âM R ( p, Dt , m ) (16) where regret, R ( p, d , m ) , is defined by

226 UNDERSTANDING CRIME TRENDS R ( p, d , m ) = max p âP ( â« V (Ï l , t +1 ) ( Dt , p Pr Ïl , t +1 Dt , p, m â )) (17) â« V ( Ïl,t +1 D , p ) Pr ( Ï t l , t +1 ) Dt , p, m . Minimax regret selects the policy with the property that the gap between the model-specific optimal policy and its performance is small- est when comparisons are made across the model space. The criterion is generally regarded as a less conservative criterion for policy evaluation than minimax. Brock et al. (2007) employ minimax regret in monetary policy evaluation. Manski (2006) applies minimax regret in the context of treatment assignment. An important finding is that optimal treatment rules can be fractional as agents with identical observables receiving differ- ent treatments. This may be of particular interest in crime policy contexts, as it suggests a trade-off between the fairness and deterrence objectives of punishment that policy makers ought to address. APPLICATIONS TO CRIMINOLOGY ISSUES In this section, we apply some of our general arguments to current controversies in criminology. Convergence in Crime Rates A first example in which more careful attention is needed to the deter- minants of aggregate crime regressions involves efforts to evaluate conver- gence among aggregate crime rates. Two examples of studies of this type are OâBrien (1999), which focuses on male-female differences in arrest rates, and LaFree (2005), which considers cross-country homicide rates. Both papers interpret convergence in terms of the time-series properties of the differences between the series of interest. Both papers lack formal attention to the determinants of individual behavior and their associated aggregate implications. The substantive social science notion of convergence involves the question of whether contempo- raneous disparities between two time series may be expected to disappear over time. As formulated in Bernard and Durlauf (1995), convergence between r1,t and r2,t means that ( lim kââ E Ï1,t + k â Ï2,t + k Ft = 0 ) (18) where Ft denotes the information available at time t. Hence the focus of OâBrien and LaFree on the presence of time trends or unit roots in the

ON THE USE OF AGGREGATE CRIME REGRESSIONS 227 d Â ifference in crime rates would seem to be sensible. The problem, identi- fied in Bernard and Durlauf (1996), is that, without a theory of how indi- vidual crime choices are determined, there is no basis for regarding either of these tests as appropriate. The reason is that the unit root and time trend Âanalyses presuppose that the series Dr1,t and Dr2,t are second-order s Â tationary processes. The statistical assumption of second-order stationarity has substantive behavioral implications. Specifically, it means that the series are generated by social processes that are local to their long-run behaviors and rules out the case in which social processes are in transition to a long-run type of behavior. When societies are in transition, the stochastic process charac- terizing a socioeconomic outcome will not have time-invariant moments, which is what is assumed in time-series analyses of the type conducted by OâBrien and LaFree. These issues have been long understood in the eco- nomic growth literature, in which convergence has been studied primarily with respect to per capita output (and in which the relationship among trends, unit roots, and convergence were precisely characterized long before the papers we are discussing). In the crime context, it is easy to develop intuition as to why time-series analysis of convergence may be invalid. Consider OâBrienâs analysis of g Â ender differences. The period 1960-1995 is one of changing gender roles and family structure, among other things. If one considers the determinants of female crime rates, there is no reason to believe that the changes between 1960 and 1975 are simply another draw from the same process generat- ing the changes between 1975 and 1990. Similarly, LaFreeâs evaluation of convergence between industrializing poor nations and industrialized rich ones assumes that intracountry homicide rate changes are generated by a second-order stationary process. However, LaFreeâs invocation of the mod- ernization process as explaining national crime dynamics is inconsistent with his statistical methodology. Countries experiencing crime that âresults when modern values and norms come into contact with and disrupt older, established systems of role allocationâ (LaFree, 2005, p. 192) are in tran- sition; their associated stochastic processes of crime rate changes will not fulfill the invariance requirements needed to apply the time-series methods we employ. These convergence analyses may be criticized from a second vantage point, namely, the absence of any distinction between conditional and unconditional convergence. Conditional convergence means that there exists a set of initial conditions such that convergence between two units (gender, country) occurs only if these initial conditions are identical. Denot- ing these conditions as Xt, conditional convergence means that ( ) lim k ââ E Ï1, t + k â Ï2, t + k Ft , X1, t = X2, t = 0. (19)

228 UNDERSTANDING CRIME TRENDS In the economic growth literature, it is well understood that conditional rather than unconditional convergence is the natural object of interest. Two countries with different savings rates are not expected to uncondi- tionally converge, and there is no substantive theoretical implication when unconditional convergence fails; see Mankiw, Romer, and Weil (1992) for the classic analysis. In the crime context, it is unclear what is learned from unconditional convergence exercises. OâBrien is relatively circumspect in interpreting his results, but even his speculations on how to explain the finding of no convergence in homicide with convergence in other crimes are not justifiable, since without a theory as to why unconditional convergence is to be expected, there are so many ways to differentiate the experiences of men and women that it is not clear whether there is a fact to be explained. As for LaFree, if there are factors outside the modernization process that determine crime ratesâand obvious candidates include socioeconomic fac- tors, such as levels of unemployment and inequality, demography, and differences in national criminal justice systemsâthen the absence of uncon- ditional convergence does not speak to the empirical relevance of modern- ization or any other theory considered in isolation. Deterrence Effect of Capital Punishment Our second example concerns recent arguments about the deterrence effects of capital punishment. We focus on two papers, the empirical study of deterrent effects by Dezhbakhsh, Rubin, and Shepherd (2003) and the normative study by Sunstein and Vermeule (2005). We choose the first paper because it has been quite influential in resurrecting claims in favor of a deterrent effect and because it has recently come under criticism by Donohue and Wolfers (2005). Dezhbakhsh, Rubin, and Shepherd do not make general policy claims about the desirability of capital punishment given their findings. Sunstein and Vermeule (2005), however, do make this connection. They argue that evidence in favor of a capital punishment deterrence effect can render the punishment morally obligatory. Hence our interest in this second paper. The behavioral foundations of Dezhbakhsh, Rubin, and Shepherd rec- ognize that the consequences for the commission of a murder involve three separate stages: apprehension, sentencing, and carrying out of the sentence. Defining the variables C = caught, S = sentenced to be executed, and E = executed, Â Dezhbakhsh, Rubin, and Shepherd estimate the murder rate regression ( ) ( ) Ïl , t = Î± + Zl , t Î² + Pl , t (C ) Î²C + Pl , t S C Î²S + Pl , t E S Î²E + Îº l , t , (20) where

ON THE USE OF AGGREGATE CRIME REGRESSIONS 229 Pl ,t (C ) = probability of being caught conditional on committing a murder, Pl ,t ( ) S C = probability of being sentenced to be executed conditional on being caught, Pl ,t ( ) E S = probability of being executed conditional on receiving a death sentence, and other variables follow the definitions associated with equation (6). Dezhbakhsh, Rubin, and Shepherd argue in favor of a deterrence effect based on the negative point estimates and statistical significance of the coefficients on the various conditional probabilities. Microfoundations From the perspective of our first argument, that aggregate models should flow from aggregation of individual behavioral equations, the Dezhbakhsh, Rubin, and Shepherd specification can be shown to be flawed. Specifically, the way in which probabilities are used does not correspond to the prob- abilities that arise in the appropriate decision problem. For Dezhbakhsh, Rubin, and Shepherd, the potential outcomes are NC = not caught, CNS = caught and not sentenced to death, CSNE = caught, sentenced to death, and not executed, CSE = caught, sentenced to death, and executed. The expected utility of a person who commits a murder is therefore Prl ,t ( NC ) ui,t ( NC ) + Prl ,t (CNS ) ui,t (CNS ) + (21) Prl ,t (CSNE ) ui,t (CSNE ) + Prl ,t (CSE ) ui,t (CSE ) . The unconditional probabilities of the four possible outcomes are, of course, related to the conditional probabilities. In terms of conditional p Â robabilities, expected utility may be written as (1 â Pr (C )) u ( NC ) + l ,t i ,t (1 â Pr ( S C )) Pr (C ) u (CNS ) + l ,t l ,t i ,t (22) (1 â Pr ( E S )) Pr ( S C ) Pr (C ) u (CNSE) + l ,t l ,t l ,t i ,t ( ) ( ) Prl ,t E S Prl ,t S C Prl ,t (C ) ui,t (CSE ) . A comparison of expressions (22) and (20) reveals that the Dezhbakhsh, Rubin, and Shepherd specification does not derive naturally from individual

230 UNDERSTANDING CRIME TRENDS choices, since the conditional probabilities in (20) interact with each other in the calculation of expected utility as in (22). If one substitutes in a linear representation of the utility functions for the different outcomes, it is evi- dent that (22) cannot be rearranged to produce an aggregate crime equation in which the conditional probabilities appear additively, as in (20); a full analysis appears in Durlauf, Navarro, and Rivers (2008). Put differently, the effect on behavior of the conditional probability of execution given a death sentence cannot be understood separately from the effects of the conditional probability of being caught and being sentenced to death if caught. We therefore conclude that the Dezhbakhsh, Rubin, and Shepherd specification fails to properly model the implicit decision problem involved in homicides. Their analysis is based on a misspecification of the implica- tions of their assumed behavioral model. Aggregation Our aggregation discussion suggests how correlations can arise between regressors and model errors because of unobserved location characteristics. Dezhbakhsh, Rubin, and Shepherd instrument only the conditional crime probabilities in (20), doing so on the basis that these probabilities are collec- tive choice variables by the localities. However, in the presence of unobserved location characteristics, it is necessary to instrument the regressors contained in Zl,t as well. Since instrumenting a subset of the variables in a regression that correlate with the regression errors does not ensure consistency of the associated subset of parameters, the estimates in Dezhbakhsh, Rubin, and Shepherd would appear to be inconsistent (in the statistical sense). Dezhbakhsh, Rubin, and Shepherd might respond to this objection by noting that they use location-specific fixed effects. However, these will not be sufficient to solve the problem, since the location-specific unobservables ( ) Î¾l ,t Ï i,t can vary over time. Policy Effect Estimation Our discussion of policy effect evaluation also calls into question the Dezhbakhsh, Rubin, and Shepherd analysis, as it assumes that the fluctua- tions in their arrest, sentencing, and execution probabilities constitute the full set of changes in policies across time periods. This seems problematic. The decision to commit a homicide, under the economic model of crime, depends on the entire range of penalties and their associated probabilities. Changes in the rates at which murderers are sentenced to life imprisonment without parole, for example, are not accounted for by Dezhbakhsh, Rubin, and Shepherd or, as far as we know, any other capital punishment deter-

ON THE USE OF AGGREGATE CRIME REGRESSIONS 231 rence studies. Hence these studies suffer from an obvious omitted variables problem. This argument can be pushed farther. As shown in Gelman et al. (2004), the probability that a given death sentence will be overturned by a state or federal appeals court is at least 2/3. These authors also find that only 5 percent of the death sentences between 1975 and 1993 led to the eventual execution of those sentenced. Relative to our choice model, the Gelman et al. findings mean that the reintroduction of capital punishment in a state, on average, substantially increases the probability that the com- mission of murder leads to the outcome CSNEâthat is, arrested, sentenced to death, and not executed. Since exonerations are rare, it is reasonable to conjecture that murderers with outcome CSNE experience longer prison sentences than they would have had they not been sentenced to death. This suggests that periods in which criminals face higher probabilities of capi- tal sentencing and actual execution are also associated with longer prison sentences. Yet this increase is not reflected in the ÂDezhbakhsh, Rubin, and Shepherd regression. Put differently, if an increase in the conditional prob- ( ) ability of a death sentence given arrest, Prl ,t S C , is associated with an increase in Prl ,t (CSNE ) , then it is no longer clear what it means to say that a Dezhbakhsh, Rubin, and Shepherd-type regression provides evidence on the effects of capital punishment. Does an increase in long prison sentences because of death sentences followed by reversals correspond to what is understood to be the deterrent effect of capital punishment? Model Uncertainty Donohue and Wolfers (2005) have argued that the Dezhbakhsh, Rubin, and Shepherd findings of strong deterrence effects are fragile, as small changes in their baseline specification can lead to an absence of a statisti- cally significant effect or even evidence that a larger number of executions is associated with a larger number of murders. Specifically, Donohue and Wolfers show that the Dezhbakhsh, Rubin, and Shepherd findings change when one alters the lag structure for the instrumental variables used for the punishment probabilities, as well as when one drops California and Texas from the sample. The latter may be interpreted as a change in the assump- tion that all states are exchangeable with respect to the model employed by Dezhbakhsh, Rubin, and Shepherd. Cohen-Cole et al. (2008) attempt to adjudicate the differences between Dezhbakhsh, Rubin, and Shepherd and Donohue and Wolfers by treating the problem as one of model uncertainty. To do this, a space of potential models was generated using different combinations of the assumptions found in the two papers. Cohen-Cole et al. conclude that the evidence for

232 UNDERSTANDING CRIME TRENDS deterrence in the sample studied by Dezhbakhsh, Rubin, and Shepherd is weak. Policy-Relevant Calculations Following our general discussion, the statistical significance of the capital punishment variables in a murder regression does not produce the appropriate information needed to make policy comparisons. This has implications for the way such evidence is employed in death penalty debates. Sunstein and Vermeule (2005) argue that evidence of a deterrent effect can produce a moral case for capital punishment, in that the decision of a government to fail to implement a life-saving policy is equivalent to the decision to implement a policy that costs lives. Sunstein and Vermeule (2005) develop their argument conditioning on evidence of a deterrence effect. Leaving aside the insouciance with which they treat the empirical literature, their argument lacks attention to the appropriate nature of the policy makerâs loss function and the nature of the uncertainty of the empirical evidence. The Sunstein and Vermeule analysis treats the expected number of lives saved as the variable of interest to the policy maker; in Dezhbakhsh, Rubin, and Shepherd, this value is a function of the estimated parameter bE in (20). The expected number of lives saved is not necessarily sufficient in describing a policy makerâs utility function, even if this function is a monotonically increasing function of the number of lives saved. As such, their attention to this figure is analogous to making a utilitarian as opposed to a welfarist calculation (see Sen, 1979). While Sunstein and Vermeule would presum- ably respond that they are assuming that the precision associated with estimates of the expected number of lives saved is high, precision needs to be defined with respect to the policy makerâs utility function. It is not an independent object. The sensitivity of deterrence evidence to model choice, as demon- strated by Donohue and Wolfers and extended in Cohen-Cole et al. (2008), raises the issues we have discussed with respect to decision making under a Â mbiguity and the evaluation of policies when one does not wish to base them on a choice of model priors. Without a justification of the choice of priors, there is no expected deterrence effect on which Sunstein and V Â ermeule can even rely. Our impression of the philosophy literature is that â t A the same time they also state that âThe foundation of our argument is a large and growing body of evidence that c Â apital punishment may well have a deterrent effect, possibly a quite powerful one. . . . The particular numbers do not much matterâ (p. 706).

ON THE USE OF AGGREGATE CRIME REGRESSIONS 233 the issue of policy evaluation under ambiguity has generally not been dis- cussed, although Gaus (2006) makes an interesting argument in favor of following principles rather than expected-effect calculations when assessing policies, the effects of which are associated with substantial uncertainty. To be clear, none of this means that Sunstein and Vermeule (2005) are incorrect in their conclusions about the ethical implications of a certain deterrent effect for a policy maker or that the death penalty is either moral or immoral per se. Rather, our claim is that the policy implications of the uncertainty associated with deterrence effects cannot be assessed outside of the policy makerâs preferences. Right-to-Carry Laws and Crime: Firearms and Violence Revisited Our third example is the controversy over the effects of shall-issue concealed weapons laws in the National Academies report Firearms and Violence (National Research Council, 2005). This report concluded (pp. 150-151): with the current evidence it is not possible to determine that there is a causal link between the right-to-carry laws and crime rates. It is also the committeeâs view that additional analysis along the lines of the current literature is unlikely to yield results that will persuasively demonstrate a causal link between right-to-carry laws and crime rates (unless substantial numbers of states were to adopt or repeal right-to-carry laws), because of the sensitivity of the results to model specification. Committee member James Q. Wilson dissented from this part of the study, on the grounds that the sensitivity to specification found in the report did not account for the sensibility of different models; in particular, he ques- tioned whether the failure of models that excluded socioeconomic control variables to find deterrent effects was of importance in assessing the deter- rent effect. Wilson observes (National Research Council, 2005, p. 270): Suppose Professor Jones wrote a paper saying that increasing the number of police in a city reduced the crime rate and Professor Smith wrote a rival paper saying that cities with few police officers have low crime rates. Suppose that neither Smith nor Jones used any control variables, such as income, unemployment, population density, or the frequency with which offenders are sent to prison in reaching their conclusions. If such papers were published, they would be rejected out of hand by the committee for the obvious reason that they failed to supply a complete account of the factors that affect the crime rate.

234 UNDERSTANDING CRIME TRENDS The committeeâs rejoinder to Wilson argued (National Research ÂCouncil, 2005, pp. 273-274): Everyone (including Wilson and the rest of the committee) agrees that control variables matter, but there is disagreement on the correct set. Thus, the facts that there is no way to statistically test for the correct specifica- tion and that researchers using reasonable specifications find different answers are highly relevant. Given the existing data and methods, the rest of the committee sees little hope of resolving this fundamental statistical problem. We believe that this conclusion is too pessimistic. The disagreement between Wilson and the rest of the National Academies committee reflects the absence in the report of an explicit evaluation of how model uncertainty interacts with evidence of shall-issue laws. While the assertion that it is impossible to statistically identify the correct specification of a statistical model is true at some level of generality (although the report is frankly unclear on what is meant by this), this argument is hardly novel; it is known in the philosophy literature as the Duhem-Quine hypothesis (Quine, 1951, is the classic statement) and refers to the idea that all theories are undeter- mined by available data. At this level of generality the National Academies committee claim is an uninteresting observation with respect to social science research, since it begs the question of the relative plausibility of assumptions. For the dispute at hand, we believe that Wilson is correct in his argument that a model whose specification includes controls suggested by social science theory should receive greater weight than one that does not. Furthermore, these two models are statistically distinguishable. To conclude that one should regard evidence of a deterrent effect as persuasive only if both models produce the same findings makes little sense. The report implicitly suggests that the models without control variables are intrinsically interest- ing: âNo link between right-to-carry laws and changes in crime is appar- ent in the raw data . . . ; it is only once numerous covariates are included that the . . . effects . . . emergeâ (p. 150). This remark ignores the classic S Â impsonâs paradox, in which a bivariate relationship has one direction, whereas a multivariate relationship does not. The standard example of Simpsonâs paradox is the positive relationship between admission to the hospital and the probability of death. â he T reportâs suggestion that randomized experiments represent the gold standard for research ignores the assumptions required for their conductâintegrity of the researcher, accu- racy of data collection, etc. An advocate of randomized experiments would presumably dismiss concerns about such factors as implausibleâbut this is precisely our point.

ON THE USE OF AGGREGATE CRIME REGRESSIONS 235 Model averaging provides a natural way of integrating the information across the alternative specifications considered in the National Academies report. As we see it, the committee could have addressed the sensitivity of shall-issue deterrence effects by constructing a set of specifications that included those found in the literature as well as others that are formed by combining the assumptions underlying these models. Intuitively, one thinks of the assumptions that differentiate models as the axes of the model space, and one fills the model space out with those combinations of assumptions that are coherent with one another. Averaging over this space would have integrated the information in the different models and indicated whether evidence of a shall-issue deterrent effect is present when one conditions on a model space rather than a particular model. One answer to our advocacy of model averaging as a tool to address model uncertainty of the type facing the National Academies committee is that a given body of empirical studies captures only a small fraction of the universe of potential models (and indeed might represent a measure 0 set). This is certainly a tenable position. But if this position is taken, then it would be irrelevant whether a given body of studies produced similar or conflicting results. If it is then claimed that the degree of consistency in results across models contained in a subspace is informative about the results that would be ascertained were the model space expanded, then it is difficult to see why the relative prior plausibility and relative evidentiary support within an initial model space are not informative as well. A second answer to the use of model averaging might rely on the absence of a principled basis for assigning prior model probabilities. We are certainly sympathetic to this view. But if this position is taken, then the implications of the body of model-specific findings of an effect of shall- issue laws to policy need to be explicitly considered. It is not obvious, for example, that the fragility that the National Academies report claims to be present in concealed weapons regressions is even an argument against the laws. Suppose that a policy maker possesses minimax preferences with respect to model uncertainty. Fragility of deterrence evidence does not l Âogically lead to rejection of the policy; one needs to know the payoffs under the different models under consideration. The National Academies report seems to take the position that, in absence of strong evidence that the laws reduce crime, they should not be implemented. But minimax preferences do not, by themselves, generate this conclusion, which really is based on the presumption that the law should not be implemented unless there is compelling evidence of crime reduction. This line of reasoning can be justified (e.g., Brock, Durlauf, and West, 2003), but it requires context- specific argumentation. Therefore, a recommendation we make for policy evaluation Â studies

236 UNDERSTANDING CRIME TRENDS such as Firearms and Violence is that claims about the robustness or f Â ragility of various findings be evaluated with respect to different loss func- tions, with particular attention to minimax and minimax regret calculations as supplements to the standard Bayesian ones. SHOULD AGGREGATE CRIME REGRESSIONS BE ABANDONED? One response to the discussion in this paper would be to search for alternative ways of uncovering aggregate criminological facts. The c Â ritiques we have raised are part of the source of interest in so-called natural experiments, in which an exogenous event of some type allows a comparison of aggregate crime outcomes (see Levitt, 1996, for a nice example). In his appendix to the Firearms and Violence study, Horowitz (2005) makes a broad general argument against the use of regression models to elucidate the determinants of crime, specifically in terms of evaluating policy effects. While his focus is on concealed weapons laws, his claims apply with equal force to other crime contexts. According to Horowitz, âIn summary, the problems posed by high-dimensional estimation, misspecified models, and lack of correct knowledge of the correct set of explanatory variables seem insurmountable with observational dataâ (National Research Council, 2005, p. 308). In contrast, he argues that random assignment of policies could in principle reveal their effects; in particular, he discusses how Ârandom assignment can allow for the estimation of average treatment effects (a par- ticular piece of legislation, such as shall-issue concealed weapons laws, is an example of a treatment). We of course concur that there does not exist an algorithm to infallibly identify the âtrueâ model of crime (or for that matter, other phenomena) when the universe of candidate models is broad enough. However, we do not believe this means that crime regressions cannot be informative about policy. Different models have both different ex ante levels of plausibility and ex post levels of goodness of fit for a given body of observational data. The different concealed weapons regressions with and without socioÂ economic controls are not equally ex ante plausible, given the state of social science. And we do not know, given our priors, how the relative goodness of fit of the different models analyzed in the National Academies report would translate into different posterior model probabilities. Our discussion of the assumptions that underlie the interpretation of aggregate crime regressions may all be interpreted as examples for H Â orowitzâs arguments about the limitations of regression analysis of crime. We do not claim to have an answer to the question of how to integrate the different types of model uncertainty we have discussed into a single integrated framework, let alone introduce such factors as the extension of

ON THE USE OF AGGREGATE CRIME REGRESSIONS 237 the basic crime model to intertemporal decision making. Our disagreement with Horowitz is that we see a role for empirical models in informing policy discussion, even though the researcher is aware of untestable or unappeal- ing assumptions underlying them. The way in which models are used to inform beliefs necessarily requires judgments; this necessity does not mean that the models are uninformative. A researcher brings a body of social science and statistical knowledge to bear in the assessment of empirical results; this knowledge matters in assessing the dependence of a result on an assumption. Put differently, not all assumptions are equally arbitrary. The need for assumptions is not unique to regression analysis with obser- vational data; all empirical work is theory-laden (to use Quineâs phrase). An experiment of the type proposed by Horowitz with respect to shall-issue weapons permit lawsârandomized legalization across statesâwould, if one is to use the findings to inform policy makers, require assumptions about (1) the degree to which potential criminals can alter the locations in which crimes are committed, (2) the nature of migration by potential criminals across state boundaries both before the experiment and in response to it, (3) the effect on the current crime choices of potential criminals of the knowledge that an experiment that may affect future laws in their state of residence is being conducted, etc. Also, the translation of findings from such an experiment into a recommendation for those states that did not imple- ment the policy requires exchangeability assumptions on the states. Does one assume that the deterrent effect of the law is identical across states? If state-level deterrent effects are heterogeneous, how is this heterogeneity to be modeledâvia random effects, varying coefficients, or some other method? Randomized experiments cannot avoid the need for judgments; as described in detail in Heckman (2000, 2005), judgment is intrinsic to social scientific inquiry. Overall, we do not see good reasons to regard natural experiments as superior to regressions with observational data in terms of their relative utility as means of understanding crime.10 It is straightforward to construct examples in which one methodology can provide insights that the other does not. Each has a contribution to make in criminological research. â bbring A and Heckman (2007) provide a comprehensive overview of the assumptions r Â equired in developing estimates of treatment effects that account for considerations of the type hinted at in our discussion. 10â ee Heckman (2005) and Manski (2007) for discussion of the limitations of experiments; S Heckman and Navarro (2004) compare the strengths and weaknesses of different empirical strategies for uncovering the determinants of individual choice.

238 UNDERSTANDING CRIME TRENDS CONCLUSION In this chapter, we have described some issues we regard as important in the econometric study of crime: microfoundations, aggregation, counterÂ factual analysis, and policy evaluation. We have tried to make clear the various assumptions that must be maintained to interpret aggregate crime regressions with respect to individual behavior and have emphasized how standard uses of these regressions to evaluate policy presuppose a number of assumptions. In light of disagreements about these assumptions, which ultimately underlie claims of fragility or robustness of an empirical result, we have outlined some ways of using model-averaging methods and statisti- cal decision theory to make progress. Throughout, we have emphasized the role of judgment in empirical work, for which no algorithm exists. ACKNOWLEDGMENTS We thank the National Science Foundation and University of ÂWisconsin Graduate School for financial support. Arthur Goldberger and Justin W Â olfers provided immensely helpful comments on a previous draft. REFERENCES Abbring, Jaap H., and James J. Heckman. (2007). Econometric evaluation of social programs, part III: Distributional treatment effects, dynamic treatment effects, dynamic discrete choice, and general equilibrium policy evaluation. In J. Heckman and E. Leamer (Eds.), Handbook of econometrics, volume 6. Amsterdam: Elsevier. Aldrich, John H., and Forrest D. Nelson. (1984). Linear probability, logit, and probit models. Beverly Hills, CA: Sage. Avramov, Doron. (2002). Stock return predictability and model uncertainty. Journal of Â inance, 64, 423-458. F Ayres, Ian, and John Donohue, III. (2003). Shooting down the âmore guns, less crimeâ h Â ypothesis. Stanford Law Review, 55, 1193-1312. Becker, Gary S. (1968). Crime and punishment: An economic analysis. Journal of Political Economy, 78(2), 169-217. Becker, Gary S. (1993). Nobel lecture: The economic way of looking at behavior. Journal of Political Economy, 101(3), 385-409. Bernard, Andrew B., and Steven N. Durlauf. (1995). Convergence in international output. Journal of Applied Econometrics, 10(2), 97-108. Bernard, Andrew B., and Steven N. Durlauf. (1996). Interpreting tests of the convergence hypothesis. Journal of Econometrics, 71(1-2), 161-173. Berry, Steven, James Levinsohn, and Ariel Pakes. (1995). Automobile prices in market equi- librium. Econometrica, 63(4), 841-890. Black, Dan A., and Daniel S. Nagin. (1998). Do right-to-carry laws deter violent crimes? Journal of Legal Studies, 27(1), 209-219. Blundell, Richard, and Thomas Stoker. (2005). Heterogeneity and aggregation. Journal of Economic Literature, 43(2), 347-391.

ON THE USE OF AGGREGATE CRIME REGRESSIONS 239 Brock, William A., and Steven Durlauf. (2001). Growth economics and reality. World Bank Economic Review, 15, 229-272. Brock, William A., Steven Durlauf, and Kenneth West. (2003). Policy analysis in uncertain economic environments (with discussion). Brookings Papers on Economic Activity, 1, 235-322. Brock, William A., Steven Durlauf, and Kenneth West. (2007). Model uncertainty and policy evaluation: Some theory and empirics. Journal of Econometrics, 136(2), 629-664. Brock, William A., Steven Durlauf, James Nason, and Giacomo Rondina. (2007). Simple versus optimal rules as guides to policy. Journal of Monetary Economics, 54(5), 1372-1396. Cantor, David, and Kenneth C. Land. (1985). Unemployment and crime rates in the post- World War II United States: A theoretical and empirical investigation. American Socio- logical Review, 50, 317-332. Cohen-Cole, Ethan, Steven Durlauf, Jeffrey Fagan, and Daniel Nagin. (2008). Model uncer- tainty and the deterrent effect of capital punishment. American Law and Economics Review, forthcoming. Dezhbakhsh, Hashem, Paul Rubin, and Joanna M. Shepherd. (2003). Does capital punishment have a deterrent effect? New evidence from post-moratorium panel data. American Law and Economics Review, 5(2), 344-376. Donohue, John, and Steven Levitt. (2001). The impact of legalized abortion on crime. Â uarterly Journal of Economics, 116(2), 379-420. Q Donohue, John, and Justin Wolfers. (2005). Uses and abuses of empirical evidence in the death penalty debate. Stanford Law Review, 58(3), 791-846. Doppelhofer, Gernot, Ronald I. Miller, and Xavier Sala-i-Martin. (2004). Determinants of long-term growth: A Bayesian averaging of classical estimates (BACE) approach. Â merican Economic Review, 94(4), 813-835. A Draper, David. (1995). Assessment and propagation of model uncertainty. Journal of the Royal Statistical Society, Series B, 57, 45-70. Durlauf, Steven, Salvador Navarro, and David Rivers. (2008). Notes on the econometrics of deterrence. Unpublished working paper, Department of Economics, University of Wisconsin, Madison. Ehrlich, Isaac. (1972). The deterrent effect of criminal law enforcement. Journal of Legal Studies, 1(2), 259-276. Ehrlich, Isaac. (1973). Participation in illegal activities: A theoretical and empirical investiga- tion. Journal of Political Economy, 81(3), 521-565. Fernandez, Carmen, Eduardo Ley, and Mark Steel. (2001). Model uncertainty in cross-country growth regressions. Journal of Applied Econometrics, 16(5), 563-576. Garratt, Anthony, Kevin Lee, M. Hashem Pesaran, and Yongcheol Shin. (2003). Forecasting uncertainties in macroeconometric modelling: An application to the UK economy. Journal of the American Statistical Association, 98(464), 829-838. Gaus, Gerald F. (2006). Social complexity and evolved moral principles. Unpublished paper, University of Arizona. Gelman, Andrew, James S. Liebman, Valerie West, and Alexander Kiss. (2004). A broken sys- tem: The persistent patterns of reversals of death sentences in the United States. Journal of Empirical Legal Studies, 1(2), 209-261. Goldberger, Arthur S. (1991). A course in econometrics. Cambridge, MA: Harvard University Press. Greenberg, David. (2001). Time series analysis of crime rates. Journal of Quantitative Crimi- nology, 17(4), 291-327. Hansen, Lars Peter, and Thomas J. Sargent. (2007). Robustness. Princeton, NJ: Princeton University Press.

240 UNDERSTANDING CRIME TRENDS Heckman, James L. (2000). Causal parameters and policy analysis in economics: A twentieth century retrospective. Quarterly Journal of Economics, 115(1), 45-97. Heckman, James L. (2005). The scientific model of causality. Sociological Methodology, 35, 1-98. Heckman, James L., and Salvador Navarro. (2004). Using matching, instrumental variables, and control functions to estimate economic choice models. Review of Economics and Statistics, 86(1), 30-57. Heckman, James L., and Salvador Navarro. (2007). Dynamic discrete choices and dynamic treatment effects. Journal of Econometrics, 136(2), 341-396. Horowitz, Joel L. (2005). Statistical issues in the evaluation of right-to-carry laws. In National Research Council, Firearms and violence: A critical review (pp. 299-308), Committee to Improve Research Information and Data on Firearms, Charles F. Wellford, John V. P Â epper, and Carol V. Petrie (Eds.), Committee on Law and Justice, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press. LaFree, Gary. (2005). Evidence for elite convergence in cross-national homicide victimization trends, 1956 to 2000. The Sociological Quarterly, 46, 191-211. Leamer, Edward E. (1978). Specification searches. New York: John Wiley & Sons. Levitt, Steven D. (1996). The effect of prison population size on crime rates: Evidence from prison overcrowding litigation. Quarterly Journal of Economics, 111, 319-351. Levitt, Steven D. (2004). Understanding why crime fell in the 1990s: Four factors that explain the decline and six that do not. Journal of Economic Perspectives, 18(1), 163-190. Lott, John R. (1998). The concealed-handgun debate. Journal of Legal Studies, 27(1), 221-291. Lott, John R., and David B. Mustard. (1997). Crime, deterrence, and right-to-carry concealed handguns. Journal of Legal Studies, 26(1), 1-68. Mankiw, N. Gregory, David Romer, and David N. Weil. (1992). A contribution to the empirics of economic growth. Quarterly Journal of Economics, 107(2), 407-437. Manski, Charles F. (2005). Statistical treatment rules for heterogeneous populations. Econo- metrica, 72, 1221-1246. Manski, Charles F. (2006). Minimax-regret treatment choice with missing outcome data. Journal of Econometrics, 139(1), 105-115. Manski, Charles F. (2007). Identification for prediction and decision. Cambridge, MA: H Â arvard University Press. Marschak, Jacob. (1953). Economic measurements for policy and prediction. In William C. Hood and Tjalling C. Koopmans (Eds.), Studies in econometric method. New York: John Wiley & Sons. Matzkin, Rosa L. (1992). Nonparametric and distribution-free estimation of the binary thresh- old crossing and the binary choice models. Econometrica, 60(2), 239-270. National Research Council. (2005). Firearms and violence: A critical review. Committee to Â Improve Research Information and Data on Firearms, Charles F. Wellford, John V. P Â epper, and Carol V. Petrie (Eds.), Committee on Law and Justice, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press. Navarro, Salvador. (2007). Control functions. In Lawrence E. Blume and Steven Durlauf (Eds.), New Palgrave dictionary of economics (revised ed.). London: Palgrave-Macmillan. OâBrien, Robert M. (1999). Measuring the convergence/divergence of âserious crimeâ arrest rates for males and females: 1960-1995. Journal of Quantitative Criminology, 15(1), 97-114. Plassmann, Florenz, and John Whitley. (2003). Confirming âmore guns, less crime.â Stanford Law Review, 55, 1313-1369. Quine, Willard Van Orman. (1951). Two dogmas of empiricism. Philosophical Review, 60, 20-43.

ON THE USE OF AGGREGATE CRIME REGRESSIONS 241 Raftery, Adrian E., David Madigan, and Jennifer A. Hoeting. (1997). Bayesian model averag- ing for linear regression models. Journal of the American Statistical Association, 92(437), 179-191. Savage, Leonard J. (1951). The theory of statistical decision. Journal of the American Statisti- cal Association, 46, 55-67. Sen, Amartya. (1979). Utilitarianism and welfarism, Journal of Philosophy, 76(9), 463-489. Stoker, Thomas M. (1993). Empirical approaches to the problem of aggregation over indiÂ viduals. Journal of Economic Literature, 31(4), 1827-1874. Sunstein, Cass R., and Adrian Vermeule. (2005). Is capital punishment morally required? Acts, omissions and life-life tradeoffs. Stanford Law Review, 58(3), 703-751. Wald, Abraham. (1950). Statistical decision functions. New York: John Wiley & Sons.