Provide a coherent and explicit chain of reasoning.
Replicate and generalize across studies.
Disclose research to encourage professional scrutiny and critique.
While any group of scholars might modify this list, it poses some commonly accepted standards that our committee used to begin its evaluation of the literature on firearm violence. In so doing we have sought to ensure that often controversial research issues are subjected to these minimum standards for research and to encourage future research in this area to strive for greater rigor.
The committee also noted that certain research strategies are very prevalent in firearms research. These include various interrupted time-series approaches (before-after studies) and the use of case-control techniques. Because these are so frequently utilized in this area of research, we provide an analysis of their use. In Appendix D there is a discussion of the difficulties of before-after type studies, and in Chapter 7 there is one on case-control designs. For advances to be made in firearm violence research, researchers must be careful to use these techniques and approaches with due recognition of their limitations and carefully consider the effect of research design on findings.
In our analysis of the use of these methods in firearms research, we found too often that the conclusions reached require the acceptance of assumptions that are at best implausible. For example, many studies (e.g., Duggan, 2001; Kaplan and Geling, 1998; Kleck and Patterson, 1993; Miller et al., 2002) of the relationship between the access to firearms and firearm violence are conducted with the state as the unit of analysis (a measure of the rate of firearm ownership is correlated with the rate of firearm violence). These results are used to advance the argument that an individual’s probability of access to firearms explains that individual’s probability of committing a violent crime with a weapon. While the problems associated with such cross-level interpretations are well known (the “ecological fallacy”; that is, inferences about individual behavior cannot be drawn from aggregate data about a group; Robinson, 1950), these authors and many who use their work to advance various firearms policies all too frequently draw inferences that cannot be supported by their analysis. Similarly in interrupted-time-series designs, the length of the series and the well-known problems associated with nonexperimental and quasi-experimental designs (see Campbell and Stanley, 1966) are frequently not given the attention required for the work to be judged acceptable. Throughout this report we hold all the research we reviewed to these reasonable standards. Especially in areas of research in which there is much public controversy, it is vital that such standards be maintained.