The response problems described above, however, cannot be ignored. To the contrary, these measurement problems may lead to substantial biases in unknown directions. If, for example, respondents are inclined to report being victimized when a crime is “successful” but conceal unsuccessful crimes, the estimated efficacy of resistance will be biased downward. In contrast, if respondents, concerned about being perceived as inept, are inclined to report successful forms of resistance but conceal ineffective forms, the estimated efficacy of self-defense will be biased upward. Without better information on the nature and extent of response problems, it is impossible to know whether and how the estimated associations between defensive gun use, crime, and injury are biased. If, as Kleck and Gertz (1995) suggest, the NCVS misses over 2 million defensive uses per year, then biases caused by reporting errors may be substantial.
Subjective assessments on the efficacy of defensive gun use have been elicited in both the NCVS and the NSDS. Data from the 1994 NCVS, for example, reveal that 65 percent of victims felt that self-defense improved their situation, while 9 percent thought that it worsened their situation (Kleck, 2001a). More direct counterfactual questions were asked in the NSDS survey, in which respondents who reported using a firearm were asked (Kleck and Gertz, 1995:316):
If you had not used a gun for protection in this incident, how likely do you think it is that you or someone else would have been killed? Would you say almost certainly not, probably not, might have, probably would have, or almost certainly would have been killed?
Nearly half of respondents perceived that someone might, probably, or almost certainly would have been killed.
Although intriguing, these assessments are of limited value. Certainly, there are obvious concerns about inaccurate reporting associated with subjective questions. Victims may be inclined to view their actions as effective regardless and may exaggerate counterfactual outcomes. Even if victims report truthfully, the existing questionnaires provide little guidance. What does a respondent mean when he states that someone might have been killed? Are all respondents using consistent criteria to interpret these questions?
A number of researchers have attempted to infer the defensive utility of firearms by examining the firearms deaths that occur in or near the victim’s