2
Data for Measuring Firearms Violence and Ownership

Scientists in the social and behavioral sciences deal with many data-related obstacles in conducting empirical research. These include lack of relevant data, data that are error-ridden, and data that are not based on properly designed statistical samples (i.e., are unrepresentative) of the targeted population. These obstacles are particularly difficult in firearms research. In firearms and violence research, the outcomes of interest, although large in absolute numbers, are statistically rare events that are not observed with great frequency, if at all, in many ongoing national probability samples. Moreover, response problems are thought to be particularly severe in surveys of firearms ownership and violence. In the committee’s view, the major scientific obstacle for advancing the body of research and further developing credible empirical research to inform policy on firearms is the lack of reliable and valid data.

This chapter summarizes some of the key data collection systems used to assess firearms policies, describes some of the key properties of useful research data, and offers some suggestions for how to begin to develop data that can answer the basic policy questions. There are no easy solutions to resolving the existing data-related problems. Often, we find that the existing data are insufficient, but how and whether to develop alternative data sources remain open questions. For these reasons the committee urges a significant increase in methodological work on measurement in the area of firearms ownership and violence.

The committee does not wish to paint an overly pessimistic picture of this research area. The existing body of research, as described in the other chapters of the report, has shed light on some of the most fundamental



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 19
Firearms and Violence: A Critical Review 2 Data for Measuring Firearms Violence and Ownership Scientists in the social and behavioral sciences deal with many data-related obstacles in conducting empirical research. These include lack of relevant data, data that are error-ridden, and data that are not based on properly designed statistical samples (i.e., are unrepresentative) of the targeted population. These obstacles are particularly difficult in firearms research. In firearms and violence research, the outcomes of interest, although large in absolute numbers, are statistically rare events that are not observed with great frequency, if at all, in many ongoing national probability samples. Moreover, response problems are thought to be particularly severe in surveys of firearms ownership and violence. In the committee’s view, the major scientific obstacle for advancing the body of research and further developing credible empirical research to inform policy on firearms is the lack of reliable and valid data. This chapter summarizes some of the key data collection systems used to assess firearms policies, describes some of the key properties of useful research data, and offers some suggestions for how to begin to develop data that can answer the basic policy questions. There are no easy solutions to resolving the existing data-related problems. Often, we find that the existing data are insufficient, but how and whether to develop alternative data sources remain open questions. For these reasons the committee urges a significant increase in methodological work on measurement in the area of firearms ownership and violence. The committee does not wish to paint an overly pessimistic picture of this research area. The existing body of research, as described in the other chapters of the report, has shed light on some of the most fundamental

OCR for page 19
Firearms and Violence: A Critical Review questions related to firearms and violence. However, in key data areas—the availability of firearms, the use of firearms, and the role of firearms in injuries and death—critical information is absent. A PATCHWORK OF DATA SETS To study firearms and violence, researchers and policy makers rely on a patchwork of data sources collected for more general purposes of monitoring the nation’s health and crime problems. No authoritative source of information exists to provide representative, accurate, complete, timely, and detailed data on the incidence and characteristics of firearm-related violence in the United States. Rather, there are many different sources of data that researchers use to draw inferences about the empirical questions of interest. Some information on firearms and violence is found in probability samples of well-defined populations, such as the National Crime Victimization Survey (NCVS) and the General Social Survey (GSS). Other information comes from administrative data, such as the Uniform Crime Reports (UCR) and the trace data of the Bureau of Alcohol Tobacco and Firearms (BATF). Still other information comes from case studies, social experiments, and other one-time surveys conducted on special populations. Table 2-1 lists characteristics of some of the commonly used data sources. Perhaps because these data sets serve many purposes, the strengths and limitations of each source have been generally well documented in the literature.1 This section provides a brief description of the some of the key data sources used in the research literature on firearms injury and violence and discussed in the report. This summary is not an exhaustive treatment of the data sources listed in Table 2-1, nor is it complete in its assessment of the specific data sources considered. Rather, it is intended to provide relevant background material on the key data. Data on Violence and Crime It is axiomatic that reliable and valid surveys on violence, offending, and victimization are critical to an understanding of violence and crime in the 1   See, for example, Annest and Mercey (1998); Biderman and Lynch (1991); Maltz (1999); MacKenzie et al. (1990); Jarvis (1992); Wiersema et al. (2000); and Riedel (1999). The National Opinion Research Center (NORC) produces an ongoing series of methodological reports on the GSS, covering topics ranging from item order and wording, to nonresponse errors, and hundreds of other methodological topics. The reports are available directly from the NORC and are listed on http://www.icpsr.umich.edu:8080/GSS under “GSS Methodological Reports.”

OCR for page 19
Firearms and Violence: A Critical Review United States and for any assessment of the quality of activities and programs aimed at reducing violence (National Research Council, 2003). Detailed data on firearm-related death, injury, and risk behaviors are limited. Most measurement of crime in this country emanates from two major data sources. The FBI’s Uniform Crime Reports has collected information on crimes known to the police and arrests from local and state jurisdictions throughout the country for almost seven decades. The National Crime Victimization Survey, a general population survey designed to discover the extent, nature, and consequences of criminal victimization, has existed since the early 1970s. Other national surveys that focus on specific problems, such as delinquency, violence against women, and child abuse, also provide important data on crime, victims, and offenders. A variety of data sources have been used to assemble information on suicide and accidents, and the National Violent Death Reporting System (NVDRS) has been funded via the Centers for Disease Control and Prevention (CDC) to collect information on all violent deaths. In this section, we describe four datasets used to monitor and assess firearms-related violence: the National Crime Victimization Survey, the Uniform Crime Reports, and two emerging systems, the National Incident-Based Reporting System and the National Violent Death Reporting System. The latter two are thought to hold some promise for improving the research information on firearms and violence. Many of the other data collection sources (listed in Table 2-1) have very limited information on firearms and have been assessed elsewhere (see, for example, Annest and Mercy, 1998; Institute of Medicine, 1999). National Crime Victimization Survey The National Crime Victimization Survey, which relies on self-reports of victimization, is an ongoing annual survey conducted by the federal government (i.e., the Census Bureau on behalf of the Department of Justice) that collects information from a representative sample of nearly 100,000 noninstitutionalized adults (age 12 and over) from approximately 50,000 households. It is widely viewed as a “gold standard” for measuring crime victimization. The largest and oldest of the crime victimization studies, it uses a rotating panel design in which respondents are interviewed several times before they are “retired” from the sample. It uses a relatively short, six-month reporting period. Respondents are instructed to report only incidents that have occurred since the previous interview and are reminded of the incidents they reported then. The initial interview is done face-to-face to ensure maximum coverage of the population; if necessary, subsequent interviews are also conducted in person. The

OCR for page 19
Firearms and Violence: A Critical Review TABLE 2-1 Selected Sources of Firearm Data Title of Data Set Sponsoring Agency Information Available Firearm-Related Injury/Death National Vital Statistics System—Final Mortality Data (NVSSF) National Center for Health Statistics/ Centers for Disease Control and Prevention Includes total numbers of firearm related deaths; death rates from homicide, suicide, unintentional, and undetermined shootings broken out by age, race, and sex National Vital Statistics System—Current Mortality Sample (NVSS) National Center for Health Statistics/ Centers for Disease Control and Prevention Provides data on selected major causes of death, as well as sex, race, age, date of death, state in which death occurred National Violent Death Reporting System (NVDRS) Centers for Disease Control and Prevention Data on violent deaths linked from medical examiners and coroners, police departments, death certificates, and crime labs; would include circumstances of firearm-related incidents National Census of Fatal Occupational Injuries (CFOI) Bureau of Labor Statistics Complete count of all work-related injury fatalities; includes job-related homicides broken out by weapon Survey of Occupational Injuries and Illnesses (SOH) Bureau of Labor Statistics Includes information on circumstances surrounding firearm-related injuries in the workplace National Traumatic Occupational Fatality Surveillance System (NTOF) National Institute for Occupational Safety and Health Includes narrative text on industry, occupation, cause of death, and injury data on age, race, and sex; includes numbers and rates of firearm-related homicides, suicides, and other deaths occurring at work National Electronic Injury Surveillance System All Injury Program (NEISSAIP) U.S. Consumer Products Safety Commission Includes intentional and nonintentional nonfatal firearm-related injuries by gender, age, type of injury, type of gun, and nature of incident National Hospital Ambulatory Medical Care Survey (NHAMCS) National Center for Health Statistics Injury visits to hospital emergency departments, including those caused by firearms

OCR for page 19
Firearms and Violence: A Critical Review Population Geographic Areas Frequency/Year Started Deceased individuals (data from death certificates) National Annual/death registration for all states started 1933, detailed demographic data ftom 1989 Deceased individuals (data from death certificates) National Annual death registration for all states started 1933 Homicide, suicide, and unintentional firearmrelated deaths, and deaths of undetermined causes National Under development Employed civilians 16 years of age and older, plus resident armed forces National Annual/started 1992 Injuries reported by employers in private industry National Annual/started 1992 Workers age 16 and older certified on death certificate as injured at work National Data available from 1980 Admissions to hospital emergency departments National Updated daily/ redesigned 1978; all injuries included starting in 2000 Admissions to hospitals with emergency departments National Annual/started 1992

OCR for page 19
Firearms and Violence: A Critical Review Title of Data Set Sponsoring Agency Information Available National Ambulatory Medical Care Survey (NAMCS) National Center for Health Statistics Includes age, sex, race, ethnicity, source of payment, and circumstances of injury-related visits, including firearm involvement National Health Interview Survey (NHIS) National Center for Health Statistics Demographic information, physician and hospital visits, and other health-related information; includes gunshot wounds and type of gun; 1994 supplement on firearm storage and safety National Mortality Followback Survey (NMFS) National Center for Health Statistics 1993 survey included information on firearm access, and circumstances of homicide, suicide, and unintentional injury deaths Data Elements for Emergency Department Systems (DEEDS) National Center for Injury Prevention and Control (CDC) Standardized data definitions, coding, and other specifications International Classification of External Causes of Injury (CECI) World Health Organization External causes of injury in mortality and morbidity systems, including mechanism of injury Firearms Industry and Retail Annual Firearms Manufacturing and Exportation Report (AFMER) Bureau of Alcohol, Tobacco, and Firearms Number of firearms produced, by type Census of Manufacturers Bureau of the Census Number of manufacturers, shipments, value, employment, payroll, and shipments by type of product for small arms manufacturing and small arms ammunition industries Producer Price Index (PPI) Bureau of Labor Statistics Prices and price change at wholesale level for various categories of firearms, including “small arms” in general, “pistols and revolvers,” “shotguns,” and “rifles, centerfire”

OCR for page 19
Firearms and Violence: A Critical Review Population Geographic Areas Frequency/Year Started Patient visits to office-based, nonfederally employed physicians National Annual/ 1995—detailed injury questions added, 1997-intent of injury added Civilian, noninstitutionalized U.S. households National Annual/ 1996—detailed injury section added Persons age 15 and older who died in the year of the survey National Irregular frequency/started in 1960s 24-hour, hospital-based emergency departments National Under development Hospital emergency department records International Under development Firearms manufacturers National Annual Manufacturers National   Producers in the mining and manufacturing industries National Monthly/started 1902

OCR for page 19
Firearms and Violence: A Critical Review Title of Data Set Sponsoring Agency Information Available Federal Firearms Licensee (FFL) List Bureau of Alcohol, Tobacco, and Firearms Licensee name, trade name, address, phone, and license number Criminal Use of Firearms National Crime Victimization Survey (NCVS) Bureau of Justice Statistics Victimizations, involving a firearm, by type of crime Uniform Crime Reporting Program (UCR): Monthly Return of Offenses Known to Police Federal Bureau of Investigation Total numbers of specific violent and property crimes, includes counts of weapon type used for robberies and aggravated assaults Uniform Crime Reporting Program: National IncidentBased Reporting System (NIBRS) Federal Bureau of Investigation Incident, victim, property, offender, and arrestee data on each incident and arrest in 22 crime categories Uniform Crime Reporting Program: Supplemental Homicide Reports (SHR) Federal Bureau of Investigation Detailed descriptions of homicides, including weapon used Youth Crime Gun Interdiction Initiative (YCGII) Bureau of Alcohol, Tobacco, and Firearms Proportion of crime guns that are recovered from juveniles, youth, and adults; top source states; type of gun used; “time to crime” BATF Firearms Trace Data Bureau of Alcohol, Tobacco, and Firearms Firearms transaction records kept by federal firearms licensees, including date of sale and name of purchaser Law Enforcement Officers Killed and Assaulted (LEOKA) Federal Bureau of Investigation Duty-related deaths and assaults of law enforcement officers, by weapon used in incident Federal Justice Statistics Program (FJSP) Bureau of Justice Statistics Data on federal criminal case processing from the receipt of a criminal matter or arrest of suspect to release from prison into supervision

OCR for page 19
Firearms and Violence: A Critical Review Population Geographic Areas Frequency/Year Started Federal firearms licensees, except collectors of curios and relics National   Persons 12 years of age and older National Annual/started 1973 Crimes reported by city, county, and state law enforcement agencies National Monthly/started 1930 Criminal incidents reported by local, state, and federal law enforcement agencies National Started 1989, under development Criminal incidents reported by police departments National Started 1976 Guns recovered from juveniles and adult criminals 55 cities in 2001 Annual/started 1997 Firearms submitted by law enforcement for tracing National Record-keeping started 1968 Local, state, and federal law enforcement officers National Annual Defendants in criminal cases, suspects in investigative matters, and offenders under supervision National Annual

OCR for page 19
Firearms and Violence: A Critical Review Title of Data Set Sponsoring Agency Information Available Census of State and Federal Correctional Facilities Bureau of Justice Statistics/ Bureau of the Census Demographic, socioeconomic, and criminal history characteristics, including gun possession and use National Violence Against Women Survey (NVAWS) National Institute of Justice/Centers for Disease Control and Prevention Prevalence, incidence, characteristics, risk factors, circumstances, responses, and consequences of rape, intimate-partner assault and stalking; includes data on firearm use in these events Arrestee Drug Abuse Monitoring (ADAM~-gun addendum National Institute of Justice Gun acquisition and use among arrestees, including gun carrying, reasons for owning a gun, being threatened with a gun, and drug use Firearms and Youth Monitoring The Future (MTF) National Institute on Drug Abuse Range of behaviors and attitudes with focus on drug use; includes frequency of gun carrying at school Youth Risk Behavior Surveillance System (YRBSS) Centers for Disease Control and Prevention Prevalence of health risk behaviors including gun-carrying, weapon carrying on school property, and weapon-related threats or injuries on school property Law and Enforcement Firearm Inquiry Statistics (FIST) Bureau of Justice Statistics Handgun applications made to FFLs, applications rejected, and reasons for rejection

OCR for page 19
Firearms and Violence: A Critical Review Population Geographic Areas Frequency/Year Started State correctional facility inmates National Every 5 to 7 years/started 1974 U.S. households National Unrepeated/ conducted 1996 Arrestees charged with felonies and misdemeanors National (gun addendum includes 11 of the 35 sites) 1996—gun addendum 6th, 8th, 10th, and 12th graders and young adults up to age 19 National Annual/ started 1972, gun question added in 1996 School-age youth grades 9 through 12; also 12- to 21-year-olds in 1992 and college students in 1995 National Every two years/started 1990 Chief law enforcement officers States operating under the Brady Act and states with statutes comparable to the Brady Act Started 1995

OCR for page 19
Firearms and Violence: A Critical Review correlate better than other possible proxies with measures of gun violence (homicide and gun assaults). As we discuss in Chapter 7, proxies raise two somewhat related but distinct methodological issues. First, proxies have been used at aggregated levels, most often the state level, to infer something about the impact of availability at the individual level on violent outcomes. For example, if the proxy is correlated with gun homicides at the state level, then it is often assumed that availability at the individual level of analysis is associated with individual manifestations of violence. More generally, these studies are used to infer whether an individual’s probability of access to firearms explains his or her probability of committing a violent crime or suicide. Aggregate measures of ownership, however, may or may not be related to actual availability in the households in which these rare events (homicides and suicides) occur. A second issue with proxies is to what extent they are inaccurate indicators of firearms availability at the geographic level of interest. Proxies create biases, yet there is almost no research on these statistical problems in the firearms literature. Without more rigorous evaluations on the impact of proxies, it is difficult to assess the research on ownership and violence. Once these biases have been assessed, proxies may be useful because they are cheaper to collect, their collection is less intrusive, and for other reasons of economy or design. The research community in this area needs to focus more attention on assessing the biases created by proxies and on the development of better direct measures of availability and use. GENERAL OBJECTIVES FOR DEVELOPING USEFUL RESEARCH DATA In this section, we discuss several basic features that data on firearm ownership and violence ought to exhibit, individually or in combination, in order for researchers, practitioners, and policy makers to better understand the role of firearms in violent injury and death, both self-and other-inflicted. In particular, the following qualities of data sets are minimally necessary for credible research and evaluation on firearm violence: representativeness, accuracy, comprehensiveness, standardization, and timeliness. Representativeness A fundamental component of any scientific data set is that it represents some population of interest in a known way. The textbook scheme is to randomly sample from a known population, but other well-defined sam-

OCR for page 19
Firearms and Violence: A Critical Review pling schemes are also used to draw inferences about known populations. The NCVS uses a complex random sampling scheme. In Chapter 7, there is a detailed discussion of case-control schemes that can be especially useful for studying rare events like violence and crime. Many of the data sets used to study firearms and violence are not random samples from well-defined populations of interest, nor are they exhaustive enumerations of any population. These types of data may provide some information, as described above, but using them to assess the effects of policy can be more complicated. Accuracy Accuracy of measurement is an essential criterion for a data source to be useful for understanding firearms and violence. Two key features of accuracy are the validity and reliability of measurement. In general terms, a measure is valid to the degree that it represents the underlying phenomenon of interest, and it is reliable to the degree that it yields the same data over repeated applications. Many of the debates over the relationship between firearms and violence center on questions of validity and reliability. For example, some analysts question the validity of the NCVS for measuring the prevalence of defensive firearms use because, as a survey of crime victims, the NCVS may not fully capture crimes that are averted by the use of firearms. Other researchers question the reliability of one-time sample surveys for measuring rare events, such as defensive use of guns. The chief function of data standardization is to ensure reliability of measurement. The more comprehensive a system, the more likely it will yield valid measurements of the connection between firearms and violence. Response errors are a vital component of the validity of any data. The validity of data that measure firearms ownership, use, and violence on the basis of respondent self-reports depends on the ability and willingness of persons to disclose highly personal and sometimes incriminating or traumatic information to interviewers. As discussed above, there are reasons to expect response errors in regard to questions about ownership and use, as elicited in the GSS and other gun use surveys. Although there is much speculation on the extent and nature of response errors (see Chapter 5), there is almost no relevant research. Likewise, validity is compromised by nonresponse rates ranging from 20 percent (in the GSS) to over 50 percent in some of the phone surveys used to measure ownership. Without making unsubstantiated assumptions about gun ownership among nonrespondents, the GSS data cannot reveal whether ownership is increasing or decreasing over time.

OCR for page 19
Firearms and Violence: A Critical Review Comprehensiveness The criterion of comprehensiveness refers to both a data set’s scope and richness of detail with respect to firearm-related violence. Scope Scope can be subdivided into the types of events that are captured and the populations covered. The scope of the NCVS, for example, is restricted to nonfatal incidents and to the characteristics of crime victims rather than offenders. Vital statistics and hospital-based information on firearm violence is also limited to the victims. The UCR, by contrast, captures information on both crime victims and offenders, but they are limited to offenses that are known to and recorded by law enforcement agencies. The NCVS includes data on both crimes reported to the police and those that victims do not report. Household-based surveys such as the NCVS and the GSS are limited to the population of persons with stable residences, thereby omitting transients and other persons at high risk for firearm violence. Such persons are included in the ADAM program, which collects information on persons who come into contact with the criminal justice system. Geographic coverage is another dimension of scope. The GSS, for example, is representative of the United States and the nine census regions, but it is too sparse geographically to support conclusions at finer levels of geographical aggregation. This lack of individual-level data from small geographical areas is a significant shortcoming in the firearms data. Presumably, we would like to be able to make statements about, for example, the probability that an individual commits suicide conditional on owning a gun (or having one available) and other covariates. This cannot be done if the smallest geographical unit that the data resolve is a multistate region. Similar statements can be made about other forms of gun violence. Perhaps no better illustration of the patchwork character of information on firearms violence in the United States exists than the multiple and nonoverlapping or partially overlapping coverage of the data sets. That should come as little surprise, inasmuch as many of the data sets were expressly intended to provide information about crime, violence, or injury that was not available from other sources. The major impetus for the development of the NCVS, for example, was to gather information on crime incidents that do not come to the attention of law enforcement agencies. The collection of information on violence from hospitals and emergency departments is intended to reveal types of violence, such as partner abuse, thought to be underreported in crime data sources. The patchwork of existing data sources, in other words, has been created with the best of intentions and has shed light on aspects of violence, including the role of firearms, that otherwise would have remained hidden

OCR for page 19
Firearms and Violence: A Critical Review from view, such as the burden on hospital emergency departments of firearm injuries (Zawitz and Strom, 2000). However, insufficient attention has been devoted to linkages across data in population coverage and the types of firearm violence covered. Can data from the UCR, the NCVS, and emergency departments be effectively linked to draw inferences about the firearms violence in the population? As with data standardization, continuing assessments of remaining gaps in the scope of firearms data should be part of an ongoing program of methodological research on firearm violence. Context An often-highlighted limitation of existing data on firearms is the lack of detail regarding the context and circumstances of firearm violence. The Supplemental Homicide Report provides limited information on the relationship between victim and offender and event circumstances (e.g., whether the homicide is related to an argument or the commission of another felony). The National Incident-Based Reporting System extends such information to other crime types, but it covers less than 20 percent of the population more than 20 years after nationwide implementation began. Youth surveys, such as Monitoring the Future (MTF) and the Youth Risk Behavior Surveillance System, collect data on multiple attributes of respondents in addition to firearm behaviors, but little information on the situations in which youth carry and use firearms. The MTF survey also includes a longitudinal component that tracks respondents over time. These panel data might be especially useful for assessing firearms acquisition and use over time. However, citing agreements with respondents regarding confidentiality, the University of Michigan’s Institute for Social Research has not made these data available to external researchers (see National Research Council, 2001). The most promising emerging data source with respect to information on the context and circumstances of firearm violence is the National Violent Death Reporting System, which will compile individual-level data from both criminal justice and public health sources on event circumstances, as well as detailed descriptions of the weapons used in violence. The NVDRS offers a model of a comprehensive data set that bridges existing data sources on individuals, events, and weapons. Standardization An essential quality of any measurement system is the collection of standard data elements from reporting units for purposes of reliable classification and comparison. Good examples of standardized data sets for measuring firearm violence are the FBI’s UCR program, the National Crime Victimization Survey, and the mortality files available from the National

OCR for page 19
Firearms and Violence: A Critical Review Vital Statistics System. Each of these data sets provides detailed formats and instructions for data collection, coding, and entry to ensure standard measurement of underlying data elements. For example, the UCR program regularly compiles information on eight serious “index offenses” (murder and nonnegligent manslaughter, rape, robbery, aggravated assault, burglary, larceny, vehicle theft, and arson) and requires local law enforcement agencies to use the same crime classification when compiling data on these offenses for reporting to the UCR. The National Vital Statistics System classifies deaths according to the International Classification of Diseases codes for cause of death. Such standard classification and coding schemes, however, are necessary but not sufficient for ensuring valid and reliable measurement. Ultimately, all data must rely on the faithfulness of their reporting units in adhering to the standard protocols, which requires continuous monitoring of data collection and adequate training of data entry personnel. All of the federally sponsored data sets that collect information on firearm violence have procedures in place to maintain standard data collection, although they vary in the degree of compliance exhibited by reporting units. Generally speaking, systems with direct control over reporting units are able to maintain higher levels of standardization. The NCVS, administered by the Census Bureau in cooperation with the Bureau of Justice Statistics, is a good example of a data source with direct control over data collection. The UCR, in contrast, has no direct control over local data collection and must rely on data checks conducted by state UCR programs, as well as its own quality controls, to ensure adherence to standard coding and classification criteria. The National Vital Statistics System mortality series lie somewhere between the NCVS and the UCR with respect to direct control over local data collection. We have limited our discussion thus far to standardization within data sets. However, because data on firearms violence comes from multiple sources and will continue to for the foreseeable future, we also must be concerned with standardization of data elements between data sources. Ongoing investigations of comparable data elements from different sources should constitute an essential part of a program of methodological research on firearm-related violence. Moreover, new and emerging data sets should be designed to ensure transparent linkages of data elements with existing data sources. Two of the most important needs identified in public health and criminological research on violence and other injuries are for the standardization of data elements and the availability of detailed characteristics surrounding each event. Several efforts under way to address these concerns, if successful, may improve the usefulness and quality of data on firearm-related deaths and injuries: the National Incident-Based Reporting System, the

OCR for page 19
Firearms and Violence: A Critical Review National Violent Death Reporting System, the Data Elements for Emergency Department Systems, and the International Classification of External Cause of Injury coding system. The NIBRS and the NVDRS have been discussed; the latter two systems are described below. Data Elements for Emergency Department Systems: CDC’s National Center for Injury Prevention and Control is coordinating an effort to develop uniform specifications for data entered into emergency department records. These specifications, known as DEEDS, are intended for use in 24-hour, hospital-based emergency departments throughout the United States. If the data definitions, coding conventions, and other recommended specifications were widely adopted, incompatibilities between emergency departments records would be substantially reduced. DEEDS does not specify an essential or minimum data set, but is designed to foster greater uniformity among individual data elements chosen for use. DEEDS also specifies standards for electronic data interchange so that data can be accessed for research purposes while maintaining confidentiality of patient records. DEEDS was first released in 1997 for testing and review. Systematic field studies, however, are still needed to assess the utility and practicality of the system. International Classification of External Causes of Injury: An international effort, under the auspices of the World Health Organization, is currently under way to develop a new classification system for coding external causes of injury in mortality and morbidity systems. This system, known as the International Classification of External Causes of Injury (ICECI) is designed to capture details about the place of occurrence, activity at time of injury, alcohol and drug involvement, objects or substances involved, intent of injury, and mechanism of injury (e.g., firearms). Specific modules that focus on injuries related to violence, transportation, sports, and work are also under development. The first draft was released in 1998; the present version, ICECI 1.0, was released in 2001. A number of shortened versions have been tested for use as injury surveillance tools in places with limited resources for surveillance. CDC has tested its own short version as a means for capturing external cause of injury information from hospital emergency departments records in the United States with promising results. The European Union is also testing portions of ICECI as part of its efforts to create a minimum data set on injuries. ICECI is designed to replace the International Classification of Diseases coding system, which is thought to lack the scope and specificity needed to inform injury research. The present version of ICECI is undergoing formal review at the World Health Organization.11 11   Details about ICECI 1.0, including the data dictionary, are available at http://www.iceci.org.

OCR for page 19
Firearms and Violence: A Critical Review Timeliness One remarkable feature of all existing data sources on firearms violence is their lack of timeliness. Other social indicators, particularly those measuring economic activity and performance, are available on a quarterly or monthly basis. By contrast, researchers, practitioners, and policy makers concerned with violent injury and death must contend with data that are infrequently collected and made available at least a year or more after they have been collected. The result is that nearly all studies of firearms violence are, in a real sense, historical in nature. Lack of timeliness in the availability of data is not a problem for investigating behavioral phenomena that change slowly over time, but the risk of firearms violence in the United States is not necessarily such a phenomenon. For example, rates of firearms violence, especially among youth, rose very rapidly to unprecedented levels during the early 1990s, only to peak and turn downward just as rapidly over the next few years. The popular characterization of those changes as an epidemic was not a misnomer, at least with respect to the speed with which they took place. Needless-to-say, monitoring such rapid and abrupt changes requires timely information. Technical barriers no longer stand in the way of the timely collection, coding, and dissemination of key indicators of firearms violence. Local law enforcement agencies report data on a monthly basis to the FBI on serious assaults, robberies, and homicides by weapon type. Emergency departments and hospitals collect information on violent injuries and death just as frequently. Electronic data entry, coding, and checking have greatly reduced the time required to compile data on firearms violence, and the Internet permits nearly instantaneous dissemination both to special access users and broader audiences. To better monitor trends in firearms and violence, the committee thinks that an important implementation objective of emerging data sets, such as the NIBRS and the NVDRS, should be dissemination of data on firearms violence on a quarterly basis. In addition, monitoring capabilities might be greatly improved if firearm-related behaviors could be added to any proposed revision of the ADAM survey, perhaps on a rotating schedule with the more detailed questions on drug use, and disseminated regularly. CONCLUSION None of the existing data sources, by itself or in combination with others, provides comprehensive, timely, and accurate data needed to answer many important questions pertaining to the role of firearms in violent events. Even some of the most basic descriptive questions cannot be an-

OCR for page 19
Firearms and Violence: A Critical Review swered with existing data. For example, the existing data do not reveal information pertinent to answering the following questions: 12 Where do youth who shoot themselves or others obtain their guns? In what proportion of intimate-partner homicides committed with a gun does the offender also take his or her own life or the lives of the victim’s children or protectors? Did the number of people shot with assault weapons change after the passage of the 1994 ban on assault weapons? What are the most common circumstances leading to unintentional firearm-related deaths? Are particular types or makes and models of firearms overrepresented in unintentional firearm-related deaths? What proportion of suicide or homicide victims were under the care of a mental health professional? What proportion were intoxicated with alcohol or illicit drugs at the time of death? How do these proportions compare with those for suicides committed by other means? There are many other such “unanswerable questions” about firearm-related violence, and even more that can be answered only with great ambiguity. Data for estimating firearm-related mortality lack timeliness and contain only limited information on key circumstantial and weapon-related variables. For firearm-related morbidity data, key circumstantial and weapon-related information is also limited, and no nationally representative data sources monitor firearm-related hospitalizations and disabilities. Data on firearm storage practices, weapon carrying, and gun safety training are not routinely collected. Data for studying noncriminal violence are lacking. Significant gaps exist in the nation’s ability to monitor firearm-related injury and assess firearm-related policies. In the committee’s view, the most important step to improve understanding of firearms and violence is to assemble better data. In the absence of improved data, the substantive questions addressed in this report are not likely to be resolved. Emerging data have the potential to make important advances in understanding firearms and violence. In particular, the National Incident-Based Reporting System and the National Violent Death Reporting System can provide a wealth of information for characterizing violent events. Whether these data will also be effective for evaluating the effects of firearms, injury reduction policies, or other firearm-related policy ques- 12   We thank Catherine Barber and David Hemenway of the Harvard School of Public Health for providing these examples by personal communication.

OCR for page 19
Firearms and Violence: A Critical Review tions is unknown and will almost certainly depend on the particular application. No one system will be effective at answering all questions, but it is important to begin by collecting accurate and reliable data to describe the basic facts about violent injury and death. Thus, we are encouraged by the efforts of the Harvard School of Public Health’s Injury Control Research Center pilot data collection program, as well as the recent seed money devoted to implement such a system at the CDC. We reiterate recommendations made by past National Academies committees (e.g., Institute of Medicine, 1999) and others to support the development and maintenance of the National Violent Death Reporting System and the National Incident-Based Reporting System. We also recognize that these types of data systems have been the subject of great controversy and, in light of well-founded concerns, strongly urge that special care be taken to ensure the credibility of these data. The design and implementation plans for these and other proposed data sets need to explicitly consider whether and how some of the more complex and important policy questions regarding firearms and violence might be resolved. There are many obstacles for developing better data: Methodological issues regarding how different data sets and prior information might be used to credibly answer the complex causal questions of interest. Survey sampling issues, including how to design surveys to effectively obtain information on rare outcomes, geographical aggregation, sample nonrepresentativeness, uncertain accuracy of self- and informant reports, lack of standardization in data elements, and uncertain reliability of cause-of-injury and fatality codes. Legal and political barriers that may make collecting important data difficult if not impossible. For example, the 1986 Firearms Owners Protection Act (the McClure-Volkmer Act) forbids the federal government from establishing any “system of registration of firearms, firearm owners, or firearms transactions or distribution.” All of these issues should be carefully considered before new data collection efforts are proposed or undertaken. The proliferation of firearm data sources, without basic efforts to evaluate their validity and reliability, to determine the possibility for linkages across data sets, and most importantly to assess exactly which questions can be addressed with a particular data set, will not lead to better policy research and violence prevention. Thus, the committee urges that work be started to think carefully about the prospects for developing data to answer specific policy questions of interest. The design for collecting data and the analysis of that data should be selected in light of the particular research question. For example, what data are needed to support research on a causal model of the relation

OCR for page 19
Firearms and Violence: A Critical Review between gun ownership or availability and suicide? Building such a model would presumably involve estimating the probability that an individual commits suicide conditional on gun ownership (or availability in some sense). What data are needed to do this? What data are needed to estimate the effects of policy interventions on the probability of suicide or on the substitution of other means of suicide for guns? What other prior information is relevant? What covariates should be included? Are data on them currently available? Do data on covariates exist in a form that could be combined with gun ownership or availability data? Is it necessary to construct a new data set that includes both ownership or availability data and the covariates? If one is interested in answering the question of whether adolescents with a gun in the home are more likely to successfully commit suicide than adolescents who do not have a gun in their home, then home-level data on gun possession and adolescent suicide are needed rather than aggregate data concerning the numbers of guns in circulation. This type of information could be used to address the basic question of what proportion of the adolescents with a gun in their home eventually commit suicide with a gun. Answering causal questions about firearms and suicide may require additional information. The same questions can be asked about the probability of committing a violent crime with a gun conditional on ownership or availability. Similarly, what data are needed to support improved research on firearms markets and how criminals or suicide victims obtain firearms? How, if at all, would improvements in trace data be used in studies of the effects of policy interventions on firearms markets or any other policy issue? What would the desired improvements contribute to research on policy interventions for reducing firearms violence? How can trace data be used, considering the deficiencies of these data? Ultimately, linking the research and data questions will help define the data that are needed. For example, attempting to answer the seemingly basic research question, “How many times each year do civilians use firearms defensively?” by using samples of data collected from crimes reported to the police is a mismatch between the data source and the research question. These surveys cannot reveal successful forms of resistance that are not reported to the police. This effort to think carefully about the data needed to answer some of the basic research questions should take place in collaboration with survey statisticians, social scientists, public health researchers, and representatives from the Bureau of Justice Statistics, the National Institute of Justice, the Centers for Disease Control and Prevention, the Bureau of Alcohol, Tobacco, and Firearms, and others. The research program should assess data limitations of the existing and proposed data sets, regularly report the results of that research both in the scientific literature and in forums acces-

OCR for page 19
Firearms and Violence: A Critical Review sible to data users, and propose modifications to the data sources when needed. Careful attention should be paid to ownership, and use data. As we demonstrate repeatedly in this report, the lack of credible data on gun ownership and limited understanding of the relationship between ownership and violence are among the most critical data barriers to better understanding firearm-related violence. Thus, the committee recommends a research effort to identify ways in which firearms acquisition, ownership, and use data can be accurately collected with minimal risk to legitimate privacy concerns. A starting point is to assess the potential of ongoing surveys. For example, efforts should be undertaken to assess whether tracing a larger fraction of guns used in crimes, longitudinal data from the Monitoring The Future survey, or enhancement of items pertaining to gun ownership in ongoing national surveys may provide useful research data. To do this, researchers need access to the data. Thus, the committee recommends that appropriate access for research purposes be given to the Monitoring The Future survey, as well as to the data maintained by regulatory and law enforcement agencies, including the trace data maintained by BATF, registration data maintained by the FBI and state agencies, and manufacturing and sales data.13 These data may or may not be useful for understanding firearms markets and the role of firearms in crime and violence. However, without access to these systems, researchers are unable to assess their potential for providing insight into some of the most important firearms policy and other research questions. We realize that many have deeply held concerns about expanding the government’s knowledge of who owns what type of guns and how they are used. We also recognize the argument that some may refuse to supply such information, especially those who are most at risk to use guns illegally. More generally, we recognize that data on firearms ownership and violence have been the subject of great controversy. Nevertheless, there is a long established tradition of making sensitive data available to researchers. In light of these well-founded concerns, the committee strongly recommends that special care be taken to ensure the integrity of the data collection and dissemination process. Concerns over security and privacy must be addressed in the granting of greater access to the existing data and in creating new data on acquisition, ownership, and use. 13   Current law prohibits the FBI from retaining data from background checks. If these data were retained and provided in an individually identifiable form for research purposes, they might provide useful information on firearms markets and measures of known gun owners nationally. To determine the properties of these data, the FBI would need to retain the records and researchers would need access to test their utility for informing policy.