|
||||||||||||||||
|
||||||||||||||||
The following HTML text is provided to enhance online readability. Many aspects of typography translate only awkwardly to HTML. Please use the page image as the authoritative form to ensure accuracy. –3–
|
||||||||||||||||
 |
 |
Page 41 |
 |
 |
 |
|
| ||||||||||||||||||||||||||||||||
|
|
|||||||||||||||||||||||||||||||
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 41
Surveying Victims: Options for Conducting the National Crime Victimization Survey
–3–
Current Demands and Constraints on the National Crime Victimization Survey
THIS CHAPTER BUILDS ON THE DISCUSSION of historical goals in Chapter 2 by examining some contemporary issues and challenges facing the measurement of victimization, in particular the demands and constraints placed on the National Crime Victimization Survey (NCVS). We begin in Section 3–A by discussing survey nonresponse, an emerging challenge facing modern surveys of all types, including federal surveys like the NCVS. Section 3–B discusses basic challenges of self response in measuring victimization, including discussion of crimes that are not well measured in police reports and that are inherently hard to measure: the capability of the NCVS to provide information on these is at once a great strength of the survey and a major, ongoing technical challenge. Section 3–C expands the discussion of analytical flexibility from the previous chapter to include issues of flexibility in measuring new types of victimization as well as changes in basic methodology, including subnational estimation, to meet user needs. We then turn to a basic underlying question—What is the value of measuring victimization?—and consider how the cost of the NCVS compares with various benchmarks in Section 3–E. Section 3–F turns to basic issues related to the coexistence of two related measures of crime in the NCVS and the Uniform Crime Reports (UCR): the general correspondence of the two series over time and resulting questions about the need for two independent
OCR for page 42
Surveying Victims: Options for Conducting the National Crime Victimization Survey
measures. We conclude in Section 3–G, considering both the historical goals of the NCVS (Chapter 2) and the challenges described in this chapter to assess the basic utility of the NCVS.
3–A
CHALLENGES TO SURVEYS OF THE AMERICAN PUBLIC
3–A.1
The Decline in Response Rates
With a response rate of 91 percent among eligible households (84 percent of eligible persons) as of 2005, the NCVS enjoys response and participation rates that are highly desirable relative to other victimization and social surveys. However, the NCVS response rates have declined over the past decade; in 1996, the NCVS household- and person-level response rates were 93 and 91 percent, respectively (Bureau of Justice Statistics, 2006a). Figure 3-1 illustrates the recent growth in the noninterview rate in the NCVS and one component of that rate in particular: refusals by anyone in the contacted household to participate in the survey. The figure presents these noninterview and refusal rates for both initial contacts (interview 1, conducted by personal visit) and for all data collection in the year (including telephone and personal interviews for contacts 2–7 with sample addresses); the initial and aggregate rates generally track each other closely.
The decline in response rates is a situation faced by almost all household surveys in the United States (Groves et al., 2002). For instance, the General Social Survey, a cross-sectional household survey conducted by the National Opinion Research Center at the University of Chicago, has experienced a declining response rate in recent years, from response rates in the high 70s (percentages) and a peak of 82 percent in 1993 to 70–71 percent in 2000–2006.1 Remedies to address declines in response rate continue to be developed. The Substance Abuse and Mental Health Services Administration, contracting with the Research Triangle Institute, implemented a $30 incentive in 2002 in order to induce respondents to return questionnaires for the National Survey of Drug Use and Health due to declining response rates. The highly detailed National Health and Nutrition Examination Survey, conducted by Westat, has experienced a similar reduction in response.
There is little evidence that the loss of response rate over time is primarily a function of what organization conducts the survey: many of the federal surveys collected by the U.S. Census Bureau (including the NCVS) for various sponsors have shown declines as well. Atrostic et al. (2001) describe measures of nonresponse for six federal surveys (including the NCVS) between 1990 and 1999, documenting consistent declines in response; Bates (2006) updates the series through 2005. An example cited in those works
1
See http://www.norc.org/Projects+2/GSS+Facts.htm [8/20/07].
OCR for page 43
Surveying Victims: Options for Conducting the National Crime Victimization Survey
Figure 3-1 Noninterview and refusal rates, National Crime Victimization Survey, 1992–2005
NOTE: Refusal rate is defined as the number of eligible interviewing units not interviewed because occupants refused to participate divided by the total number of eligible interviewing units. Refusals are a component in the noninterview rate, which also includes interviews not completed due to other reasons (e.g., language difficulty or no one at home). Noninterviews are termed “Type A” results. Rates are based on unweighted data.
SOURCES: Data from Bates (2006); definitions from Atrostic et al. (2001).
is the Consumer Expenditure Diary (CED) survey, data from which are an input used to derive the consumer price index. Between 1991 and 2003, the initial nonresponse rate (failure to respond to the first interview, which, like the NCVS, must be done by personal visit) steadily increased from about 15 percent to about 30 percent; in 2005, the CED had an overall nonresponse rate of 31.1 percent (Bureau of Labor Statistics, 2007:87).
A notable exception to the pattern of declining response rates in federal surveys is the American Community Survey (ACS), the replacement for the traditional decennial census long-form sample that asked census respondents for additional social and demographic information. However, the ACS also holds a distinct advantage over other Census Bureau surveys because—inheriting from the decennial census—responses to the ACS are required by law (and respondents are so advised). A test conducted by fielding the ACS with wording on the mailing materials suggesting that response is voluntary (e.g., “Your Response Is Important to Your Community”) rather than mandatory (e.g., “Your Response Is Required by Law”) demonstrated a radically reduced mail response rate: an overall drop of 20.7 percentage points (Griffin et al., 2003, 2004).
OCR for page 44
Surveying Victims: Options for Conducting the National Crime Victimization Survey
The fact that declines in response rates are not isolated to private surveys rather than federal surveys (or vice versa) suggests that large-scale changes in the relationship between survey data collectors (generally) and the U.S. public have occurred in recent years. While there is no convincing empirical evidence to test alternative theories of the causes of the decline, the most popular hypotheses include:
the lack of trust in the institutions requesting the survey participation;
confusion in potential respondents’ minds between marketing approaches and survey participation requests;
loss in discretionary time at home due to increased commute time to work and other out-of-home activities;
increase in the sheer volume of survey participation requests, making participation in any particular survey less novel; and
the increased investment in electronic and other devices to prevent strangers from contacting the public.
There are two principal forms of nonresponse, each of which appears to have separate causes. The first is unit nonresponse: for a household survey like the NCVS, this is nonresponse that arises because the household at a particular address could not be contacted or declined to participate at all. The inability to contact U.S. households is driven both by apparent increases in the out-of-home activities of the public and by changes in how the public views approaches from strangers. There are now more walled subdivisions, locked multiunit structures, and intercom systems that permit residents to control the access of strangers to their housing units. For telephone contact, answering machines and “caller ID” features permit residents to limit telephone contact to those persons they wish to talk to. Hence, populations that invest in these housing features and appliances are disproportionately not contacted. These tend to be urban dwellers, younger, more transient persons, and those who live alone. This broader, structural form of nonresponse is inherent to all surveys. As we discuss in Section 5–B, it is an open question whether the administration of the NCVS by the U.S. Census Bureau is a net positive or negative (or neither) in affecting unit response.
The second type of nonresponse is within-unit nonresponse: given that contact is successfully made at an address, do all the survey-eligible persons at that address cooperate and answer the survey questions? There is evidence that persons who are interested inherently in the announced topic of the survey tend to respond (Groves et al., 2004). There is also evidence that women cooperate more prevalently than men (DeMaio, 1980); that urban dwellers cooperate less frequently than those in rural areas (Groves and Couper, 1998); and that those who live alone and middle-aged persons are less coop-
OCR for page 45
Surveying Victims: Options for Conducting the National Crime Victimization Survey
erative. To the extent that person nonresponse relies on interest in the survey’s topic area, the purview of the NCVS presents complications on both ends of a continuum. For people who have been victimized—particularly highly sensitive crimes like sexual assault—interviewers may face a difficult task in building rapport so that respondents are willing to talk about their experiences. Likewise, interviewers have to be trained to handle the opposite situation: people who have not experienced recent victimization and hence attempt to bow out of the survey because they think it irrelevant.
Survey design features may have some role in affecting response rate, or at least in curbing the loss of response rate. There is evidence that longitudinal surveys of persons—in which multiple contacts are made with the same households and people, forging longer term “relationships” between interviewers and subjects—have experienced lower declines in participation. The NCVS is a longitudinal survey of addresses, not persons, and thus may be affected by turnover of individual persons or families at sampled addresses. However, nonmovers—people who remain at the same address over time—can experience up to seven NCVS requests and thus—conceptually—the NCVS response rate should enjoy some benefit from those repeated contact efforts. As Lepkowski and Couper (2002) note, however, the propensity to respond in later waves of a longitudinal survey is dependent on the enjoyment of the prior wave. If NCVS respondents in one wave find the survey less than pleasant, there may be lower propensity to respond in the next wave.
Telephone surveys appear to suffer more dramatic nonresponse rate increases than face-to-face surveys. This finding primarily comes from evidence from random-digit-dialed surveys (Curtin et al., 2000). The NCVS does use the telephone for waves 2–7 of interviewing, but, given this is common to other longitudinal surveys, it is unlikely that the use of the telephone in NCVS interviewing is, in itself, a principal cause of lower response rates.
It is important to note that nonresponse rates are only proxy indicators of one aspect of the quality of NCVS estimates. The key issue is whether the propensity to be successfully measured among NCVS sample members is correlated with the likelihood of victimization. Tests conducted alongside the British Crime Survey (BCS) and the Scottish Crime Survey (SCS) provide useful evidence along these lines. Lynn (1997) describes a BCS experiment that urged nonrespondents to provide some limited information; people who said that they did not want to be interviewed were pressed to give very short answers about the extent of recent victimizations against them. These capsule assessments were found to be consistent with victimization estimates among people who completed the survey. Similar findings were registered in an SCS test documented by Hope (2005); that test also compared responses gained by face-to-face interviewing compared with telephone response, since a change to telephone collection was being con-
OCR for page 46
Surveying Victims: Options for Conducting the National Crime Victimization Survey
sidered for the 2004 administration of the SCS. In the test, face-to-face and telephone interviews were conducted in parallel; the face-to-face interviews recorded a 67 percent response rate compared with 49 percent by telephone, with the difference attributed to refusals to be interviewed. When victimization estimates from the two modes of administration were compared, the telephone rates were found to be consistently higher, raising the possibility that the telephone administration had the effect of oversampling persons with incidents to report. A follow-up test recontacted some respondents from the first survey and compared victimization estimates for the group of people who responded on the first contact with those who had their refusals “converted” to responses in the second pass. Victimization rates were found to be lower for the “converted” group than the initial respondents, corroborating the hypothesis that refusals are more likely to include nonvictims (people with no incidents to report) rather than victims.
3–A.2
The Rise in Survey Costs
From the fiscal and operational standpoint, the major consequence of increased nonresponse is increased survey costs. These are incurred when the data collection effort seeks to maximize response rates, devoting field resources to repeated attempts to contact households for interviews. Cost inflation would be more modest if only one call were made to each household. In the NCVS and other surveys seeking high-quality estimates, repeated callbacks and efforts at persuasion are introduced on sample cases that have not yet been interviewed. Repeated calls in face-to-face surveys require the interviewer to drive to the sample unit and attempt contact. If no one in the household is at home, another call—often on another trip—is required. If a contact is achieved but the householder is reluctant to participate at the time, another call is made. What results from such a recruitment protocol is that noninterviews require more effort than interviews; the cost of a failure is larger than the cost of a successful interview. As the difficulty of making contact and gaining cooperation increases over time, the costs of the total effort increase if response rates are to be maintained.
In short, attempting to achieve high response rates in a survey of a population presenting growing difficulty in making contact and gaining cooperation will lead to cost inflation.
3–A.3
The Linkage Between Response Rates and Nonresponse Error
It is traditional to attempt to maximize response rates in an effort to reduce nonreponse error. This flows from a simple deterministic view of nonresponse error in a sample mean (like the number of victimizations reported divided by the number of persons) as a function of nonresponse rates and the difference between respondent and nonrespondent means. Increas-
OCR for page 47
Surveying Victims: Options for Conducting the National Crime Victimization Survey
ingly, empirical studies have shown that a stochastic view of nonresponse is more appropriate, viewing each decision to be a respondent as subject to uncertainty. In this view, high correlation between the likelihood of participating and the survey measures produces nonresponse bias in such descriptive statistics. Which NCVS estimates might illustrate such links to response propensities is at this point an unknown question. Some NCVS estimates might be biased from the nonreponse and others might not. New studies are appropriate to gauge what value BJS should place on high NCVS response rates, given both the current design and for future alternative designs.
Given the ubiquity of the nonresponse problem across federal surveys, recent U.S. Office of Management and Budget (2006b) guidelines call for analyses of nonresponse bias when either unit or item nonresponse hits certain levels. The NCVS unit response rate is such that this threshold has not been crossed; still, we know of no effort by BJS or the Census Bureau to mount a full nonresponse bias study for the NCVS.
3–B
CHALLENGES OF SELF-RESPONSE IN MEASURING VICTIMIZATION
3–B.1
Cognitive Challenges: Telescoping and Forgetting
As noted above, the NCVS emerged from the National Crime Survey only after several years of conceptual development and methodological research. The research was path-breaking in that it helped launch what is sometimes called the cognitive aspects of survey measurement (CASM) movement (aided by a Committee on National Statistics workshop, National Research Council, 1984). Much of the labor of that redesign effort (Biderman et al., 1986) targeted improved reporting among respondents to the NCS. Importing key notions from cognitive processing models, it was noted that autobiographical reports were fraught with weaknesses. Memories were viewed as being formed at an “encoding” step, in which sense-based observations were retained, often in a manner that was heavily dependent on the situation during the experience of the events. Not all encoded memories were easily retrieved upon a desire to do so. The studies found that “forgetting” events that did occur was a challenge to the survey. Consistent with long-standing results from cognitive psychology it was found that events that did not induce emotional reactions (“nonsalient”), those that happened frequently, and those that occurred far back in time tended to be underreported. Thus, “forgetting” was a problem for the NCS.
Research on context-dependent recall suggested that individual words and mentions of types of related events were effective “cues” to memory recall. Much of the research on the screener questions, therefore, was attempting to improve the rate of reporting of incidents as a way to attack
OCR for page 48
Surveying Victims: Options for Conducting the National Crime Victimization Survey
the problem of “forgetting.” The “short cue” version of the instrument that resulted from the research attempted to provide a rich set of cues for each victimization type. In this regard, a marriage between the incident report and the screening questions was key. The screener questions were designed to maximize recall, even at the risk of overreporting incidents through duplicate reports about the same event or misdating of an event that occurred outside the reference period. The role of the incident reports was to duplicate those reports.
The finding that forgetting was a function of the salience of the event to the person and the length of time since the event implied that smaller victimizations occurring further back in time were most fraught with reporting errors. The length of the reference period (the time from the start of the eligible time period for events to be in scope to the end of the period) and the length of the recall period (the time between the start of the in-scope period and the day of the interview) were issues that could affect the quality of reports. Longer periods yielded poorer reports (Miller and Groves, 1985; Czaja et al., 1994), generally a mix of forgetting and misdating events. The redesign recommended a 6-month reference period, a recommendation based on the findings of increased measurement error due to forgetting and telescoping in 12-month reference periods.
There was another antidote to misdating or telescoping errors, which was already in place in the NCS—the use of a bounding interview. A bounding interview in the context of the NCS was the first wave interview with each respondent, in which events in the 6-month reference period before the interview were reported. No data from the bounding interview were used in estimation (another recommendation stemming from findings of forward telescoping errors). Instead, the events reported in the bounding interview were made known to the second wave interviewer to verify that a incident reported in that interview was not a duplicate of a report in the first, bounding interview. This was thought to reduce forward telescoping errors in the NCS estimates. Some research in the redesign focused on whether the data from the bounding interview might be integrated through statistical models into the estimates, but that never led to such a recommendation.
The panel notes that the design features of the reference period, the cuing mechanisms of the screener questions, the nature of the incident reports, and the use of the bounding interview technique are mutually connected. It is difficult to evaluate one of these features without simultaneously considering the others.
OCR for page 49
Surveying Victims: Options for Conducting the National Crime Victimization Survey
3–B.2
Measuring Hard-to-Measure Crimes
A key conceptual strength of the NCVS is its ability to elicit information about victimization incidents that are not reported to police. This is particularly the case for such personally sensitive crimes as rape and domestic violence, as well as simple assault and other incidents that are acts of violence but that victims may decline to report (or judge that they are not sufficiently severe to report) to authorities. The 1992 redesign concentrated on improving the screening questionnaire to more effectively and accurately probe respondents to recall and report such incidents, doing so by increasing the density of cues and using multiple frames of reference. Both the pre-and postredesign questionnaires emphasized the need to broach questions in language that is accessible and understandable (and not steeped in legal jargon) in order to boost cooperation and accurate recall.
However, improvements to the screening and cuing procedures leave open the question of whether reporting of hard-to-measure crimes is full and complete and whether other approaches may be preferable. Conceptually, the measurement of sensitive crimes through personal interviewing is a sounder approach than reliance on police reports, but it is important to consider that some crimes that are not reported to police may not be reported to interviewers, either. From the technical perspective, some hard-to-measure crimes—notably domestic violence—present continuing measurement challenges due to their high frequency; determining an accurate count is a formidable difficulty, and detailed information on specific incidents even more so. In this section, we briefly discuss the challenges in getting accurate survey reports in two areas: measurement of rape and domestic violence and description of repeated (series) victimizations.
Rape, Domestic Violence, and Simple Assault
Several researchers have reported that the NCVS yielded lower estimates of the incidence of rape and domestic violence than other surveys. For example, before the 1992 switch to a redesigned instrument, the National Crime Survey produced estimates of domestic violence that were an order of magnitude smaller than those produced by other surveys.2 Similar results obtain
2
Bachman and Taylor (1994) compared the then-available estimates of family violence against women from the NCVS to results from the National Family Violence Survey (NFVS). The cross-sectional NFVS was conducted twice, in 1975 and 1985, by the Family Research Laboratory of the University of New Hampshire and reached a sample of 2,143 and 6,002 households in the two administrations, respectively. The survey suggested that around 160 per 1,000 married couples experienced at least one “violent incident” in 1975 and 1985. By comparison, the NCVS—which did not ask specifically about violence by family members before the redesign—yields an estimate of the annual rate of famly violence against women of just 3.2 per 1,000. However, the two survey estimates are not directly comparable because they frame
OCR for page 50
Surveying Victims: Options for Conducting the National Crime Victimization Survey
Table 3-1 Rape and Assault Rates, National Crime Victimization Survey and National Violence Against Women Survey, 1995
NVAWS
NCVS
Adjusted NCVS
Rape
8.7
1.9
2.6
Intimate Partner Assault
44.2
6.6
26.7
Assault
58.9
25.8
80.4
NOTES: Rates per 1,000 population. “Adjusted NCVS” are estimates calculated by including the reported count of incidents in series victimizations in the estimates.
SOURCE: Rand and Rennison (2005).
for rape: the pre-1992 NCS instrument did not define rape for the respondent and did not directly ask respondents whether they had been victims of attempted or completed rape (Bachman and Taylor, 1994:506). The redesigned NCVS asked respondents more directly about family violence and rape. The new instrument asks specifically about violence or threats perpetrated by “a relative or family member.” The redesigned instrument also asks more directly about “unwanted sexual activity,” including from those who are well known to respondents. A range of probes also distinguishes between verbal and other threats, attempts, and completed rapes.
Post-redesign research comparing results from the new NCVS instrument with previous versions has largely focused on broader crime categories (Kindermann et al., 1997) or differences by analytic groups (Cantor and Lynch, 2005) and not on specific crimes like rape or domestic violence. Still, several studies compared NCVS rates with those generated by other surveys on these categories. Rand and Rennison (2005) contrasted rape and assault rates in the 1995 NCVS with those from the National Violence Against Women Survey (NVAWS), a telephone survey of U.S. adults. They obtained the estimates for annual incidence shown in the NCVS and NVAWS columns of Table 3-1.
Rand and Rennison (2005) suggest several explanations for the discrepancy between the data sources: the NVAWS may elicit more victimizations by asking about rapes and assaults more explicitly: the NVAWS may be more vulnerable to telescoping, in which incidents outside the reference period are included; or the two data sources may diverge because of their measurement of recurring victimization. As a one-time, single-interview survey, the NVAWS had no capacity for bounding responses, “suggesting that [NVAWS]
incidents differently: the NFVS instrument asked specifically about “conflict” among family members rather than more detailed probes about violent incidents.
OCR for page 51
Surveying Victims: Options for Conducting the National Crime Victimization Survey
estimates are likely inflated to some unknown degree” (Rand and Rennison, 2005:274). (Response rates in the NVAWS are also much lower than the NCVS, although it is difficult to know what biases might result.) The NCVS records as a single series victimization a group of six or more victimizations that were similar in nature but difficult for the respondent to recall individually. Rand and Rennison (2005:275) estimate that series victimizations account for about 10 percent of violent incidents against women. BJS publications exclude series victimizations from annual estimates. After adjusting for age and crime types and counting the number of incidents among series victimizations, Rand and Rennison (2005) obtained the estimated rates of annual incidence reported in the “Adjusted NCVS” column of Table 3-1.
Despite the adjustment for series victimization, rates of rape and intimate partner assault are lower in the NCVS than in the NVAWS. However, Rand and Rennison (2005:279) found that the difference is statistically significant only in the case of intimate partner assault. The discrepancy between the data sources is largest for intimate partner violence, suggesting that at least part of the divergence may be due to the classification of intimate partners rather than the measurement of victimization.
Research on the NCVS redesign also suggests that measurement of simple assaults (without a weapon resulting in minor injury) also depends closely on the survey instrument. With a broader screening interview that cued respondents to consider events they might not define as crimes, the redesigned survey recorded roughly twice the number of simple assaults than the old NCVS (Lynch, 2002). While it is difficult to gauge whether there is still underreporting of less serious personal crime in the NCVS, research on the redesign underlines the sensitivity of estimates to the survey instrument.
The possibility that such crime types as sexual victimization and domestic violence may still be underreported in the standard personal interview context—despite improvements in cuing and screening—highlights the importance of researching means for incorporating self-response options in the NCVS. These include such approaches as web administration and turning the computer laptop around for parts of an interview so that respondents read and answer some questions without interaction with the interviewer. We discuss these further in Chapter 4.
Repeated Victimizations
Repeated victimizations may be underestimated in the NCVS because of the way in which series victimizations are handled. As described in Section C–3.d, NCVS interviewers collect specific information (using an Incident Report form) for each victimization incident reported by a respondent except in instances when six or more very similar incidents occurred within the 6-month reference period. In those cases, a single incident form
OCR for page 70
Surveying Victims: Options for Conducting the National Crime Victimization Survey
commonly described as “statutory performance indicators” or SPIs.6 The SPI data are collected and audited annually by the Audit Commission and are also published on government websites. BCS data are formally required for several of these indicators (Home Office, 2007): for example, in the set of SPIs defined for 2006–2007, “the percentage of people who think their local police do a good job” (SPI 2a), “perceptions of anti-social behaviour” (SPI 10b), and the violent crime rate (SPI 5b). Meeting these statutory guidelines requires that the BCS be regularly funded and capable of providing estimates at the local government level.
3–F
ISSUES RELATED TO THE COEXISTENCE OF THE NCVS AND THE UCR
For more than three decades, the nation has had two national indicators of crime: the Uniform Crime Reporting program and the National Crime Victimization Survey. As described in Chapter 2, the two programs overlap in the crimes they cover (and both are used to generate national-level estimates of violent crime) but also differ in some important definitional ways. Despite the definitional differences between the two measures and their complementary nature, a fundamental question still arises in public discussions of crime statistics: Is it necessary to have two data systems for the purpose of estimating and evaluating trends in crime?
One part of that broader underlying question concerns the trends shown by the two series and the degree to which they agree or converge over time: In other words, do police records generally reflect victimization trends, and vice versa? A second part of the bigger question is more philosophical, concerning the necessity of two series: Does there remain the need for a second indicator completely independent of the official police reports?
3–F.1
Do Police Record Reports Reflect Victimization Trends?
The question about the concurrence of the NCVS and UCR trends is made more salient by the fact that there appears to have been a convergence in recent years of UCR- and NCVS-based national estimates of serious violent crime (i.e., rape, robbery, and aggravated assault). In other words, estimates from the NCVS of the number of crimes victims say they have reported to the police and the number that are recorded in the UCR program have grown closer in recent years (see Figure 3-2). On its face, the evidence of the most recent years of the series might suggest a redundancy—that national crime trends may be adequately described by UCR and that the NCVS role as a crime trend monitor may have diminished.
6
Additional information on best value performance indicators is available at http://www.bfpi.gov.uk/pages/faq.asp.
OCR for page 71
Surveying Victims: Options for Conducting the National Crime Victimization Survey
The correspondence or divergence of the UCR and the NCVS takes on extra importance when there appears to be a shift in long-term crime trends. Between 1992 and 2004 the United States experienced a long period of declining crime, by both measures. More recently there have been signs of an upturn in crime in a number of cities, and the national crime rate as measured by the UCR has increased in two consecutive years. It is precisely at turning points such as the current one that rich and timely measures of a variety of aspects of crime are required, to better understand the trajectory that the nation seems to be taking.
Although the UCR and NCVS estimates of the number of serious violent crimes reported to the police have generally become more similar over the past decade, this convergence in levels is not yet fully understood, nor is it clear whether the similarities in estimates will continue in the future (see Lynch and Addington, 2007). This is because the convergence in the estimates does not necessarily reflect a reduction in error by one or both series. Rather, the two series can produce different estimates and varying long- and short-term trends because they measure different aspects of the crime problem using dissimilar procedures. Furthermore, even if the convergence does reflect some reduction in error associated with one or both series, there is little evidence to suggest that this pattern will remain constant in the future.
As evident in Figure 3-2, annual estimates of the total number of serious violent crimes derived from NCVS data have often been higher than the annual counts in the UCR. There are several reasons why this may occur. Most importantly, the NCVS data include crimes that are not reported to the police. Approximately 49 percent of violent victimizations and 36 percent of property victimizations are reported to the police (Hart and Rennison, 2003). In addition, NCVS counts may be higher if police departments do not record all of the incidents that come to their attention or do not forward the reports to the national UCR program.
For some types of crimes in the NCVS and the UCR, it is possible to reconcile apparent discrepancies in annual estimates by adjusting the NCVS counts to include only those incidents said to have been reported to the police. When such adjustments are made, levels and trends in burglary, robbery, and motor vehicle theft appear generally similar in the NCVS and UCR. However, UCR and NCVS levels and trends in serious violent crime, such as aggravated assault and rape, exhibit many discrepancies after these kinds of adjustments are made. These differences in both levels and trends in aggravated assault and rape may result from changes concerning the public’s willingness to report crime to the police, changes in the way police departments record crime, or some other factor. It is clear that the differences in the methodologies of the UCR and NCVS programs must be considered when assessing both levels and trends of crime in the nation. However, the fact that the extent of agreement in current levels of crimes depends on the
OCR for page 72
Surveying Victims: Options for Conducting the National Crime Victimization Survey
Figure 3-2 National Crime Victimization Survey and Uniform Crime Reports estimates of serious violent crimes, 1973–2005
NOTES: Serious violent crimes include rape, robbery, aggravated assault, and homicide; homicide estimates from the UCR are added to the NCVS series. “NCVS Actual” includes crimes not reported to the police as well as those that are (“NCVS Reported”). NCVS estimates before 1993 are based on data year; for 1993 and later years collection year is used (see Table C-2).
SOURCE: National Crime Victimization Survey, Bureau of Justice Statistics and Uniform Crime Reports, Federal Bureau of Investigation. Data from http://www.ojp.gov/bjs/glance/tables/4meastab.htm [11/1/07].
nature of the offense makes it difficult to claim that UCR data alone could be a sufficient indicator for estimating current levels of “crime.”
If the goal is to assess long-term trends in crime, then a high correlation between UCR and NCVS trends would suggest that either data series would serve as a reasonable proxy for some analytical purposes. McDowall and Loftin (2007) assessed the correlations between UCR and NCVS national trends for index crimes for the period 1973–2003. Using a correlation standard of 0.80 or higher to indicate sufficient agreement in trends, they found that only two crimes came close to or exceeded this standard: robbery (r = 0.76) and burglary (r = 0.93). The next highest correlation was found for motor vehicle theft (r = 0.67). However, the remaining crime types exhibited much lower or even negative correlations. For larceny theft, the correlation was weak (r = 0.20), and for rape and assault, the correlations were negative (r = −0.16 and r = −0.21, respectively) (McDowall and Loftin, 2007:101). Like current level estimates, the trend correlations varied according to crime type. Analysts studying robbery and burglary can expect
OCR for page 73
Surveying Victims: Options for Conducting the National Crime Victimization Survey
generally similar results using either UCR or NCVS trend data for this time period. For other types of crime, however, this will not be the case.
One of the main hypotheses about why assault trends might differ in the UCR and NCVS series is heightened police productivity resulting in a growth of police estimates of assaults (O’Brien, 1996). Rosenfeld (2007) hypothesized that if the divergence in the two series was the result of changes in the way police were handling less serious assaults, then one should expect to see similar trends in the UCR and NCVS gun assault rates, but divergent trends in the nongun assault rates because the perceived seriousness of gun assaults and the ways in which such crimes are handled by the police are much less susceptible to change over time. In addition, if the hypothesis is correct, the ratio of nongun to gun assaults should have increased more in the UCR than in the NCVS. Using UCR and NCVS aggravated assault data for the period 1980–2001, Rosenfeld found that the correlation between the UCR and NCVS estimates of gun assaults was 0.74, while the correlation for nongun assaults was 0.16 (not significantly different from 0). In addition, the ratio of nongun to gun assault rates in the UCR grew, while the same ratio using the NCVS data did not. Thus, the two data series provided similar information about trends in gun-related aggravated assault, but they differed in their patterns for aggravated assaults without guns. The form of the UCR and NCVS nongun assault trends also suggested that changes in police recording and categorization of such incidents stabilized during the 1990s as the two series began to exhibit more similar trends.
In their comprehensive examination of UCR and NCVS trends, McDowall and Loftin (2007) found that the two series for each of the index crimes began tracking each other more closely in the 1990s. McDowall and Loftin argue that this would suggest a structural break in the UCR data series, which might indicate that the estimates from the two series will continue to follow each other more closely in the future. The reason that modifications in the UCR are thought to be responsible for the increased correspondence is that there have been changes in the recordkeeping systems of police departments, while the NCVS methodology remained relatively more stable over the same period (McDowall and Loftin, 2007:111). The recordkeeping capacities of police departments improved as a result of technological innovations, as well as increased numbers of personnel involved in this task. However, the authors note that the agreement in the series is fairly recent and based on a limited number of data points: thus it is premature to conclude that it will continue in the future (McDowall and Loftin, 2007:114).
Whether it is reasonable for state and local governments to believe that their local police data accurately capture trends in crime is more contentious. When state and local governments are interested in assessing trends in crime in their own areas, they typically must rely solely on police data because victimization survey data are rarely available for small geographical areas. The
OCR for page 74
Surveying Victims: Options for Conducting the National Crime Victimization Survey
collection of reliable crime survey data is costly, and most state and local governments have not had the resources to conduct their own victimization surveys, especially on an annual basis. As a result, many wonder whether conclusions about the recent convergence in national police and victim survey data apply to their local areas.
The limited amount of research that has addressed the comparability of UCR and NCVS trends in local areas has used data from special tabulations of NCVS data. One such special subset allows researchers to produce victimization estimates for the 40 largest metropolitan core-county areas in the country (Bureau of Justice Statistics, 2007b). NCVS estimates can be generated from this newly available file for comparisons to UCR data for those same places and years (Lauritsen and Schaum, 2005). Existing research using these data has found that the correlations in the trends for robbery and aggravated assault vary considerably across metropolitan areas, although much less so for robbery than for aggravated assault (Lauritsen, 2006a). For robbery, the average UCR-NCVS trend correlation across the 40 largest metropolitan areas for 1979–2004 is 0.59 (range = 0.02 to 0.95), while the average correlation for aggravated assault is 0.16 (range = −0.75 to 0.76). In addition, the trend correlations tend to be higher in the more populated metropolitan areas, which may reflect earlier adoption of crime records management technology by the larger police departments. Lauritsen concluded that there is a good deal of variation at the level of the metropolitan statistical area in the correlations between the two sets of trends that is masked at the national level, and, as a result, it would be unwise for local areas to assume that their local UCR data provide good indicators of nonlethal violence trends. In addition, Wiersema (1999) developed an area-identified NCVS data set from public-use data files, coded with geographic identifiers down to the census tract level, that was briefly available through Census Bureau research data centers. These area-identified data have been used to support some subnational analyses; see, e.g., Baumer (2002); Baumer et al. (2003); Lauritsen (2001). However, the data have been taken out of circulation; we discuss this further in Section 5–A.
In sum, although much less is known about how UCR data trends compare with NCVS trends for state and local areas, it appears that at the national level, index crime trends have become more similar in recent years. However, just as structural breaks in UCR data collection appear to be responsible for the increasingly similar trends in the 1990s, changes may occur in the future. These alterations can result from a variety of factors, ranging from resource shortages in police departments to local political pressures regarding crime rates. Changes in the NCVS estimates can occur as well, as a result of declines in participation rates, sample coverage problems, limited resources, or other factors. Without both sources of crime information,
OCR for page 75
Surveying Victims: Options for Conducting the National Crime Victimization Survey
it extremely difficult to fully understand the meaning of future changes in crime rates from either data series.
McDowall and Loftin (2007) and others (e.g., Lynch and Addington, 2007) argue that too much emphasis should not be placed on the general issue of convergence in national crime trends. Rather, the complementarities of the two data systems should be emphasized and the selection of one series over another should depend on the research question at hand. For some types of analyses, researchers can use both data sources to assess and understand the strengths and limitations of findings. However, the NCVS is currently the only available source of national data for describing and understanding trends in certain types of crime. These crime types include those that are defined according to specific conditions of the incident (such as intimate partner violence); victim characteristics (such as violence against women or crimes against the elderly); and crimes that are believed to be severely underrepresented in police data for assorted reasons (such as hate crimes, sexual violence, and identity theft).
3–F.2
Independence of the NCVS and the UCR
The origin of the NCVS as an independent estimate compared with the UCR was a development from the social and political climate of the 1960s. Cantor and Lynch (2000:97) note that “the confluence of several forces”—including a general mistrust of institutions—“made the 1960s an auspicious time for the development of victim surveys.” Specifically, “reforms of several of the Nation’s metropolitan police departments were accompanied by exposés of the previous practice of killing crime on the books”—that is, suppressing levels of reported crime. Against this backdrop, “victim surveys brought the ‘patina of science’ ” and an air of accuracy and impartiality to crime statistics; “there was greater trust that the resulting [NCVS] crime estimates were not purposely manipulated” because “the Census Bureau and survey research agencies were not interested parties with respect to the crime problem” (Cantor and Lynch, 2000:88–89).
Beyond the question of whether the UCR and the NCVS respond to the same underlying phenomena, the question can be raised about whether the need for a independent, national-level, and victimization-based measure of the traditional index crimes persists. If it were concluded that there was no need for such an independent national-level measure, then a different class of NCVS design options becomes feasible, if not preferable: for instance, crime-type coverage between the NCVS and the UCR would be reallocated so that the UCR becomes the sole source of national indicators of some crimes, while the NCVS is focused more on hard-to-measure or newly emerging crime types.
We think that the NCVS has strong policy relevance as a national-level
OCR for page 76
Surveying Victims: Options for Conducting the National Crime Victimization Survey
measure of crime independent of the UCR and should continue to function as such, for several key reasons. These include:
The NCVS as an objective measure: Given that police are not an uninterested party in crime rates, there is always an inherent possibility of minimizing or reducing reported crime. Put more colloquially, having the police as the reporter of crime suggests the possibility of “cooking the books,” lowering reported counts or possibly declining to report altogether. The UCR can do some imputation (and does), but an independent measure as a counter to this possibility still has merit. (Note, though, that this argument is weakened in the absence of local-area NCVS-based estimates, which would be the best check on individual department reporting.)
Voluntary UCR reporting leads to coverage gaps: The UCR program relies on the voluntary cooperation of more than 17,000 seprate law enforcement agencies. Complete nonresponse to the UCR program, for individual years or for long stretches of time, occurs and is sometimes pervasive for some states and large localities (see, e.g., discussions of UCR coverage in Maltz, 1999, 2007). Again, imputation helps to bridge gaps in national-level UCR estimates, but for representativeness in coverage, the conceptual advantage still goes to a nationally representative sample like that employed in the NCVS.
Independent measure as “calibration” device: It is useful to have two related-but-not-identical measures in simultaneous operation simply because they may not always agree. The United States has two independent measures of jobs and employment (in the Current Employment Statistics and the Current Population Survey); it has multiple measures of health insurance prevalence and of disability. The differences among the indicators enhance general understanding of the dynamics of the phenomena under study. Divergent or discrepant findings resulting from the two series may signal some structural problem with either of the individual measures and draw attention to potential problems in methodology. An original implicit notion in the creation of the NCVS—grounded in distrust of police reporting—may have been the use of the NCVS as a check on the UCR; however, it is equally valid to say that the two series can serve as an operational and conceptual check on each other.
At the same time, in speaking of “calibration,” it is important to bear in mind that one data source is not always unequivocally right and the other wrong; both the NCVS and UCR are subject to measurement flaws. UCR measurement can suffer from lack of reporting by law enforcement agencies (discussed below) and underreporting due to victims’ hesitance to come for-
OCR for page 77
Surveying Victims: Options for Conducting the National Crime Victimization Survey
ward to the police. But, likewise, crimes may not be reported to interviewers either, as discussed in Section 3–B.2; other examples of crimes missed in NCVS exist, including the finding by Cook (1985) that National Crime Survey estimates captured only about one-third of gun assaults resulting in gunshot injury that were apparent in emergency room data. Such underestimates might arise from disproportionate gunshot prevalence among those not part of the household population or not listed as household members (and thus not sampled), those who were never contacted or who refused to participate, as well as those respondents who failed to report incidents to the NCVS interviewer.
Having an independent measure is important as long as there remains reason to believe that not all crime is reported to police and that not all crimes known to the police are completely tallied in the UCR. That said, the utility of an UCR-independent measure of crime should not prevent consideration of design options that reduce lockstep similarity between the UCR and the NCVS (e.g., measuring exactly the same set of “index crimes” except for homicide).
For several years, the National Incident-Based Reporting System (NIBRS) has been developed as a next-generation version (and replacement for) the UCR. The presence of a strong and complete NIBRS program might further blur the line between the UCR and the NCVS as separate indicators of crime. However, NIBRS development has been slow, and its coverage (i.e., cooperation by agencies in providing more detailed incident reporting) is still quite small. As of September 2007—about 15 years after development of initial NIBRS protocols—only about 26 percent of law enforcement agencies that contribute data to UCR were submitting NIBRS-compliant information; see Section D–2 for additional detail.
3–G
ASSESSMENT
As is true of many multipurpose social indicators, the basic utility of the NCVS to the American public is difficult to characterize in tangible terms. Because it does not currently provide estimates at small areas of geography, its role in allocating federal or state funds for criminal justice improvements is limited, and it does not readily lend itself to focusing specific police interventions in specific neighborhoods. However, through its focus on providing detail on all types of crime and violence—reported to the police or not—and its rigorous design based on a representative sample with uniform national coverage, the NCVS has undeniable importance as a critical statistical indicator. For an informed assessment of the state of public welfare, federal statistical agencies like BJS have a core mandate “to be a credible source of relevant, accurate, and timely statistics” (National Research Coun-
OCR for page 78
Surveying Victims: Options for Conducting the National Crime Victimization Survey
cil, 2004:3). The NCVS provides information on the extent, consequences, and causes of violent behavior that are not available at the same level of comprehensiveness and quality from any other source. Accordingly, direct reports from BJS on NCVS trends are frequently sought for information and for assessment of new policy, and NCVS data play an important role in national appraisals of child welfare (Federal Interagency Forum on Child and Family Statistics, 2007) and public health (U.S. Department of Health and Human Services, 2000).
In our assessment, the need for a victimization-based measure of crime—another indicator, separate from the official police reports of the UCR—is as significant today as it was when the NCVS was first conceptualized. This is the case not out of any inherent distrust of official reports to police or demonstrated inaccuracy therein, but rather for the reason suggested most concisely by the President’s Commission on Law Enforcement and Administration of Justice (1967:18): “No one way of describing crime describes it well enough.” The importance of the crime problem in the United States demands ongoing monitoring from multiple perspectives. In this monitoring, the NCVS is a vital complement to the police reports of the UCR, providing valuable information on the context and causes of victimization in ways that summary counts can never do by themselves (and in which even a fully implemented NIBRS would still be lacking).
However, in its size and available resources, the current NCVS is not capable of matching the original vision of the survey. As the costs of collecting information from the U.S. public have risen, the NCVS budget has not kept pace. Budget reductions have led to cutbacks in NCVS activities, most often through cuts in the total sample size. As it is currently configured, the NCVS does not meet the goal of being able to accurately measure year-to-year change in crime trends. That is, the standard errors of change estimates are too large to detect changes of importance to the country; BJS has had to use averages from 2-year groups of data in order to make statements about change (see Catalano, 2006), even though inferences from these rolling averages are not as intuitive to users and members of the public as direct estimates of change.7 To state this as a finding:
Finding 3.1: As currently configured and funded, the NCVS is not achieving and cannot achieve BJS’s legislatively mandated goal to “collect and analyze data that will serve as a continuous and comparable national social indication of the prevalence, in-
7
For additional information on the use and interpretation of rolling-average estimates from federal survey data, see National Research Council (2007); the American Community Survey will use 3-year and 5-year averages in order to produce estimates for small areas and populations.
OCR for page 79
Surveying Victims: Options for Conducting the National Crime Victimization Survey
cidence, rates, extent, distribution, and attributes of crime …” (42 U.S.C. 3732(c)(3)).
Clearly, given the panel’s charge to consider options for the conduct of the NCVS, one possibility is not to conduct the NCVS at all; we reject that option. To take that option violates the legislative responsibilities of the Bureau of Justice Statistics. Furthermore, the panel thinks that BJS is the appropriate locus of responsibility for victimization measurement. As a federal statistical agency, it alone has the mandate for independent, objective, statistical measurement, with the transparency that can establish public trust in the information (see National Research Council, 2004).
Thus, although there should be no need for the panel to do so, we feel obliged to state a recommendation that is already explicit in the mission of BJS:
Recommendation 3.1: BJS must ensure that the nation has quality annual estimates of levels and changes in criminal victimization.
However, the natural corollary is that the resources necessary to adequately achieve this mission must be forthcoming:
Recommendation 3.2: Congress and the administration should ensure that BJS has a budget that is adequate to field a survey that satisfies the goal in Recommendation 3.1.
NCVS’ unique substantive niche is providing information on crimes that are particularly likely to go unreported to the police, for whatever reason—whether fear or stigma, individual distrust of authority, or the perception that a violent act or threat is not significant enough a “crime” to report.
Recommendation 3.3: BJS should continue to use the NCVS to assess crimes that are difficult to measure and poorly reported to police. Special studies should be conducted periodically in the context of the NCVS program to provide more accurate measurement of such events.
OCR for page 80
Surveying Victims: Options for Conducting the National Crime Victimization Survey
This page intentionally left blank.