Surveying Victims Options for Conducting the National Crime Victimization Survey (2008) / Chapter Skim
Currently Skimming:
4 Matching Design Features to Desired Goals
4 Matching Design Features to Desired Goals
Pages 81-116
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 81... ...
, we find seven fundamental goals to have particular salience and use them as the basis for evaluating various NCVS design options. As we elaborate below, we suggest this as a set of desirable goals; they are certainly not the only possible goals, and others may place different weights on particular goal statements.
|
From page 82... ...
, outlining a set of survey design packages, some representing relatively minor changes to the current design and others overhauling the basic approach to measuring victimization. Section 4–C describes the trade-offs in cost, error, and utility associated with various design features and presents our general assessments.
|
From page 83... ...
Eliminating or vastly reducing incident report details for less serious crimes may result in greater savings, espe cially if such experiences account for some of the field costs associated with future efforts to improve retention and participation. In general, condensing the content of the incident report portion of the questionnaire was seen as a better alternative because "it would reduce respondent burden and perhaps minimize errors such as survey fatigue and future participation." However, we again note that large portions of the NCVS costs arise not in interviewing but in contacting and gaining the cooperation of sample households.
|
From page 84... ...
time in sample bias. Including series incidents in the estimates Reductions in standard errors due to Some reduction in standard errors due to High-end users have always had access to higher victimization rates would permit the increase in the rate of victimization, series incidents.
|
From page 85... ...
Use police statistics to improve stratifica- To the extent that stratification reduced If stratification was used without sample This revision stops short of a wide-scale MATCHING DESIGN FEATURES TO DESIRED GOALS tion in the sample standard errors, commensurate sample cuts, then there would be gains in preci- revision of survey content but is not cuts could reduce costs. sion with changes in sample stratification.
|
From page 86... ...
The decision to include unbounded, first interviews in NCVS estimates was made as our panel was being established and assembled, and so we do not think it proper to second-guess it; we understand the fiscal constraints under which the decision was made. However, it serves as an example of a seemingly short-term fix with major ramifications, and it would have benefited from further study prior to implementation.1 4–B ALTERNATIVES TO THE CURRENT DESIGN IN THE LONG TERM Tables 4-2 and 4-3 present a set of design packages that can be compared with the current NCVS design.
|
From page 87... ...
Instead, they are meant to suggest a range of choices and broad visions for the survey. It should also be noted that we do not have the capacity to provide an estimated price for all of these various options; while some can clearly be assumed to be less costly than the current NCVS design, others may not.
|
From page 88... ...
However, the advantage of rotating panels on response rates is likely to be a function of how frequently sample households are visited; households may drop off in their cooperation with a survey the longer they remain in the sample. By comparison, repeated cross-sectional surveys may have a lower response rate but can be more flexible in terms of content and design modifications.
|
From page 89... ...
. By comparison, most other ongoing victimization surveys (e.g., the British Crime Survey and the International Crime Victimization Survey; see Appendix E)
|
From page 90... ...
Subsampling Frequent Events In the current NCVS design, respondents are asked to complete an incident report for every victimization incident counted during the screening portion of the questionnaire. A possible approach to reduce overall interviewing time (and hence lower costs somewhat)
|
From page 91... ...
The implementation of event history methods in the NCVS would require major restructuring in the existing screener portion of the survey. However, they could become particularly important if respondents were asked to recall victimization incidents over a longer time frame, as would be the case with a switch from a 6-month to a 12-month reference period.
|
From page 92... ...
The British Crime Survey type model (3) is built around a coresupplement design but would represent a major shift from the current NCVS as it would abandon the rotating panel design for an annual cross-section sample.
|
From page 93... ...
involves at least two major compromises: it would abandon both the rotating panel design and the potential face-to-face interviewing capability of the current NCVS, focusing instead on a cross-sectional telephoneonly survey.3 Replication of the ICVS one-stage screening process would also probably substantially reduce the quality of reporting in the survey. However, the instrument for such a design could fairly readily accommodate topical supplements.
|
From page 94... ...
BCS Type -- Cross-sectional design Single Household multistage cluster Repeated 12 months CAPI Yes Yes Yes Yes with a 12-month reference period, event cross history methods, routine supplements section Local-Area Boost Models -- Cross-sectional design with a 12-month reference period, event history methods, and routine sup plements, plus rolling sample to support subnational estimates (4) State Boost -- Periodic increase of Single Household multistage cluster Cross- 12 months CAPI Yes No Yes Yes sample to tiers of states sample clustered by states with section states rotating in and out (5)
|
From page 95... ...
Ide ally, there would be a continuing, small-level national sample for calibration; a pure version of this MATCHING DESIGN FEATURES TO DESIRED GOALS model would make the "NCVS" strictly the compilation of the state-level surveys (10) Periodic Survey without incident Single Phone (likely RDD)
|
From page 96... ...
between the Census Bureau and state demographic units, which has become an important partnership agreement for improvements in various population estimate and general census processes. Most significantly, though, we consider this model because of BJS's unique placement relative to other federal statistical agencies: inside the Office of Justice Programs, whose core mission is providing assistance to state and local law enforcement agencies.
|
From page 97... ...
This option is the starkest contrast with the current NCVS design, as it would involve giving up the goal of producing annual rates of criminal victimization and the collection of detailed incident information. However, it is also -- almost certainly -- the lowest cost alternative considered in this table.
|
From page 98... ...
. Most state health departments let contracts to commercial or university survey research units to conduct BRFSS interviews; in 2006, only 14 state health departments conducted the interviews in-house.
|
From page 99... ...
can be made within the framework of a rotating panel design as is currently used in the NCVS. Although the nature of the current NCVS as a longitudinal sample of addresses is one of its key attributes and has provided the basis for bounding interviews, that longitudinal structure is an underutilized feature.
|
From page 100... ...
• Other options: Please specify: Area for free response The Texas Crime Poll was originally conducted as a systematic random sample from the frame of persons with valid Texas driver's licenses, collected by mail; in 1977, the poll included 642 respondents (a 67 percent response rate)
|
From page 101... ...
A careful examination is required and consistent with the principles of a federal statistical agency. 4–C ASSESSMENTS OF DESIGN FEATURES AND PACKAGES Table 4-3 summarizes our basic assessment of how well the design packages described by Table 4-2 satisfy seven desirable goals, as articulated at the outset of this chapter.
|
From page 102... ...
of and Crime States and Crime Measure Products Police Etiology Problems Localities Rates Timely Crimes Pros Cons Current NCVS • Annual estimates • • × × • × • • High-quality national estimates of • Limited detail on crime events rates for "street crimes" • Inflexible information content of level and • Long, stable time series • Not particularly timely change • DoJ controls survey content • Scope of basic BJS reports restricted • Special reports • Omnibus vehicle reserves resources to rate estimation, though data have on social context for victimization data been used extensively in other ways of crime and on • Limited coordination with other • Expensive specific types of agencies or entities required • Little information of direct use to crime and states and localities subpopulations of victims Core-Supplement (simple) • • • × • × • • DoJ controls survey content • Development and fielding of • Omnibus vehicle reserves resources supplements will add to survey cost • Annual estimates for victimization data • Some increases in coordination with of level and • High-quality national estimates of other agencies change rates for street crimes • Additional staffing for BJS • Regular series of • Long, stable time series for core • Little information of direct use to topical reports items states and localities based on • More information on context, scheduled etiology, and consequences supplements • More information on emerging crime problems • More information on issues beyond rates • Reduced cost from trimming core SURVEYING VICTIMS
|
From page 103... ...
Table 4-3 (continued) BCS Type • • • × • × • • Save money with 12-month reference • Reduction in quality of national period estimates • Greater flexibility in information • Lower response rates in cross-section content • Increase in recall bias with longer • DoJ controls survey content reference period • Omnibus vehicle reserves resources • Telescoping without bounding for victimization data interview • Limited coordination with other • Complicates introduction of agencies or entities required self-administered technology • Loss of information on all household members, due to one respondent per household Local-Area Boost • • • • • × • • High-quality national estimates of • Lower response rates in cross-section rates for street crimes • Greater recall loss from longer • More information on context, reference period etiology, and consequences, but not • Modest increased telescoping with as much as core-supplement event history • Development and fielding of MATCHING DESIGN FEATURES TO DESIRED GOALS • More information on emerging crime problems supplements will add survey cost • More information on issues beyond • Additional staffing for BJS rates • Greater ability to provide state and local estimates and other subnational data • DoJ controls survey content • Omnibus vehicle reserves resources for victimization data • Limited coordination with other agencies or entities required 103
|
From page 104... ...
of and Crime States and Crime Measure Products Police Etiology Problems Localities Rates Timely Crimes Pros Cons Boost with Supple ments • • • • • × • • High-quality national estimates for • Some reductions in quality of rates of street crimes information • More information on context, • Lower response rates in cross-section etiology, and consequences but not as • Greater recall loss from longer much as core-supplement reference period • More information on emerging • Modest increase in telescoping with crime problems cross-section and event history • More information on issues beyond • Development and fielding of rates supplements will add survey cost • Greater ability to provide state and • Additional staffing for BJS local estimates and other subnational data • DoJ controls survey content • Omnibus vehicle reserves resources for victimization data • Limited coordination with other agencies or entities required ICVS Type × × • × • × × • Reduced costs of sampling and • Large reductions in the quality of the administration with all-telephone data RDD design • Much lower response rates • Changing information content is • Increased recall bias due to longer simple reference period • DoJ controls survey content • Underreporting due to abandonment • Limited coordination with other of screener/incident logic agencies or entities required • Increased telescoping • Increase in out-of-scope events due to lack of incident form • Loss of all geographical information • Loss of detail incident information • Reduced resource set devoted to victim statistics SURVEYING VICTIMS
|
From page 105... ...
Table 4-3 (continued) Partnership Model × • • × • × • • Better reporting on hard-to-find • Large sample sizes would be needed crimes in specialized surveys in narrower-scope component • Increases information on context, surveys, due to relative rareness of etiology, and consequences of these some crime types crimes • Loss of omnibus vehicle on crime • Divides labor for crime statistics may lead to reduction in resources between UCR, survey and health dedicated to victimization statistics • Increased coordination required to field surveys and to maintain coverage of important crimes • Loss of control by DoJ • Time series disrupted • Large reductions in information on context, etiology, and consequences of street crime Surveillance Model × × • • • × × • Ability to provide extensive • Loss of omnibus vehicle on crime information on victimization in states may lead to reduction in resources and localities dedicated to victimization • Consistent with law enforcement • Increased coordination with states assistance focus of OJP and localities required to field surveys and to maintain coverage of important crimes • Loss of control by DoJ • Time series disrupted MATCHING DESIGN FEATURES TO DESIRED GOALS • Large reductions in information on context, etiology, and consequences of street crime • Funding uncertainty due to reliance on BJA funding Crime Poll × × • × • • × • Reduced cost from abandoning goal • No estimates of victimization of estimating victimization rates • Break in time series of victimization • Smaller samples rates • Attitudes more prevalent than • No information on context, etiology, victimization and consequences of victimization • More extensive information on • Time series disrupted attitudes toward justice issues • Much lower response rates NOTES: • indicates that the panel believes that the goal is well served by the design package; • that the goal is adequately served; × that the goal is poorly served.
|
From page 106... ...
The NCVS design is a model that has been adopted by international victimization surveys as well as subnational surveys in the United States, and it is a good and useful exemplar. The principal fault of the current NCVS design is not a design flaw or methodological deficiency, or even that the design inherently costs too much to sustain, but rather -- simply -- that it costs more than has been tenable under current federal budgetary priorities.
|
From page 107... ...
Barring those very optimistic outcomes, we also recognize that -- in terms of its sample size -- the current NCVS is at a critical point: its sample size has slipped sufficiently that terminating the survey would effectively be preferable to sustaining additional across-the-board cuts in sample size. 4–C.1 Length of Reference Period In light of these arguments, we suggest switching to a 12-month reference period (thus achieving savings by reducing the number of contacts with sample households)
|
From page 108... ...
suggest about a 30 percent general reduction in reporting by doubling the length of the reference period to 12 months. Consequently, it is important that a move toward a 12-month reference period be paired with research on developing event history techniques and other methods for improving accurate recall over a lengthy time window.
|
From page 109... ...
The BCS-type model (3) described in our tables would combine a switch to a 12-month reference period with another major change: converting from a rotating panel, multiple-interview-per-household design to a crosssectional single-interview-per-household structure.
|
From page 110... ...
It is meant to prevent the overall content of the survey from becoming stale; in some cases, a topical supplement could serve as a "methods panel" for testing new questions that might, in time, be added to the core NCVS content. We think that the implementation of such NCVS topic supplements as the Police-Public Contact Survey and the School Crime Survey have broadened the scope of the victimization survey and that further branching out into topic supplements will serve to firm up the constituencies for the NCVS and other BJS products.
|
From page 111... ...
As noted in Section 4–A, reducing the screener portion of the interview could slightly decrease interview length and yield very small cost savings, but the quality of resulting data would suffer. That said, we have not comprehensively reviewed the incident form to suggest items to cut, either.
|
From page 112... ...
software and Internet mapping tools have become more widespread, markets have developed for richer spatial data on social variables like crime and victimization. Recommendation 4.5: BJS should investigate the use of model ing NCVS data to construct and disseminate subnational esti mates of major crime and victimization rates.
|
From page 113... ...
In addition to small-domain modeling using NCVS data, it may also be useful to explore ways to strengthen victimization surveys conducted by states and localities. The surveillance model (9)
|
From page 114... ...
Reviewing the optimality of the numbers of primary sampling units versus number of housing units within primary sampling unit may also be beneficial. Efficiency gains can also occur if information can be exploited to predict which housing units tend to have higher victimization rates; in that case, one can then sample the blocks containing such housing units at higher rates.
|
From page 115... ...
changing the stratification of housing units; (iv) selecting housing units with unequal probabilities, so that probabilities are higher where victimization rates are higher; and (v)
|
From page 116... ...
Recommendation 4.9: The falling response rates of NCVS are likely to continue, with attendant increasing field costs to avoid their decline. BJS should sponsor nonresponse bias studies, fol lowing current OMB guidelines, to guide trade-off decisions among costs, response rates, and nonresponse error.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.