3
Comparison of Self-Report and Official Data for Measuring Crime

Terence P. Thornberry and Marvin D. Krohn

There are three basic ways to measure criminal behavior on a large scale. The oldest method is to rely on official data collected by criminal justice agencies, such as data on arrests or convictions. The other two rely on social surveys. In one case, individuals are asked if they have been victims of crime; in the other, they are asked to self-report their own criminal activity. This paper reviews the history of the third method—self-report surveys—assesses its validity and reliability, and compares results based on this approach to those based on official data. The role of the self-report method in the longitudinal study of criminal careers is also examined.

HISTORICAL OVERVIEW

The development and widespread use of the self-report method of collecting data on delinquent and criminal behavior together were one of the most important innovations in criminology research in the twentieth century. This method of data collection is used extensively both in the United States and abroad (Klein, 1989). Because of its common use, we often lose sight of the important impact that self-report studies have had on the study of the distribution and patterns of crime and delinquency, the etiological

This study was supported by the National Consortium on Violence Research.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 43
Measurement Problems in Criminal Justice Research: Workshop Summary 3 Comparison of Self-Report and Official Data for Measuring Crime Terence P. Thornberry and Marvin D. Krohn There are three basic ways to measure criminal behavior on a large scale. The oldest method is to rely on official data collected by criminal justice agencies, such as data on arrests or convictions. The other two rely on social surveys. In one case, individuals are asked if they have been victims of crime; in the other, they are asked to self-report their own criminal activity. This paper reviews the history of the third method—self-report surveys—assesses its validity and reliability, and compares results based on this approach to those based on official data. The role of the self-report method in the longitudinal study of criminal careers is also examined. HISTORICAL OVERVIEW The development and widespread use of the self-report method of collecting data on delinquent and criminal behavior together were one of the most important innovations in criminology research in the twentieth century. This method of data collection is used extensively both in the United States and abroad (Klein, 1989). Because of its common use, we often lose sight of the important impact that self-report studies have had on the study of the distribution and patterns of crime and delinquency, the etiological This study was supported by the National Consortium on Violence Research.

OCR for page 43
Measurement Problems in Criminal Justice Research: Workshop Summary study of criminality, and the study of the juvenile justice and criminal justice systems. Sellin made the simple but critically important observation that “the value of a crime rate for index purposes decreases as the distance from the crime itself in terms of procedure increases” (1931:337). Thus, prison data are less useful than court or police data as a measure of actual delinquent or criminal behavior. Moreover, the reactions of the juvenile and criminal justice systems often rely on information from victims or witnesses of crime. It does not take an expert on crime to recognize that a substantial amount of crime is not reported and, if reported, is not officially recorded. Thus, reliance on official sources introduces a number of layers of potential bias between the actual behavior and the data. Yet, through the first half of the twentieth century, our understanding of the behavior of criminals and those who reacted to crime was based almost entirely on official data. While researchers were aware of many of these limitations, the dilemma they faced was how to obtain valid information on crime that was closer to the source of the behavior. Observing the behavior taking place would be one method of doing so, but given the illegal nature of the behavior and the potential consequences if caught committing the behavior, participants in crime are reluctant to have their behavior observed. Even when observational studies have been conducted—for example, gang studies (e.g., Thrasher, 1927)—researchers could observe only a very small portion of the crimes that took place. Hence, observational studies had limited utility in describing the distribution and patterns of criminal behavior. If one could not observe the behavior taking place, self-reports of delinquent and criminal behavior would be the data source nearest to the actual behavior. There was great skepticism, however, about whether respondents would be willing to tell researchers about their participation in illegal behaviors. Early studies (Porterfield, 1943; Wallerstein and Wylie, 1947) found that not only were respondents willing to self-report their delinquency and criminal behavior, they did so in surprising numbers. Since those very early studies, the self-report methodology has become much more sophisticated in design, making it more reliable and valid and extending its applicability to myriad issues. Much work has been done to improve the reliability and validity of self-reports, including the introduction of specialized techniques intended to enhance the quality of self-report data. These developments have made self-report studies an integral part of the way crime and delinquency are studied. Although the self-report method began with the contributions of

OCR for page 43
Measurement Problems in Criminal Justice Research: Workshop Summary Porterfield (1943, 1946) and Wallerstein and Wylie (1947), the work of Short and Nye (1957, 1958) “revolutionized ideas about the feasibility of using survey procedures with a hitherto taboo topic” and changed how the discipline thought about delinquent behavior itself (Hindelang et al., 1981:23). Short and Nye’s research is distinguished from previous self-report measures in their attention to methodological issues, such as scale construction, reliability and validity, and sampling and their explicit focus on the substantive relationship between social class and delinquent behavior. A 21-item list of criminal and antisocial behaviors was used to measure delinquency, although in most of their analyses a scale comprised of a subset of only seven items was employed. Focusing on the relationship between delinquent behavior and the socioeconomic status of the adolescents’ parents, Nye et al. (1958) found that relatively few of the differences in delinquent behavior among the different socioeconomic status groups were statistically significant. Short and Nye’s work stimulated much interest in both use of the self-report methodology and the relationship between some measure of social status (socioeconomic status, ethnicity, race) and delinquent behavior. The failure to find a relationship between social status and delinquency served at once to question extant theories built on the assumption that an inverse relationship did in fact exist and to suggest that the juvenile justice system may be using extra-legal factors in making decisions concerning juveniles who misbehave. A number of studies in the late 1950s and early 1960s used self-reports to examine the relationship between social status and delinquent behavior (Akers, 1964; Clark and Wenninger, 1962; Dentler and Monroe, 1961; Empey and Erickson, 1966; Erickson and Empey, 1963; Gold, 1966; Reiss and Rhodes, 1959; Slocum and Stone, 1963; Vaz, 1966; Voss, 1966). These studies advanced the use of the self-report method by applying it to different, more ethnically diverse populations (Clark and Wenninger, 1962; Gold, 1966; Voss, 1966), attending to issues concerning validity and reliability (Clark and Tifft, 1966; Dentler and Monroe, 1961; Gold, 1966), and constructing measures of delinquency that specifically addressed issues regarding offense seriousness and frequency (Gold, 1966). These studies found that, while most juveniles engaged in some delinquency, relatively few committed serious delinquency repetitively. With few exceptions, these studies supported the general conclusion that, if there were any statistically significant relationship between measures of social status and self-reported delinquent behavior, it was weak and clearly did not mirror the findings of studies using official data sources.

OCR for page 43
Measurement Problems in Criminal Justice Research: Workshop Summary During this period of time researchers began to recognize the true potential of the self-report methodology. By including questions concerning other aspects of an adolescent’s life as well as a delinquency scale on the same questionnaire, researchers could explore a host of etiological issues. Theoretically interesting issues concerning the family (Dentler and Monroe, 1961; Gold, 1970; Nye et al., 1958; Stanfield, 1966; Voss, 1964), peers (Erickson and Empey, 1963; Gold, 1970; Matthews, 1968; Reiss and Rhodes, 1964; Short, 1957; Voss, 1964), and school (Elliott, 1966; Gold, 1970; Kelly, 1974; Polk, 1969; Reiss and Rhodes, 1963) emerged as the central focus of self-report studies. The potential of the self-report methodology in examining etiological theories of delinquency was perhaps best displayed in Hirschi’s (1969) Causes of Delinquency. The use of self-report studies to examine theoretical issues continued throughout the 1970s. In addition to several partial replications of Hirschi’s arguments (Conger, 1976; Hepburn, 1976; Hindelang, 1973; Jensen and Eve, 1976), other theoretical perspectives such as social learning theory (Akers et al., 1979), self-concept theory (Jensen, 1973; Kaplan, 1972), strain theory (Elliott and Voss, 1974; Johnson, 1979), and deterrence theory (Anderson et al., 1977; Jensen et al., 1978; Silberman, 1976; Waldo and Chiricos, 1972) were evaluated using data from self-report surveys. Another development during this period was the introduction of national surveys on delinquency and drug use. Williams and Gold (1972) conducted the first nationwide survey, with a probability sample of 847 boys and girls 13 to 16 years old. Monitoring the Future (Johnston et al., 1996) is a national survey on drug use that has been conducted annually since 1975. It began as an in-school survey of a nationally representative sample of high school seniors and was expanded to include eighth- and tenth-grade students. One of the larger undertakings on a national level is the National Youth Survey (NYS), conducted by Elliott and colleagues (1985). The NYS began in 1976 by surveying a national probability sample of 1,725 youth ages 11 through 17. The survey design was sensitive to a number of methodological deficiencies of prior self-report studies and has been greatly instrumental in improving the self-report method. The NYS is also noteworthy because it is a panel design, having followed the original respondents into their thirties. Despite the expanding applications of this methodology, questions remained about what self-report instruments measure. The discrepancy in findings regarding the relationship between social status and delinquency

OCR for page 43
Measurement Problems in Criminal Justice Research: Workshop Summary based on self-report data versus official (and victim) data continued to perplex scholars. Early on, self-reports came under heavy criticism on a number of counts, including the selection of respondents and the selection of delinquency items. Nettler (1978:98) stated that “an evaluation of these unofficial ways of counting crime does not fulfill the promise that they would provide a better enumeration of offensive activity.” Gibbons (1979:84) was even more critical in his summary evaluation, stating: The burst of energy devoted to self-report studies of delinquency has apparently been exhausted. This work constituted a criminological fad that has waned, probably because such studies have not fulfilled their early promise. Two studies were particularly instrumental at that time in pointing to flaws in self-report measures. Hindelang and colleagues (1979) illustrated the problems encountered when comparing the results from studies using self-reports and those using official data or victimization data by comparing characteristics of offenders across the three data sources. They observed more similarity in those characteristics between victimization and Uniform Crime Reports data than between self-report data and the other two sources. They argued that self-report instruments did not include the more serious crimes for which people are arrested and that are included in victimization surveys. Thus, self-reports tap a different, less serious domain of behaviors than either of the other two sources, and discrepancies in observed relationships when using self-reports should not be surprising. The differential domain of crime tapped by early self-report measures could also explain the discrepancy in findings regarding the association between social status and delinquency. Elliott and Ageton (1980) also explored the methodological shortcomings of self-reports. They observed that a relatively small number of youth commit a disproportionate number of serious offenses. However, most early self-report instruments failed to include serious offenses in the inventory and truncated the response categories for the frequency of offenses. In addition, many of the samples did not include enough high-rate offenders to clearly distinguish them from other delinquents. By allowing respondents to report the number of delinquent acts they committed rather than specifying an upper limit (e.g., 10 or more) and by focusing on high-rate offenders, Elliott and Ageton found relationships between engaging in serious delinquent behavior and race and social class that are more consistent with results from studies using official data. Hindelang and colleagues (1979) and Elliott and Ageton (1980) sug

OCR for page 43
Measurement Problems in Criminal Justice Research: Workshop Summary gested designing self-report studies so that they would acquire sufficient data from those high-rate, serious offenders who would be most likely to come to the attention of the authorities. They also suggested a number of changes in the way in which self-report data are measured, so that the data reflect the fact that some offenders contribute disproportionately to the rate of serious and violent delinquent acts. The development of instruments to better measure serious offenses and the suggestion to acquire data from high-rate offenders coincided with a substantive change in the 1980s in the focus of much criminology work on the etiology of offenders. The identification of a relatively small group of offenders who commit a disproportionate amount of crime and delinquency led for a call to focus research efforts on “chronic” or “career” criminals (Blumstein et al., 1986; Wolfgang et al., 1972, 1987). Blumstein et al.’s observation that we need to study the careers of criminals, including early precursors of delinquency, maintenance through the adolescent years, and later consequences during the adult years, was particularly important in recognizing the need for examining the life-course development of high-rate offenders with self-report methodology. The self-report methodology continues to advance in terms of both its application to new substantive areas and the improvement of its design. Gibbons’s (1979) suggestion that self-reports were just a fad, likely to disappear, is clearly wrong. Rather, with improvements in question design, administration technique, reliability and validity, and sample selection, this technique is being used in the most innovative research on crime and delinquency. The sections that follow describe the key methodological developments that have made such applications possible. DEVELOPMENT OF THE SELF-REPORT METHOD Self-report measures of delinquent behavior have advanced remarkably in the 30-odd years since their introduction by Short and Nye (1957). Considerable attention has been paid to the development and improvement of their psychometric properties. The most sophisticated and influential work was done by Elliott and colleagues (Elliott and Ageton, 1980; Elliott et al., 1985; Huizinga and Elliott, 1986) and by Hindelang, Hirschi, and Weis (1979, 1981). From their work a set of characteristics for acceptable (i.e., reasonably valid and reliable) self-report scales has emerged. Five of the most salient of these characteristics are the inclusion of (1) a wide array of offenses, including serious offenses; (2) frequency response sets; (3)

OCR for page 43
Measurement Problems in Criminal Justice Research: Workshop Summary screening for trivial behaviors; (4) application to a wider age range; and (5) the use of longitudinal designs. Each is discussed below. • Inclusion of a wide array of delinquency items. The domain of crime covers a wide range of behaviors, from petty theft to aggravated assault and homicide. If the general domain of delinquent and criminal behavior is to be represented in a self-report scale, it is necessary for the scale to cover that same wide array of human activity. Simply asking about a handful of these behaviors does not accurately represent the theoretical construct of crime. In addition, empirical evidence suggests that crime does not have a clear unidimensional structure that would facilitate the sampling of a small number of items from a theoretically large pool to adequately represent the entire domain. These considerations suggest that an adequate self-report scale for delinquency will be relatively lengthy. Many individual items are required to represent the entire domain of delinquent behavior, to represent each of its subdomains, and to ensure that each subdomain (e.g., violence, drug use) is itself adequately represented. In particular, it is essential that a general self-reported delinquency scale tap serious as well as less serious behaviors. Early self-report scales tended to ignore serious criminal and delinquent events and concentrated almost exclusively on minor forms of delinquency. Failure to include serious offenses misrepresents the domain of delinquency and contaminates comparisons with other data sources. In addition, it misrepresents the dependent variable of many delinquency theories (e.g., Elliott et al., 1985; Thornberry, 1987) that attempt to explain serious, repetitive delinquency. • Inclusion of frequency response sets. Many early self-report studies relied on response sets with a relatively small number of categories, thus tending to censor high-frequency responses. For example, Short and Nye (1957) used a four-point response with the highest category being “often.” Aggregated over many items, these limited response sets had the consequence of lumping together occasional and high-rate delinquents, rather than discriminating between these behaviorally different groups. • Screening for trivial behaviors. Self-report questions have a tendency to elicit reports of trivial acts that are very unlikely to elicit official reactions and even acts that are not violations of the law. This occurs more frequently with the less serious offenses but also plagues responses to serious

OCR for page 43
Measurement Problems in Criminal Justice Research: Workshop Summary offenses. For example, respondents have included as thefts such pranks as hiding a classmate’s books in the respondent’s locker between classes, or as serious assault events that are really roughhousing between siblings. Some effort must be made to adjust or censor the data to remove these events if the delinquency of the subjects is to be reflected properly and if the rank order of subjects with respect to delinquency is to be portrayed properly. Two strategies are generally available. First, one can ask a series of follow-up questions designed to elicit more information about an event, such as the value of stolen property, the extent of injury to the victim, and the like. Second, one can use an open-ended question asking the respondent to describe the event and then probe to obtain the information necessary to classify the act. Both strategies have been used with some success. • Application to a wider age range. With increasing emphasis on the study of crime across the entire life course, self-report surveys have had to be developed to take into account both the deviant behavior of very young children and the criminal behavior of older adults. The behavioral manifestations of illegal behaviors or the precursors of such behavior can change depending on the stage in the life course at which the assessment takes place. For the very young child, measures have been developed that are administered to parents to assess antisocial behavior such as noncompliance, disobedience, and aggression (Achenbach, 1992). For the school-age child, Loeber and colleagues (1993) have developed a checklist that expands the range of antisocial behaviors to include such behaviors as stubbornness, lying, bullying, and other externalizing problems. There has been less development of instruments targeted at adults. Weitekamp (1989) has criticized self-report studies for being primarily concerned with the adolescent years and simply using the same items for adults. This is particularly crucial given the concern over the small but very significant problem of chronic violent offenders. • Use of longitudinal designs. Perhaps the most significant development in the application of the self-report methodology is its use in following the same subjects over time in order to account for changes in their criminal behavior. This has enabled researchers to examine the effect of age of onset, to track the careers of offenders, to study desistance, and to apply developmental theories to study both the causes and consequences of criminal behavior over the life course.

OCR for page 43
Measurement Problems in Criminal Justice Research: Workshop Summary While broadening the range of issues that can be examined, application of the self-report technique within longitudinal panel designs introduces potential threats to the reliability and validity of the data. In addition to concern over construct continuity in applying the technique to different-aged respondents, researchers need to consider the possibility of panel or testing effects. All of these newer procedures are likely to improve the validity, and to some extent the reliability, of self-report scales since they improve our ability to identify delinquents and to discriminate among different types of delinquents. These are clearly desirable qualities. To gain these desirable qualities, however, requires a considerable expansion of the self-report schedule. This can be illustrated by describing the major components of the index currently being used in the Rochester Youth Development Study (Thornberry et al., in press) as well as the other two projects of the Program of Research on the Causes and Correlates of Delinquency (see Browning et al., 1999). The inventory includes 32 items that tap general delinquency and 12 that tap drug use, for a total of 44 items. For each item the subjects are asked if they committed the act since the previous interview. For all items to which they respond in the affirmative, a series of follow-up questions are asked, such as whether they had been arrested. In addition, for the most serious instance of each type of delinquency reported in the past six months, subjects are asked to describe the event by responding to the question: “Could you tell me what you did?” If that open-ended question does not elicit the information needed to describe the event adequately, a series of questions, which vary from 2 to 14 probes depending on the offense, are asked. Although most of these specific questions are skipped for most subjects since delinquency remains a rare event, this approach to measuring self-reported delinquency is a far cry from the initial days of the method, when subjects used a few categories to respond to a small number of trivial delinquencies with no follow-up items. Below we evaluate the adequacy of this approach for measuring delinquent and criminal behavior. RELIABILITY AND VALIDITY For any measure to be scientifically worthwhile it must possess both reliability and validity. Reliability is the extent to which a measuring procedure yields the same result on repeated trials. No measure is absolutely, perfectly reliable. Repeated use of a measuring instrument will always pro

OCR for page 43
Measurement Problems in Criminal Justice Research: Workshop Summary duce some variation from one application to another. That variation can be very slight or quite large. So the central question in assessing the reliability of a measure is not whether it is reliable but how reliable it is; reliability is always a matter of degree. Validity is a more abstract notion. A measure is valid to the extent to which it measures the concept you set out to measure and nothing else. While reliability focuses on a particular property of the measure—namely, its stability over repeated uses—validity concerns the crucial relationship between the theoretical concept one is attempting to measure and what one actually measures. As is true with the case of reliability, the assessment of validity is not an either/or proposition. There are no perfectly valid measures, but some measures are more valid than others. We now turn to an assessment of whether self-reported measures of delinquency are psychometrically acceptable. Assessing Reliability There are two classic ways to assess the reliability of social science measures: test-retest reliability and internal consistency. Huizinga and Elliott (1986) make a convincing case that the test-retest approach is fundamentally more appropriate for assessing self-reported measures of delinquency. Internal consistency means that multiple items measuring the same underlying concept should be highly intercorrelated. Although a reasonable expectation for attitudinal measures, this expectation is less reasonable for behavioral inventories such as self-report measures of delinquency. Current self-report measures typically include 30 to 40 items measuring a wide array of delinquent acts. Just because someone was truant is no reason to expect that they would be involved in theft or vandalism. Similarly, if someone reports that they have been involved in assaultive behavior, there is no reason to assume they have been involved in drug sales or loitering. Indeed, given the relative rarity of involvement in delinquent acts, it is very likely that most people will respond in the negative to most items and in the affirmative to only a few items. This is especially the case if we are asking about short reference periods (e.g., the past year or past six months). There is no strong underlying expectation that the responses will be highly intercorrelated, and therefore an internal consistency approach to assessing reliability may not be particularly appropriate. (See Huizinga and Elliott, 1986, for a more formal discussion of this point.)

OCR for page 43
Measurement Problems in Criminal Justice Research: Workshop Summary Some theories of crime (e.g., Gottfredson and Hirschi, 1990; Jessor et al., 1991) assume there is an underlying construct, such as self-control, that generates versatility in offending. If so, there should be high internal consistency among self-reported delinquency items. While this result may be supportive of the theoretical assumption, it is not necessarily a good indicator of the reliability of the measures. If internal consistency were low, it may not have any implication for reliability but may simply mean that this particular theoretical assumption was incorrect. Nevertheless, we do note that studies that have examined the internal consistency of self-report measures generally find acceptable alpha coefficients. For example, Hindelang and colleagues report alphas between 0.76 and 0.93 for various self-report measures (1981:80). We will focus our attention on the test-retest method of assessing reliability. This approach is quite straightforward. A sample of respondents is administered a self-reported delinquency inventory (the test) and then, after a short interval, the same inventory is readministered (the retest). In doing this the same questions and the same reference period should be used at both times. The time lag between the test and the retest is also important. If it is too short, it is likely the answers provided on the retest will be a function of memory. If so, estimates of reliability would be inflated. If the time period between the test and the retest is too great, it is likely the responses given on the retest would be less accurate than those given on the test because of memory decay. In this case the reliability of the scale would be underestimated. There is no hard-and-fast rule for assessing the appropriateness of this lag, but somewhere in the range of one to four weeks appears to be optimal. A number of studies have assessed the test-retest reliability of self-reported delinquency measures. In general, the results indicate that these measures are acceptably reliable. The reliability coefficients vary somewhat depending on the number and types of delinquent acts included in the index and the scoring procedures used (e.g., simple frequencies or ever-variety scores), but scores well above 0.80 are common. In summarizing much of the previous literature in this area, Huizinga and Elliott (1986:300) state: Test-retest reliabilities in the 0.85 - 0.99 range were reported by several studies employing various scoring schemes and numbers of items and using test-retest intervals of from less than one hour to over two months (Kulik et al., 1968; Belson, 1968; Hindelang et al., 1981; Braukmann et al., 1979;

OCR for page 43
Measurement Problems in Criminal Justice Research: Workshop Summary In all but one case (violent offenses for the older group measured by official data) there is a positive effect of past offenses on future offenses after unobserved heterogeneity is held constant. In the exceptional case the number of arrests for violent offenses is so sparse over time that we do not think the estimate is reliable. Overall, therefore, these results based on the same subjects suggest that self-report and official data yield the same substantive conclusion on this central issue. Both data sources indicate there are both static and dynamic processes at work that produce the observed association between past and future offenses. CONCLUSIONS The self-report method for measuring crime and delinquency has developed substantially since it was introduced a half century ago. It is now one of the fundamental ways to scientifically measure criminality, and it forms the bedrock of etiological studies. The challenges confronting this approach to measurement are daunting; after all, individuals are asked to tell about their own undetected criminality. Despite this fundamental challenge, the technique seems to be successful and capable of producing valid and reliable data. Early self-report scales had substantial weaknesses, containing few items and producing an assessment of only minor forms of offending. Gradually, as the underlying validity of the approach became evident, the scales expanded in terms of breadth, seriousness, and comprehensiveness. Contemporary measures typically cover a wide portion of the behavioral domain included under the construct of crime and delinquency. These scales are able to measure serious as well as minor forms of crime, major subdomains (such as violence, property crimes, and drug use), and different parameters of criminal careers (such as prevalence, frequency, and seriousness) and identify high-rate as well as low-rate offenders. This is substantial progress for a measurement approach that began with a half dozen items and a four-category response set. The self-report approach to measuring crime has acceptable, albeit far from perfect, reliability and validity. Of these two basic psychometric properties, the evidence for reliability is stronger. There are no fundamental challenges to the reliability of these data. Test-retest measures (and internal consistency measures) indicate that self-reported measures of delinquency are as reliable as, if not more reliable than, most social science measures.

OCR for page 43
Measurement Problems in Criminal Justice Research: Workshop Summary Validity, as noted above, is much harder to assess as there is no gold standard by which to judge self-reports. Nevertheless, current scales seem to have acceptable levels of content and construct validity. The evidence for criterion validity is less clear-cut. At an overall level, criterion validity seems to be in the moderate-to-strong range. While there is certainly room for improvement, the validity appears acceptable for most analytical tasks. At a more specific level, however, there is a potentially serious problem with differential validity in that African American males have lower validity than do Hispanic males. Additional research on this topic is imperative. While basic self-report surveys appear to be reliable and valid, researchers have experimented with a variety of data collection methods to improve the quality of reporting. Several of these attempts have produced ambiguous results; for example, there is no clear-cut benefit to the mode of administration (interview vs. questionnaire) or the use of randomized response techniques. There is one approach that appears to hold great promise— audio-assisted computerized interviews, which produce increased reporting of many sensitive topics, including delinquency and drug use. Greater use of this approach is warranted. In the end, the available data indicate that the self-report method is an important and useful way to collect information about criminal behavior. The skepticism of early critics like Nettler (1978) and Gibbons (1979) has not been realized. Nevertheless, the self-report technique can clearly be improved. The final topic addressed in this chapter concerns suggestions for future research. Future Directions Much of our research on reliability and validity simply assesses these characteristics; there is much less research on improving their levels. For example, it is likely that both validity and reliability would be improved if we experimented with alternative items for measuring the same behavior and identified the strongest ones. Similarly, reliability and validity vary across subscales (e.g., Huizinga and Elliott, 1986); improving subscales will not only help them but also the overall scale as they are aggregated. This chapter raised the issue of differential validity for African American males. It is crucial that more is learned about the magnitude of this bias and, if it exists, its source. Future research should address this issue directly and attempt to identify techniques for eliminating it. These re

OCR for page 43
Measurement Problems in Criminal Justice Research: Workshop Summary search efforts should not lose sight of the fact that the problem may be with the criterion variable (official records) and not the self-reports. The self-report method was developed in and for cross-sectional studies. Using it in longitudinal studies, especially ones that cover major portions of the life course, creates a new set of challenges. Maintaining the age appropriateness of the items while at the same time ensuring content validity is a knotty problem that we have just begun to address. There is some evidence that repeated measures may create testing effects. More research is needed to measure the size of this effect and its sources and to identify methods to reduce its threat to the validity of self-report data in the longitudinal studies so crucial to etiological investigation. The similarities and differences in our understanding of criminal career parameters in self-report data and official data are just beginning to be investigated. This approach began with official data but is increasingly coming to rely on self-report data. It is important that we understand more about the validity of both types of data for these purposes. Finally, we recommend that methodological studies be done in a cross-cutting fashion so that several of these issues—reliability and validity, improved item selection, assessing panel bias—can be investigated simultaneously. In particular it is important to examine all of these methodological issues when data are collected using audio-assisted computerized interviewing. For example, studies that have found differential validity or testing effects have all used paper-and-pencil interviews. Whether these same problems are evident under the enhanced confidentiality of audio interviews is an open question. It is clearly a high-priority one as well. There is no dearth of work that can be done to assess and improve the self-report method. If the progress of the past half century is any guide, we are optimistic that the necessary studies will be conducted and that they will improve this basic way of collecting data on criminal behavior. REFERENCES Achenbach, T.M. 1992 Manual for the Child Behavior Checklist/2-3 and 1992 Profile. Burlington: University of Vermont. Akers, R.L. 1964 Socio-economic status and delinquent behavior: A retest. Journal of Research in Crime and Delinquency 1:38-46.

OCR for page 43
Measurement Problems in Criminal Justice Research: Workshop Summary Akers, R.L., M.D. Krohn, L. Lanza-Kaduce, and M. Radosevich 1979 Social learning and deviant behavior: A specific test of a general theory. American Sociological Review 44:636-655. Akers, R.L., J. Massey, W. Clarke, and R.M. Lauer 1983 Are self-reports of adolescent deviance valid? Biochemical measures, randomized response, and the bogue pipeline in smoking behavior. Social Forces 62(September):234-251. Anderson, L.S., T.G. Chiricos, and G.P. Waldo 1977 Formal and informal sanctions: A comparison of deterrent effects. Social Problems 25:103-112. Aquilino, W.S. 1994 Interview mode effects in surveys of drug and alcohol use. Public Opinion Quarterly 58:210-240. Aquilino, W.S., and L. LoSciuto 1990 Effects of interview mode on self-reported drug use. Public Opinion Quarterly 54:362-395. Beebe, T.J., P.A. Harrison, J.A. McRae, Jr., R.E. Anderson and J.A. Fulkerson 1998 An evaluation of computer-assisted self-interviews in a school setting. Public Opinion Quarterly 62:623-632. Belsky, J., S. Woodworth, and K. Crnic 1996 Troubled family interaction during toddlerhood. Development and Psychopathology 8:477-495. Belson, W.A. 1968 The extent of stealing by London boys and some of its origins. Advancement of Science 25:171-184. Blackmore, J. 1974 The relationship between self reported delinquency and official convictions amongst adolescent boys. British Journal of Criminology 14:172-176. Blumstein, A., J. Cohen, J.A. Roth, and C.A. Visher 1986 Criminal Careers and Career Criminals. Washington, DC: National Academy Press. Brame, R., S. Bushway, R. Paternoster, and T.P. Thornberry 2001 Temporal Linkages in Violent and Nonviolent Criminal Activity. Unpublished manuscript, University of South Carolina, Columbia. Braukmann, C.J., K.A. Kirigin, and M.M. Wolf 1979 Social Learning and Social Control Perspectives in Group Home Delinquency Treatment Research. Paper presented to the American Society of Criminology, Philadelphia. Broder, P.K., and J. Zimmerman 1978 Establishing the Reliability of Self-Reported Delinquency Data. Williamsburg, VA: National Center for State Courts. Browning, K., D. Huizinga, R. Loeber, and T.P. Thornberry 1999 Causes and Correlates of Delinquency Program, Fact Sheet, Washington, DC: U.S. Department of Justice, Office of Juvenile Justice and Delinquency Prevention. Campbell, S.B. 1987 Parent-referred problem three-year-olds: Developmental changes in symptoms. Journal of Child Psychology and Psychiatry 28:835-845.

OCR for page 43
Measurement Problems in Criminal Justice Research: Workshop Summary Campbell, S.B. 1990 Behavioral Problems in Preschool Children: Clinical and Developmental Issues. New York: Guilford Press. Christensen, R. 1997 Log-Linear Models and Logistic Regression, 2nd ed. New York: Springer-Verlag. Clark, J.P., and L.L. Tifft 1966 Polygraph and interview validation of self-reported delinquent behavior. American Sociological Review 31:516-523. Clark, J.P., and E.P. Wenninger 1962 Socioeconomic class and area as correlates of illegal behavior among juveniles. American Sociological Review 28:826-834. Conger, R. 1976 Social control and social learning models of delinquency: A synthesis. Criminology 14:17-40. Drug Use Forecasting 1990 Drug Use Forecasting Annual Report: Drugs and Crime in America. Washington, DC: U.S. Department of Justice. Dentler, R.A., and L.J. Monroe 1961 Social correlates of early adolescent theft. American Sociological Review 26:733-743. Elliott, D.S. 1966 Delinquency school attendance and dropout. Social Problems 13:306-318. 1994 Serious violent offenders: Onset, developmental course, and termination. Criminology 32:1-21. Elliott, D.S., and S.S. Ageton 1980 Reconciling race and class differences in self-reported and official estimates of delinquency. American Sociological Review 45:95-110. Elliott, D.S., and H.L. Voss 1974 Delinquency and Dropout. Lexington, MA: D.C. Heath. Elliott, D.S., D. Huizinga, and S.S. Ageton 1985 Explaining Delinquency and Drug Use. Beverly Hills, CA: Sage. Empey, L.T., and M. Erickson 1966 Hidden delinquency and social status. Social Forces 44(June):546-554. Erickson, M. and L.T. Empey 1963 Court records, undetected delinquency and decision-making. Journal of Criminal Law, Criminology, and Police Science 54:456-469. Farrington, D.P. 1973 Self-reports of deviant behavior: Predictive and stable? Journal of Criminal Law and Criminology 64:99-110. 1986 Stepping Stones to Adult Criminal Careers. In Development of Antisocial and Prosocial Behavior, D. Olweus, J. Block, and M.R. Yarrow, eds. New York: Academic Press. 1989a Early predictors of adolescent aggression and adult violence. In Violence and Victims. Washington, DC: Springer. 1989b Self-reported and official offending from adolescence to adulthood. In Cross-

OCR for page 43
Measurement Problems in Criminal Justice Research: Workshop Summary National Research in Self-Reported Crime and Delinquency, M.W. Klein, ed. Los Angeles: Kluwer Academic Publishers. Farrington, D.P., R. Loeber, M. Stouthamer-Loeber, W.B. Van Kammen, and L. Schmidt 1996 Self-reported delinquency and a combined delinquency seriousness scale based on boys, mothers, and teachers: Concurrent and predictive validity for African-American and Caucasians. Criminology 34:493-517. Farrington, D.P., D. Jolliffe, J.D. Hawkins, R.F. Catalano, K.G. Hill, and R. Kosterman 2000 Comparing Delinquency Careers in Court Records and Self-reports. Unpublished manuscript, Cambridge University, United Kingdom. Gibbons, D.C. 1979 The Criminological Enterprise: Theories and Perspectives. Englewood Cliffs, NJ: Prentice-Hall. Gold, M. 1966 Undetected delinquent behavior. Journal of Research in Crime and Delinquency 3:27-46. 1970 Delinquent Behavior in an American City. Belmont, CA: Brooks/Cole. Gottfredson, M.R., and T. Hirschi 1990 A General Theory of Crime. Stanford, CA: Stanford University Press. Hardt, R.H., and S. Petersen-Hardt 1977 On determining the quality of the delinquency self-report method. Journal of Research in Crime and Delinquency 14:247-261. Hathaway, R.S., E.D. Monachesi, and L.A. Young 1960 Delinquency rates and personality. Journal of Criminal Law, Criminology, and Police Science 50:433-440. Hawkins, J.D., R.F. Catalano, and J.Y. Miller 1992 Risk and protective factors for alcohol and other drug problems in adolescence and early adulthood: Implications for substance abuse prevention. Psychological Bulletin 112:64-105. Hepburn, J.R. 1976 Testing alternative models of delinquency causation. Journal of Criminal Law and Criminology 67:450-460. Hindelang, M.J. 1973 Causes of delinquency: A partial replication and extension. Social Problems 20: 471-487. Hindelang, M.J., T. Hirschi, and J.G. Weis 1979 Correlates of delinquency: The illusion of discrepancy between self-report and official measures. American Sociological Review 44:995-1014. 1981 Measuring Delinquency. Beverly Hills, CA: Sage. Hirschi, T. 1969 Causes of Delinquency. Berkeley: University of California Press. Huesmann, L.R., L.D. Eron, M.M. Lefkowitz, and L.O. Walder 1984 The stability of aggression over time and generations. Developmental Psychology 20:1120-1134. Huizinga, D., and D.S. Elliott 1983 A Preliminary Examination of the Reliability and Validity of the National Youth

OCR for page 43
Measurement Problems in Criminal Justice Research: Workshop Summary Survey Self-Reported Delinquency Indices. National Youth Survey Project Report 27. Boulder, CO: Behavioral Research Institute. 1986 Reassessing the reliability and validity of self-report delinquent measures. Journal of Quantitative Criminology 2:293-327. Huizinga, D., A.W. Weiher, S. Menard, R. Espiritu, and F.A. Esbensen 1998 Some not so boring findings from the Denver Youth Survey. Paper presented at the annual meeting of American Society of Criminology, Washington, D.C. Jensen, G.F. 1973 Inner containment and delinquency. Journal of Criminal Law and Criminology 64:464-470. Jensen, G.F., and R. Eve 1976 Sex differences in delinquency. Criminology 13:427-448. Jensen, G.F., M.L. Erickson, and J.P. Gibbs 1978 Perceived risk of punishment and self-reported delinquency. Social Forces 57:57-78. Jessor, R. 1998 New Perspectives on Adolescent Risk Behavior. New York: Cambridge University Press. Jessor, R., J.E. Donovan, and F.M. Costa 1991 Beyond Adolescence: Problem Behavior and Young Adult Development. Cambridge, England: Cambridge University Press. Johnson, R.E. 1979 Juvenile Delinquency and Its Origins. Cambridge, England: Cambridge University Press. Johnston, L.D., P.M. O’Malley, and J.G. Bachman 1996 National Survey Results on Drug Use from the Monitoring the Future Study, 1975-1995. Washington, DC: U.S. Government Printing Office. Kaplan, H.B. 1972 Toward a general theory of psychosocial deviance: The case of aggressive behavior. Social Science and Medicine 6:593-617. Kelly, D.H. 1974 Track position and delinquent involvement: A preliminary analysis. Sociology and Social Research 58:380-386. Klein, M.W., ed. 1989 Cross-National Research in Self-Reported Crime and Delinquency. Los Angeles: Kluwer Academic Publishers. Krohn, M.D., G.P. Waldo, and T.G. Chiricos 1974 Reported delinquency: A comparison of structured interviews and self-administered check-lists. Journal of Criminal Law and Criminology 65:545-553. Krohn, M.D., A.J. Lizotte, and C.M. Perez 1997 The interrelationship between substance use and precocious transitions to adult statutes. Journal of Health and Social Behavior 38:87-103. Krohn, M.D., T.P. Thornberry, C. Rivera, and M. LeBlanc 2001 Later delinquency careers. In Child Delinquents: Development, Intervention, and Service Needs, R. Loeber and D.P. Farrington, eds. Thousand Oaks, CA: Sage.

OCR for page 43
Measurement Problems in Criminal Justice Research: Workshop Summary Kulik, J.A., K.B. Stein, and T.R. Sarbin 1968 Disclosure of delinquent behavior under conditions of anonymity and nonanonymity. Journal of Consulting and Clinical Psychology 32:506-509. Lauritsen, J.L. 1998 The age-crime debate: Assessing the limits of longitudinal self-report data. Social Forces 76:1-29. LeBlanc, M. 1989 Designing a self-report instrument for the study of the development of offending from childhood to adulthood: Issues and problems. Pp. 371-398 in Cross-National Research in Self-Reported Crime and Delinquency, M.W. Klein, ed. Los Angeles: Kluwer Academic Publishers. Lehnen, R.G., and A.J. Reiss 1978 Response effects in the National Crime Survey. Victimology: An International Journal 3:110-124. Loeber, R., M. Stouthamer-Loeber, W.B. Van Kammen, and D.P. Farrington 1989 Development of a new measure of self-reported antisocial behavior for young children: Prevalence and reliability. Pp. 203-225 in Cross-National Research in Self-Reported Crime and Delinquency, M.W. Klein, ed. Los Angeles: Kluwer Academic Publishers. Loeber, R., P. Wung, K. Keenan, B. Giroux, and M. Stouthamer-Loeber 1993 Developmental pathways in disruptive child behavior. Development and Psychopathology 5:101-133. Loeber, R., D.P. Farrington, M. Stouthamer-Loeber, T.E. Moffitt, and A. Caspi 1998 The development of male offending: Key findings from the first decade of the Pittsburgh Youth Study. Studies on Crime and Crime Prevention 7:1-31. Matthews, V.M. 1968 Differential identification: An empirical note. Social Problems 14:376-383. Maxfield, M.G., B.L. Weiler, and C.S. Widom 2000 Comparing self-reports and official records of arrests. Journal of Quantitative Criminology 16:87-100. Menard, S. and D.S. Elliott 1993 Data set comparability and short-term trends in crime and delinquency. Journal of Criminal Justice 21:433-445. Moffitt, T.E. 1993 Life-course-persistent and adolescence-limited antisocial behavior: A developmental taxonomy. Psychological Review 100:674-701. 1997 Adolescence-limited and life-course-persistent offending: A complementary pair of developmental theories. Pp. 11-54 in Developmental Theories of Crime and Delinquency, Volume 7: Advances in Criminological Theory, T.P. Thornberry, ed. New Brunswick, NJ: Transaction Publishers. Nagin, D.S., and R. Paternoster 1991 On the relationship of past and future participation in delinquency. Criminology 29:163-190. 2000 Population heterogeneity and state dependence: State of the evidence and directions for future research. Journal of Quantitative Criminology 16:117-145.

OCR for page 43
Measurement Problems in Criminal Justice Research: Workshop Summary Nettler, G. 1978 Explaining Crime. New York: McGraw-Hill. Nye, F.I., J.F. Short, and V.J. Olson 1958 Socioeconomic status and delinquent behavior. American Journal of Sociology 63:381-389. Olweus, D. 1979 Stability and aggressive reaction patterns in males: A review. Psychological Bulletin 86:852-875. Osgood, D.W., P. O’Malley, J. Bachman, and L. Johnston 1989 Time trends and age trends in arrests and self-reported illegal behavior. Criminology 27:389-418. Patterson, G.R. 1993 Orderly change in a stable world: The antisocial trait as a chimera. Journal of Consulting and Clinical Psychology 61:911-919. Patterson, G.R., and R. Loeber 1982 The Understanding and Prediction of Delinquent Child Behavior. Research proposal to National Institute of Mental Health from the Oregon Social Learning Center, Eugene. Polk, K. 1969 Class strain and rebellion among adolescents. Social Problems 17:214-224. Porterfield, A.L. 1943 Delinquency and outcome in court and college. American Journal of Sociology 49:199-208. 1946 Youth in Trouble. Fort Worth, TX: Leo Potishman Foundation. Reiss, A.J., Jr., and A.L. Rhodes 1959 A Socio-Psychological Study of Adolescent Conformity and Deviation. Washington, DC: U.S. Office of Education. 1963 Status deprivation and delinquent behavior. Sociological Quarterly 4:135-149. 1964 An empirical test of differential association theory. Journal of Research in Crime and Delinquency 1:5-18. Richman, N., J. Stevenson, and P.J. Graham 1982 Preschool to School: A Behavioural Study. London: Academic Press. Rojek, D.G. 1983 Social status and delinquency: Do self-reports and official reports match? In Measurement Issues in Criminal Justice, G.P. Waldo, ed. Beverly Hills, CA: Sage. Sampson, R.J., and J.H. Laub 1990 Crime and deviance over the life course: The salience of adult social bonds. American Sociological Review 55:609-627. Saris, W.E. 1991 Computer-Assisted Interviewing. Beverly Hills, CA: Sage. Sellin, T. 1931 The basis of a crime index. Journal of Criminal Law and Criminology 22:335-356. Shaw, D.S., and R.Q. Bell 1993 Developmental theories of parental contributors to antisocial behavior. Journal of Abnormal Child Psychology 21:35-49.

OCR for page 43
Measurement Problems in Criminal Justice Research: Workshop Summary Short, J.F. 1957 Differential association and delinquency. Social Problems 4:233-239. Short, J.F., Jr., and F.I. Nye 1957 Reported behavior as a criterion of deviant behavior. Social Problems 5:207-213. 1958 Extent of unrecorded juvenile delinquency: Tentative conclusions. Journal of Criminal Law and Criminology 49:296-302. Silberman, M. 1976 Toward a theory of criminal deterrence. American Sociological Review 41:442-461. Slocum, W.L., and C.L. Stone 1963 Family culture patterns and delinquent-type behavior. Marriage and Family Living 25:202-208. Skolnick, J.V., C.J. Braukmann, M.M. Bedlington, K.A. Kirigin, and M.M. Wolf 1981 Parent-youth interaction and delinquency in group homes. Journal of Abnormal Child Psychology 9:107-119. Stanfield, R. 1966 The interaction of family variables and gang variables in the aetiology of delinquency. Social Problems 13:411-417. Thornberry, T.P. 1987 Toward an interactional theory of delinquency. Criminology 25:863-891. 1989 Panel effects and the use of self-reported measures of delinquency in longitudinal studies. Pp. 347-369 in Cross-National Research in Self-Reported Crime and Delinquency, M.W. Klein, ed. Los Angeles: Kluwer Academic Publishers. Thornberry, T.P., ed. 1997 Developmental Theories of Crime and Delinquency. New Brunswick, NJ: Transaction Publishers. Thornberry, T.P., and M.D. Krohn 2001 The development of delinquency: An interactional perspective. Pp. 289-305 in Handbook of Law and Social Science: Youth and Justice, S.O. White, ed. New York: Plenum. Thornberry, T.P., M.D. Krohn, A.J. Lizotte, C.A. Smith, and K. Tobin In Press The Toll of Gang Membership: Gangs and Delinquency in Developmental Perspective. New York: Cambridge University Press. Thrasher, F. 1927 The Gang: A Study of 1,313 Gangs in Chicago. Chicago: University of Chicago Press. Tourangeau, R. and T.W. Smith 1996 Asking sensitive questions: The impact of data collection, mode, question format, and question context. Public Opinion Quarterly 60:275-304. Tracy, P.E., and J.A. Fox 1981 The validity of randomized response for sensitive measurements. American Sociological Review 46:187-200. Tremblay, R.E., C. Japel, D. Perusse, P. McDuff, M. Boivin, M. Zoccolillo, and J. Montplaisir 1999 The search for the age of ‘onset’ of physical aggression: Rousseau and Bandura revisited. Criminal Behaviour and Mental Health 9:8-23.

OCR for page 43
Measurement Problems in Criminal Justice Research: Workshop Summary Turner, C.F., J.T. Lessler, and J. Devore 1992 Effects of mode of administration and wording on reporting of drug use. Pp. 177-220 in Survey Measurement of Drug Use: Methodological Studies, C.F. Turner, J.T. Lessler, and J.C. Gfroerer, eds. Washington, DC: U.S. Department of Health and Human Services. Vaz, E.W. 1966 Self-reported juvenile delinquency and social status. Canadian Journal of Corrections 8:20-27. Voss, H.L. 1963 Ethnic differentials in delinquency in Honolulu. Journal of Criminal Law and Criminology 54:322-327. 1964 Differential association and reported delinquent behavior: A replication. Social Problems 12:78-85. 1966 Socio-economic status and reported delinquent behavior. Social Problems 13:314-324. Waldo, G.P., and T.G. Chiricos 1972 Perceived penal sanction and self-reported criminality: A neglected approach to deterrence research. Social Problems 19:522-540. Wallerstein, J.S., and C.J. Wylie 1947 Our law-abiding law-breakers. Probation 25:107-112. Weis, J.G., and D.V. Van Alstyne 1979 The Measurement of Delinquency by the Randomized Response Method. Paper presented at the meeting of the American Society of Criminology, Philadelphia. Weitekamp, E. 1989 Some problems with the use of self-reports in longitudinal research. Pp. 329-346 in Cross-National Research in Self-Reported Crime and Delinquency, M.W. Klein, ed. Los Angeles: Kluwer Academic Publishers. Widom, C.S. 1989 Child abuse, neglect, and violent criminal behavior. Criminology 27:251-271. Williams, J.R., and M. Gold 1972 From delinquent behavior to official delinquency. Social Problems 20(2):209-229. Wolfgang, M.E., R.M. Figlio, and T. Sellin 1972 Delinquency in a Birth Cohort.Chicago: University of Chicago Press. Wolfgang, M.E., T.P. Thornberry, and R.M. Figlio 1987 From Boy to Man, From Delinquency to Crime.Chicago: University of Chicago Press. Wright, D.L., W.S. Aquilino and A.J. Supple 1998 A comparison of computer-assisted and paper-and-pencil self-administered questionnaires in a survey on smoking, alcohol, and drug use. Public Opinion Quarterly 62:331-353.