Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics (2009)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

–1– Introduction F INDING CONTINUAL EVIDENCE of “the inadequacy of the available data on crime and the criminal justice system,” the President’s Commission on Law Enforcement and Administration of Justice (1967:269) recommended that “a National Criminal Justice Statistics Cen- ter should be established in the Department of Justice.” The commission anticipated that this center would: serve as a central focus for other statistics related to the crime problem, such as costs of crime, census data, and victim surveys. It would have to work in close coordination with the FBI’s [Federal Bureau of Investi- gation’s] Uniform Crime Reports Section, the Children’s Bureau of the Department of Health, Education, and Welfare, the Federal Bureau of Prisons, and other existing agencies with continuing responsibility for collecting and reporting related statistics. It would combine their infor- mation into an integrated picture of crime and criminal justice. The Omnibus Crime Control and Safe Streets Act of 1968 created the Law Enforcement Assistance Administration (LEAA) within the U.S. Department of Justice to provide financial and technical support to state and local law enforcement agencies. The act specifically authorized LEAA to “collect, evaluate, publish, and disseminate statistics and other information on the condition and progress of law enforcement in the several States” (P 90- .L. 351 § 515(b); 82 Stat. 207). To meet this mandate, a National Criminal Justice Information and Statistics Service (NCJISS) was founded within the new LEAA.1 The Justice Systems Improvement Act of 1979 (P 96-157) .L. 1 The NCJISS originally bore the name “National Criminal Justice Statistics Center,” draw- ing directly from the commission’s recommendation, but the name was changed in short order. 23 

24 JUSTICE STATISTICS renamed and authorized the NCJISS as the Bureau of Justice Statistics (BJS) within a new Office of Justice Assistance, Research, and Statistics. Subse- quent reauthorization (the Justice Assistance Act of 1984, P 98–473) com- .L. pleted the process of converting the former LEAA into the Office of Justice Programs (OJP), which remains BJS’s administrative parent agency within the Justice Department. From these beginnings, BJS has developed into one of the principal sta- tistical agencies of the federal government. Armed with a broad mandate to provide statistical measures on the justice system, the agency maintains dozens of data collection series. Each year, it releases about 40 bulletins or reports summarizing its findings, disseminated through BJS’s own website (http://www.ojp.usdoj.gov/bjs/) or through the OJP-administered National Criminal Justice Reference Service (http://www.ncjrs.gov/). BJS data are gen- erally made available in processed spreadsheets on the BJS website or in mi- crodata form at the National Archive of Criminal Justice Data hosted by the University of Michigan (http://www.icpsr.umich.edu/NACJD/). BJS also sup- ports the maintenance of an online Sourcebook of Criminal Justice Statistics (http://www.albany.edu/sourcebook). BJS’s report series and dissemination venues are described more completely in Box 1-1, and an illustrative front page of a BJS bulletin is shown in Figure 1-1. BJS’s signature data collection, and its most demanding in terms of bud- get resources, is the National Crime Victimization Survey (NCVS), an ef- fort that—like BJS itself—is the direct result of a recommendation by the President’s Commission on Law Enforcement and Administration of Justice (1967). The NCVS serves as a critical indicator of crime and violence in the United States because it includes crimes that are not reported to police as well as those that are. In this respect, it serves as an important counterpart to the Uniform Crime Reporting (UCR) program of the FBI—the nation’s other key indicator of violent crime levels—because the UCR is strictly lim- ited to incidents reported to law enforcement authorities. More significantly, the importance of the NCVS stems from its flexibility as a detailed survey measurement tool, permitting valuable insight into the nature and etiology of victimization incidents as well as public perceptions of and encounters with other parts of the justice system. Although the NCVS represents a dominant share of BJS’s budgetary re- sources, the balance of BJS’s data collection portfolio covers an enormous range of phenomena. BJS is well known for its body of data series on popu- lations under correctional supervision; these provide important information on the levels and dynamics of correctional populations, which are (like other institutionalized populations) commonly excluded from household surveys 

INTRODUCTION 25 Box 1-1 Bureau of Justice Statistics Publications and Data Dissemination Venues Historically, BJS publications have followed a few fairly well-defined types, as described in the Bureau of Justice Statistics Style Book (Bureau of Justice Statistics, 1997a:5): • Bulletins summarize findings from new data releases from BJS’s more permanent data collections. They are meant to be relatively concise, having five to eight tables, and frequently have a detailed methodology section that describes the processes used to collect the data. The Criminal Victimization series based on the National Crime Victimization Survey (NCVS; e.g., Rand and Catalano, 2007) and the midyear count reports from the National Prisoner Statistics Program (e.g., Harrison and Beck, 2006; Sabol and Couture, 2008) are examples of BJS Bulletins. • Special Reports cover “more restricted topics than do Bulletins, describing in 10 to 15 tables statistical relationships among findings from one or more datasets.” They are meant to be more individual in nature than the Bulletins. Example Special Reports include analyses of victimization rates by level of urbanicity (Duhart, 2000), detailed concentration on violent felons using State Court Processing Statistics data (Reaves, 2006), tabulations of citizen complaints regarding police use of force from the Law Enforcement Management and Adminstrative Statistics survey (Hickman, 2006), and results from a supplemental survey of civil appeals (Cohen, 2006). • Data Briefs are meant to cover a very limited set of findings and are generally a maximum of two formatted pages. For instance, a special NCVS tabulation of carjacking incidents covers three formatted pages (Klaus, 2004). However, depending on the topic, they can run longer; for instance, the analysis by Mumola (2007b) of cause-of-death information collected as part of the Deaths in Custody Reporting Program has a three-page narrative but includes eight pages of appendix tables. Because of space limitations, Data Briefs do not cover methodology in any depth but can contain references to other, more extensive reports. In recent years, very short reports have also been issued under the Fact Sheet label (see, e.g., Hughes, 2007, on analysis of National Center for Health Statistics data on unidentified human remains; and Hickman, 2003, on tribal law enforcement departments). • Selected Findings “gathe[r] the most important facts, statistics, and conclusions from a number of data sources, usually separate statistical reporting programs within BJS.” Examples of Selected Findings reports include summaries from both inmate and probationer data sets on prior physical or sexual abuse (Harlow, 1999) and analysis of punitive damage awards in selected large counties (Cohen, 2005b). Other BJS publications have been labeled as BJS Technical Reports, including the results of pilot testing of computer security questions in the NCVS (Rantala, 2004) and results from a special NCVS data file for the New York, Los Angeles, and Chicago metropolitan statistical areas (Lauritsen and Schaum, 2005). Some longer publications are left “unbranded” by any of these labels, such as BJS’s tabulation of statistics from various sources of American Indians and their experience of crime (Greenfeld and Smith, 1999; Perry, 2004) and the Compendium of Federal Justice Statistics (Bureau of Justice Statistics, 2006a). (continued) 

26 JUSTICE STATISTICS Box 1-1 (continued) Established by BJS’s parent Office of Justice Programs (OJP), the National Criminal Justice Reference Service (NCJRS; http://www.ncjrs.gov) maintains a library of OJP publications dating to the 1970s, many of which are downloadable online and others that can be ordered in hard copy. In addition to OJP and its component agencies, the NCJRS is also sponsored by the Office of National Drug Control Policy in the Executive Office of the President. “By referral from BJS,” NCJRS also “handles major distributions as needed for White House and [Department of Justice] events and attends major conferences representing the statistical products available from BJS” (Bureau of Justice Statistics, 2005a:24). The National Archive of Criminal Justice Data (NACJD; http://www.icpsr.umich.edu/ NACJD/) is the designated official repository for data collections funded by three OJP bureaus: BJS, the National Institute of Justice, and the Office for Juvenile Justice and Delinquency Prevention. In addition to the OJP bureaus, NACJD also collects and disseminates data sets from research projects that are contributed by investigators; it also archives summary data files from the Federal Bureau of Investigation’s Uniform Crime Reporting and National Incident-Based Reporting System collections (Heraux, 2007). (Section 4–C describes both collections in more detail.) The NACJD archive was created in 1978 and is hosted by the Inter-university Consortium for Political and Social Research (ICPSR), University of Michigan, and receives funding from the John D. and Catherine T. MacArthur Foundation in addition to the three OJP bureaus. Arrangements with NACJD and ICPSR also give BJS an important venue for promoting accurate and effective use of BJS data. As part of ICPSR’s regular Summer Training Program in Quantitative Methods, BJS sponsors an annual 4-week seminar program on the quantitative analysis of crime and criminal justice data; since 2000, the seminar has focused on key methodological and presentation issues rather than attempting a comprehensive review of all BJS data series. The seminar is open to researchers from academia, nonprofit organizations, and government agencies, but annual attendance is capped at 10 persons. For fiscal year 2008, BJS’s contribution to the maintenance of NCJRS and NACJD were estimated at $1,100,000 and $900,000, respectively. administered by the government.2 The “justice system” that BJS monitors with its data collections is sprawling, including not only correctional facil- ities and offices but the entire infrastructure of law enforcement: police departments at the state and local levels and their various support bureaus. It also includes the judiciary: the full array of federal and state courts, which vary strongly in organizational structure and information resources. BJS’s 2 An exception is the American Community Survey—the replacement for the traditional de- cennial census long-form sample—which does include prison populations in its coverage of the “group quarters” population. However, the BJS data series are likely better measures of cor- rectional populations, particularly for shorter-term or mixed-term facilities such as local jails where the decennial census definition of “usual residence” may lead to poor counts; see Sec- tion 5–B.11. National Research Council (2006) provides additional discussion of correctional populations and the decennial census. 

INTRODUCTION 27 Figure 1-1 First page of example from BJS “Bulletin” series of data releases legal mandate is such that its work is not limited to the collection and anal- ysis of data; it is also tasked with providing financial assistance to state and local governments for the development and maintenance of information re- sources such as background check databases. Some of BJS’s data collections are performed on a regular basis whereas others are more sporadic (or one- time efforts), and some involve direct field data collection and interviewing whereas others are based on administrative and agency records. 

28 JUSTICE STATISTICS Despite this wide scope, BJS has endured effectively flat funding for most of its existence. Figure 1-2 presents BJS’s budget requests and total appropri- ated amounts for each fiscal year since 1981. Converting BJS’s final budget allocations into real 2008 dollars, the funding has oscillated slightly but held relatively constant over the entire range. The figure also shows the amount of BJS’s budget allocated to the NCVS for each fiscal year since 1990, in nominal dollars; that line suggests an upward creep in the basic costs of survey data collection. During fiscal years 2001 through 2007, the NCVS consumed at least 51.2 percent (in both 2001 and 2002) and as much as 64.0 percent (in 2004) of the total BJS appropriations. These fiscal con- straints have led BJS to reduce the sample size of NCVS over time, along with other cost-cutting measures; what began as a survey of 72,000 house- holds in 1972 reached only 38,000 households in 2006. In recent years, the diminishing NCVS sample size has combined with generally low and de- creasing estimated overall victimization rates, with the result that only large percentage changes in violent crime victimization rates—at least 8 percent— would be a statistically significant year-to-year change.3 As noted in the U.S. Office of Management and Budget (2007:8) annual review of statistical pro- gram funding, “cost cutting measures applied to the NCVS continue to have significant effects on the precision of the estimates—year-to-year change es- timates are no longer feasible and have been replaced with two-year rolling averages” in BJS reports on victimization. In addition to decreased precision in the NCVS estimates, fiscal constraints and the large NCVS share of the overall budget have raised trade-off decisions in the portfolio of BJS pro- grams: Should some resources currently devoted to the NCVS be applied to collections in other areas such as law enforcement or judicial processing (at the expense of NCVS accuracy), or do the variety and extent of non-NCVS collections detract from the accurate victimization-based measurement of crime and violence? In 2005, BJS was subjected to review under the U.S. Office of Manage- ment and Budget’s Performance Assessment Rating Tool (PART).4 In general, PART found fault with only a few areas and BJS received high marks. How- ever, one need that the evaluation suggested was an independent assessment of BJS’s effectiveness: BJS would benefit from a comprehensive review that could provide spe- cific evidence of BJS’s impact overall. Major reviews of BJS statistical 3 This figure is from a presentation by Michael Rand, BJS, at the panel’s first meeting. For 2005, the violent crime victimization rate was estimated as 21.2 per 1,000 population, and the 95 percent confidence interval of this rate is ±8.1 percent. Since 2000, the estimated annual violent crime victimization rates have dipped from 27.9 to 21.2 per 1,000, and the 95 percent intervals have been in excess of ±7 percent for each of those annual estimates. These figures are discussed further in the panel’s interim report (National Research Council, 2008b:App. C). 4 The detailed report of the PART evaluation is accessible at http://www.whitehouse.gov/ omb/expectmore/detail/10003805.2005.html. 

INTRODUCTION 29 (a) Expenditures in nominal fiscal year dollars (millions) (b) BJS total allocations in real 2008 dollars (millions) Figure 1-2 Bureau of Justice Statistics budget requests to Congress (fiscal years 1981–2009), final total appropriations (1981–2008), and National Crime Victimization Survey (NCVS) data collection costs (1990–2008) NOTES: Budget request and appropriations figures include both base and program costs. Final appropriations reflect amounts after any applicable budget recission or across-the-board cut. NCVS data collection costs exclude additional costs for developing and respecifying sample based on new decennial censuses, as well as costs associated with the automation (conversion to computer-based administration) of the survey. For fiscal year 2008, NCVS spending does not include an additional $3.9 million designated for redesign activities. Consistent with the approach used in National Research Council (2009:Table A-1), nominal dollars are converted to real 2008 dollars by the gross domestic product chain-type price indexes for federal government nondefense consumption expenditures (based on Table 3.10.4, line 34, at http://www.bea.gov/ national/nipaweb/SelectTable.asp?Selected=N#S3). SOURCE: Data provided by the Bureau of Justice Statistics to the panel at its February 2007, December 2007, and April 2008 meetings. 

30 JUSTICE STATISTICS activities conducted by the Census Bureau, the American Statistical As- sociation, the National Academy of Sciences, and other external groups have concluded that BJS adheres to standards of quality and practice that are consistent with the expectations for a national statistics agency. Such reviews are important for providing feedback and confirming suc- cess at an operational level. However, they do not provide any infor- mation on BJS’s ultimate impact, which focuses on the production of national crime and justice statistics. Is BJS collecting the right kinds of statistics to meet the nation’s needs? Are changes needed in the nation’s system for collecting and producing crime and justice statistics? Would the nation be better served by an alternative organization (e.g., a con- solidated statistical agency) for producing crime and justice statistics? These are the kinds of questions a more comprehensive review could address. 1–A CHARGE TO THE PANEL Embracing this suggestion for external, independent review—and near- ing a milestone of 30 years of operation in its present form—BJS requested that the National Research Council’s Committee on National Statistics, in collaboration with the Committee on Law and Justice, establish a Panel to Review the Programs of the Bureau of Justice Statistics. The full charge to the panel is to: examine the full range of programs of the Bureau of Justice Statistics (BJS) in order to assess and make recommendations for BJS’ priorities for data collection. The review will examine the ways in which BJS statistics are used by Congress, executive agencies, the courts, state and local agencies, and researchers in order to determine the impact of BJS programs and the means to enhance that impact. The review will assess the organization of BJS and its relationships with other data gathering entities in the Department of Justice, as well as with state and local governments, to determine ways to improve the relevance, quality, and cost-effectiveness of justice statistics. The review will consider priority uses for additional funding that may be obtained through budget ini- tiatives or reallocation of resources within the agency. A focus of the panel’s work will be to consider alternative options for conducting the National Crime Victimization Survey, which is the largest BJS program. The goal of the panel’s work will be to assist BJS to refine its priorities and goals, as embodied in its strategic plan, both in the short and longer terms. The panel’s recommendations will address ways to improve the impact and cost-effectiveness of the agency’s statistics on crime and the criminal justice system. Given the prominence of the NCVS in BJS operations—and its domi- nance of BJS budget resources—the panel was specifically asked to evaluate options for conducting the NCVS in our first year of work before turning 

INTRODUCTION 31 to the agency’s data collections in other areas. The panel’s interim report, Surveying Victims: Options for Conducting the National Crime Victimiza- tion Survey (National Research Council, 2008b), focuses on that portion of the panel’s charge. To give our work a unified presentation, we include the summary from the interim report as Appendix B. This summary includes all of the formal recommendations from that interim report and, to be clear, we stand by all of the guidance in that report—indeed, a few of the rec- ommendations from the interim report are directly restated in this volume. That said, our intent in this final report is to complement the interim report rather than update it or incorporate it in full; the interim report delves into the methodology of the NCVS and the range of possible models for the con- duct of the NCVS in much greater detail than the treatment of the NCVS in this report (as one part of the full suite of BJS programs) permits. 1–B WHAT IS THE BUREAU OF JUSTICE STATISTICS? It is impossible to properly evaluate the programs of BJS without dis- cussing the agency itself—its functions and mandates, and its placement in both the U.S. Department of Justice and the overall federal statistical system. In this section, to get a sense of the responsibilities of and demands placed on BJS, we discuss various answers to the basic question, “What is the Bureau of Justice Statistics?” To be clear, we do not imply any hierarchy of importance through the ordering of this list, save that it makes sense to start with the two most basic definitions of the agency under the law.5 In this section—and throughout this report—we refer frequently to the enumerated duties of BJS as they are presented in the agency’s authorizing legislation; these are listed in Box 1-2. 1–B.1 Mission-Type Definitions A first, basic definition of BJS is that it is a gatherer of information on crime and on the justice system. The first stated purpose of the section of leg- islation authorizing BJS is “to provide for and encourage the collection and analysis of statistical information concerning crime, juvenile delinquency, and the operation of the criminal justice system and related aspects of the civil justice system” (42 USC § 3731). The second, dual purpose of BJS under its authorizing language is to serve as a developer of justice information systems on all governmental lev- els. Specifically, BJS is tasked to “support the development of information 5 The portions of the Justice Systems Improvement Act of 1979 that created BJS (and subse- quent revisions) are codified as Title 42, Chapter 46, Subchapter III of the U.S. Code (42 USC §§ 3741–3745). Functions of BJS and its role within OJP are also defined in Title 28, Part 0, Subpart P-1 of the Code of Federal Regulations (28 CFR §§ 0.90, 0.93). 

32 JUSTICE STATISTICS Box 1-2 Statutory Functions of the Bureau of Justice Statistics The Bureau is authorized to— 1. make grants to, or enter into cooperative agreements or contracts with public agencies, institutions of higher education, private organizations, or private individuals for purposes related to this subchapter; grants shall be made subject to continuing compliance with standards for gathering justice statistics set forth in rules and regulations promulgated by the Director; 2. collect and analyze information concerning criminal victimization, including crimes against the elderly, and civil disputes; 3. collect and analyze data that will serve as a continuous and comparable national social indication of the prevalence, incidence, rates, extent, distribution, and attributes of crime, juvenile delinquency, civil disputes, and other statistical factors related to crime, civil disputes, and juvenile delinquency, in support of national, State, and local justice policy and decisionmaking; 4. collect and analyze statistical information, concerning the operations of the criminal justice system at the Federal, State, and local levels; 5. collect and analyze statistical information concerning the prevalence, incidence, rates, extent, distribution, and attributes of crime, and juvenile delinquency, at the Federal, State, and local levels; 6. analyze the correlates of crime, civil disputes and juvenile delinquency, by the use of statistical information, about criminal and civil justice systems at the Federal, State, and local levels, and about the extent, distribution and attributes of crime, and juvenile delinquency, in the Nation and at the Federal, State, and local levels; 7. compile, collate, analyze, publish, and disseminate uniform national statistics concerning all aspects of criminal justice and related aspects of civil justice, crime, including crimes against the elderly, juvenile delinquency, criminal offenders, juvenile delinquents, and civil disputes in the various States; 8. recommend national standards for justice statistics and for insuring the reliability and validity of justice statistics supplied pursuant to this chapter; 9. maintain liaison with the judicial branches of the Federal and State Governments in matters relating to justice statistics, and cooperate with the judicial branch in assuring as much uniformity as feasible in statistical systems of the executive and judicial branches; 10. provide information to the President, the Congress, the judiciary, State and local governments, and the general public on justice statistics; 11. establish or assist in the establishment of a system to provide State and local governments with access to Federal informational resources useful in the planning, implementation, and evaluation of programs under this Act; 12. conduct or support research relating to methods of gathering or analyzing justice statistics; 13. provide for the development of justice information systems programs and assistance to the States and units of local government relating to collection, analysis, or dissemination of justice statistics [Clause revised in 1984, replacing more general “financial and technical assistance” language]; 14. develop and maintain a data processing capability to support the collection, aggregation, analysis and dissemination of information on the incidence of crime and the operation of the criminal justice system [Clause added in 1984]; (continued) 

INTRODUCTION 33 Box 1-2 (continued) 15. collect, analyze and disseminate comprehensive Federal justice transaction statistics (including statistics on issues of Federal justice interest such as public fraud and high technology crime) and to provide technical assistance to and work jointly with other Federal agencies to improve the availability and quality of Federal justice data [Clause added in 1984]; 16. provide for the collection, compilation, analysis, publication and dissemination of information and statistics about the prevalence, incidence, rates, extent, distribution and attributes of drug offenses, drug related offenses and drug dependent offenders and further provide for the establishment of a national clearinghouse to maintain and update a comprehensive and timely data base on all criminal justice aspects of the drug crisis and to disseminate such information [Clause added in 1988]; 17. provide for the collection, analysis, dissemination and publication of statistics on the condition and progress of drug control activities at the Federal, State and local levels with particular attention to programs and intervention efforts demonstrated to be of value in the overall national anti-drug strategy and to provide for the establishment of a national clearinghouse for the gathering of data generated by Federal, State, and local criminal justice agencies on their drug enforcement activities [Clause added in 1988]; 18. provide for the development and enhancement of State and local criminal justice information systems, and the standardization of data reporting relating to the collection, analysis or dissemination of data and statistics about drug offenses, drug related offenses, or drug dependent offenders [Clause added in 1988]; 19. provide for improvements in the accuracy, quality, timeliness, immediate acces- sibility, and integration of State criminal history and related records, support the development and enhancement of national systems of criminal history and re- lated records including the National Instant Criminal Background Check System, the National Incident-Based Reporting System, and the records of the National Crime Information Center, facilitate State participation in national records and information systems, and support statistical research for critical analysis of the improvement and utilization of criminal history records [Clause added in 1988 and revised in 2006, including reference to specific program names]; 20. maintain liaison with State and local governments and governments of other nations concerning justice statistics; 21. cooperate in and participate with national and international organizations in the development of uniform justice statistics; 22. ensure conformance with security and privacy requirement of section 3789g of this title and identify, analyze, and participate in the development and implementation of privacy, security and information policies which impact on Federal and State criminal justice operations and related statistical activities [Clause added in 1984, replacing previous clause on privacy and security]; and 23. exercise the powers and functions set out in subchapter VIII of this chapter. [These include authority to issue rules and regulations, consult with the Bureau of Justice Assistance on grant evaluation programs, deny or terminate grants for failure to comply with regulations, convene hearings (including subpoena power) as necessary, and issue an annual report to Congress and the president.] SOURCE: Excerpt, 42 USC § 3732(c); italicized comments based on revision history and notes maintained by U.S. Code Service. 

34 JUSTICE STATISTICS and statistical systems at the Federal, State, and local levels to improve the efforts of these levels of government to measure and understand the levels of crime, juvenile delinquency, and the operation of the criminal justice system and related aspects of the civil justice system” (42 USC § 3731). Federal reg- ulation that describes the role of the BJS director takes a broader view of the audience for BJS support: The director “performs functions and administers programs, including provision of financial assistance, [to] provide a variety of statistical services for the criminal justice community” (28 CFR § 0.93). These first two basic roles are embodied in the two clauses of BJS’ for- mal mission statement, which BJS describes as an “operationalized” version of “the statutory statement of purpose” for the agency (Bureau of Justice Statistics, 2005a:10): It is the mission of BJS to collect, process, analyze, and disseminate accurate and timely information on crime and the administration of jus- tice and to assist States and localities to improve criminal justice record- keeping. 1–B.2 Statistical System Definitions A fundamental definition of BJS—which we examine in greater detail in Chapter 5—is that it is a principal agency in the decentralized U.S. federal statistical system. Relative to other countries, the U.S statistical system is exceptionally decentralized; rather than vest authority for the production of official statistics in a single entity, the U.S. model has been to add mea- surement programs (and agencies) over time as needs for information have evolved. Currently, about 80 federal government agencies each spent at least $500,000 on statistical activities, including data analysis and develop- ment of statistical models in addition to actual data collection efforts (U.S. Office of Management and Budget, 2007:Table 1). BJS is among the agen- cies that meet the definition of “federal statistical agency” expressed in the Committee on National Statistics’ white paper, Principles and Practices for a Federal Statistical Agency: “a unit of the federal government whose princi- pal function is the compilation and analysis of data and the dissemination of information for statistical purposes” (National Research Council, 2005b:2). More formally, BJS is considered a “principal” statistical agency because it holds a seat on the Interagency Council on Statistical Policy (ICSP), which is established as a coordinating body for federal statistics; see Box 1-3. Consistent with BJS’s role as a statistical agency, BJS’s enabling legislation includes the provision that (42 USC § 3735): Data collected by [BJS] shall be used only for statistical or research pur- poses, and shall be gathered in a manner that precludes their use for law enforcement or any purpose relating to a private person or public agency other than statistical or research purposes. 

INTRODUCTION 35 Box 1-3 “Principal Statistical Agencies” and the Interagency Council on Statistical Policy The 1995 reauthorization of the Paperwork Reduction Act vested authority to “coordinate the activities of the Federal statistical system”—to ensure “the efficiency and effectiveness of the system” and the “integrity, objectivity, impartiality, utility, and confidentiality of information collected for statistical purposes”—in the U.S. Office of Management and Budget (OMB; 44 USC § 3504(e)(1)). OMB’s Statistical and Science Policy Office, headed by the Chief Statistician of the United States, is located in OMB’s Office of Information and Regulatory Affairs. The act also formalized the role of the Interagency Council on Statistical Policy, a group of heads of major statistical agencies that meets monthly and discusses common issues. The ICSP predates the 1995 amendments but functioned on a more informal basis prior to being written into law (44 USC § 3504(e)(8)). The ICSP is chaired by the Chief Statistician and those agencies with seats on the council are typically referred to as the “principal statistical agencies” of the U.S. federal government. Though the law mandates the creation of the ICSP it does not explic- , itly list its membership, and so agencies have been added to the ICSP by OMB over time. Currently, the ICSP has 14 members located in 12 cabinet departments or independent agencies. The major statistical agencies represented on the council are: • from the Department of Agriculture, the Economic Research Service and the Na- tional Agricultural Statistics Service; • from the Department of Commerce, the Bureau of Economic Analysis and the Census Bureau; • from the Department of Education, the National Center for Education Statistics; • from the Department of Energy, the Energy Information Administration; • from the Department of Health and Human Services, the National Center for Health Statistics • from the Department of Justice, the Bureau of Justice Statistics; • from the Department of Labor, the Bureau of Labor Statistics; and • from the Department of Transportation, the Bureau of Transportation Statistics. Four other agencies or offices are also designated as principal statistical agencies—the Office of Environmental Information (Environmental Protection Agency), the Statistics of Income Division of the Internal Revenue Service (Treasury), the Science Resources Statistics Division (National Science Foundation), and the Office of Policy (Social Security Administration)—but are not included on some tabulations of major statistical agencies (e.g., U.S. Office of Management and Budget, 2007:App. B). It is also fair to extend this definition in at least two other respects, rel- ative to other units in the federal statistical system. First, BJS is one of the smallest of the principal statistical agencies in the federal system. BJS’s current organizational structure, including counts of positions classified as statisticians, is shown in Figure 1-3. Of the 10 major agencies represented on the ICSP BJS’s actual fiscal year 2006 funding level of $50.2 million ranks , ninth, ahead only of the Bureau of Transportation Statistics (U.S. Office of Management and Budget, 2007:Table 1).6 Its full-time permanent staff of 6 This $50.2 million does not match the funding level shown in Figure 1-2 because, as 

36 JUSTICE STATISTICS Figure 1-3 Bureau of Justice Statistics organizational structure, June 2008 SOURCE: Adapted from organizational chart provided by BJS to the panel, July 2008. 51 (again, as of actual numbers in fiscal year 2006) was the smallest of the 10 major agencies, the next-smallest being the National Center for Educa- tion Statistics at 91 (U.S. Office of Management and Budget, 2007:App. B). Introducing a hearing on the limitations of existing crime statistics, then- Representative Charles Schumer (D-New York) summarized the small re- sources of BJS relative to those of its peers (U.S. House of Representatives, Committee on the Judiciary, 1991:116): Polls show that crime and the economy are the two most important issues to most Americans, so let me draw a comparison between the two. With the economy, we have the Labor Department’s Bureau of Labor Statistics. It has a staff of well over 2,000 people, an annual budget of $240 million, and it does an excellent job of measuring all sorts of economic indicators. When the Bureau of Labor Statistics says something, people know it is true. Markets go up and down, waiting for those statistics to be announced. For crime, the lead agency is the Bureau of Justice Statistics. It has a staff of 50 people, a budget of about $20 million, less than one-quarter of 1 percent of what we spend at the Federal level for the war on drugs and a miniscule proportion of what we spend as a nation on law enforcement in general. It is no wonder we have such an incomplete understanding of what is going on [in crime]. Referring to Box 1-2, it is useful to make another definitional point: BJS is a statistical agency with one of the most elaborate and extensive lists of noted in U.S. Office of Management and Budget (2007:Table 1), “the amounts for BJS [include] estimated salaries and expenses that are not directly appropriated” and that “the FY 2006 amounts for BJS include carryover funds and any other prior year recoveries.” 

INTRODUCTION 37 formal duties compared to those of its peers. By comparison, an entire title of the U.S. Code is dedicated to “Census” issues (Title 13) but the code ar- ticulates no formal list of duties for the Census Bureau or its director; rather, the code outlines duties and responsibilities of the Secretary of Commerce, and the Director of the Census is charged only to “perform such duties as may be imposed upon him by law, regulations, or orders of the Secretary” (13 USC § 21). The basic mandate of the Bureau of Labor Statistics is con- cise: “to acquire . . . useful information on subjects connected with labor, in the most general and comprehensive sense of that word, and especially upon its relation to capital, the hours of labor, the earnings of laboring men and women, and the means of promoting their material, social, intellectual, and moral prosperity” (29 USC § 1). The legal duties of some of the newer statistical agencies (such as BJS) tend to be more specific than those of the older agencies: enabling law describes nine specific data collection themes for the National Center for Health Statistics (42 USC § 242k(1)), while 9 specific duties and 13 data collection themes are outlined for the Bureau of Transportation Statistics (49 USC § 111(c)). Still, relative to its peers, the legal demands put on BJS are unusually detailed and wide-ranging, from specific data collection types (e.g., duty 16 on collection of information on drug-related crime) to administrative support functions for state and local agencies. 1–B.3 Organizational Definitions A shorthand definition that BJS commonly uses to describe itself is that it is “the statistical arm of the U.S. Department of Justice.” This phrasing is used in the agency’s strategic plan (Bureau of Justice Statistics, 2005a:1) and in other materials (Greenfeld, 2004). The language used in the strate- gic plan as context for this definition strikes a few slightly different notes from language used in BJS’s legally defined duties (see Box 1-2), saying that BJS is “responsible for the collection, analysis, publication, and dissemina- tion of statistical information on crime, criminal offenders, victims of crime, and the operations of justice systems at all levels of government” (Bureau of Justice Statistics, 2005a:1). A semantic point, but an important one, is that BJS’s self-description as a “statistical arm” of the Department of Justice (rather than “statistical agency” or “statistical office,” or some such term) subtly connotes a dedication to the furtherance of Department of Justice policy objectives rather than the atmosphere of independence that statistical agencies should be permitted. Indeed, as we describe in more detail later in Section 5–A.2, BJS has endured incidents in which Justice Department pol- icy practices have directly conflicted with statistical agency independence, culminating in a high-profile (and damaging) removal of a BJS director. Setting aside the semantics of “statistical arm” for the moment, the for- 

38 JUSTICE STATISTICS mulation of BJS as “the” statistical arm of the Justice Department is odd. As it is presently operated, it is more correct to say that BJS is a statistical arm of the U.S. Department of Justice. Although BJS is responsible for a great deal of the statistical information collected by the Department of Jus- tice (the organizational structure of which is described in Figure 1-4), it is far from the only data-gathering entity in the department.7 Spanning the many program branches, policy branches, and federal law enforcement agencies in the department, an illustrative (but by no means exhaustive) list of major data-driven activities includes: • Use of census and other data by the Civil Rights Division to study equity in legislative districting in certain states and the potential for vote dilution; • Maintenance of extensive administrative (purchase and transfer) databases by the Bureau of Alcohol, Tobacco, Firearms, and Explo- sives, as well as operation of a national database of images of bullet and cartridge case evidence related to local crime scenes; • Analysis of the effectiveness of “weed and seed” grants to localities by the Community Capacity Development Office; • Compilation of data on arrests and seizures made by Drug Enforce- ment Administration officers; • Population and inventory reports maintained by the Federal Bureau of Prisons; and • Analyses of drug markets by the National Drug Intelligence Center, in- cluding fielding of a National Drug Threat Survey to law enforcement agencies. The UCR program—the record of crimes reported to the police—is arguably the longest-standing and highest-profile of the department’s statistical series. However, it is administered by the FBI, not BJS. Similarly, primary responsi- bility for data collections on juvenile offenders and victims is generally held by the Office of Juvenile Justice and Delinquency Prevention (OJJDP), one of BJS’s sister agencies in OJP; this is the case even though, as a reading of the duties articulated in Box 1-2 suggests, BJS’ authorizing language is replete in references to juvenile delinquency. (We discuss the relationships between BJS and the FBI and OJJDP in particular, in Section 4–C and Section 2–C.3, , respectively.) 7 The fiscal year 2008 version of Statistical Programs of the U.S. Government (U.S. Office of Management and Budget, 2007:Table 1) lists four Department of Justice agencies in its basic table of agencies with direct funding for “statistical activities” of at least $500,000: BJS ($50.2 million in fiscal 2006), the Bureau of Prisons ($13.0 million), the FBI ($7.6 million), and the Drug Enforcement Administration ($2.2 million). 

INTRODUCTION Figure 1-4 U.S. Department of Justice organizational structure, March 2009 NOTE: The Bureau of Justice Statistics (BJS) is a constituent agency of the Office of Justice Programs (OJP), indicated here by boldface type. Box 1-4 depicts the organizational structure within OJP, and Figure 1-3 shows the internal organization of BJS. SOURCE: Adapted from http://www.usdoj.gov/dojorg.htm, organizational chart dated March 2, 2009. 39 

40 JUSTICE STATISTICS Administratively, BJS is an organ of OJP within the U.S. Department of Justice. The current structure of OJP is described more fully in Box 1-4. This administrative placement has two fundamental ramifications for BJS and its work. First, it means that BJS is administratively nested within the Justice Department; BJS’s director reports to the attorney general and the deputy at- torney general through a designated assistant attorney general. This type of administrative layering and reporting structure is increasingly common for federal statistical agencies, although it is not ideal for a statistical agency’s purpose as an independent broker of information. Second, the adminis- trative tie to OJP means that, like other units in the office, BJS inherits a strong focus of attention on the needs of state and local law enforcement, since OJP is the legal successor of the previous Law Enforcement Assistance Administration. OJP is a program agency that takes as its general mission “provid[ing] federal leadership in developing the nation’s capacity to pre- vent and control crime, administer justice, and assist crime victims” (Bureau of Justice Statistics, 2005a:10). The closing sentence of BJS’s basic authoriz- ing language explicitly directs that “[BJS] shall give primary emphasis to the problems of State and local justice systems” (42 USC § 3731). 1–B.4 Functional Definitions In terms of BJS’s functions, it may be said that BJS is principally a paying sponsor of data collection efforts and a manager of grants. Although BJS staff are actively engaged in the design of data collections and the analysis of re- sulting data, many of BJS’s data series—including signature collections such as the NCVS—are not conducted in-house. Instead, they are done by con- tracting with an external data collection agent. Notably, BJS contracts with the Census Bureau for many of its data series, including the NCVS and its censuses of prisons and jails. It also issues contracts with, among others, the National Opinion Research Center, Westat, the Police Executive Research Forum, and the National Center for State Courts. For fiscal year 2008, bud- get estimates called for BJS to spend $45 million out of an expected $61.5 million in direct funding on purchasing statistical services (73 percent). Of that total, about half ($23.2 million) was budgeted to go to other federal agencies (e.g., the Census Bureau), a smaller share to private-sector orga- nizations ($18.2 million), and a small share to state and local governments ($3.6 million) (U.S. Office of Management and Budget, 2007:Table 3). In addition to data collection–specific contracts, a major part of BJS’s work is the provision of grants to state and local law enforcement units, consistent with the strong state and local focus described above. In particular, BJS ad- ministers the National Criminal History Improvement Program of grants to state and local governments, and has issued grants to try to encourage coop- 

INTRODUCTION 41 Box 1-4 Organizational Structure of the Office of Justice Programs The Office of Justice Programs (OJP) was established in 1984, inheriting major functions from the former Law Enforcement Assistance Administration (LEAA). LEAA had been formed in 1968 to coordinate the flow of federal financial resources to state and local law enforcement efforts. OJP’s mission is “to increase public safety and improve the fair administration of justice across America through innovative leadership programs,” and it has adopted as its “vision” the following (U.S. Department of Justice, Office of Justice Programs, 2006:3): To be the premier resource for the justice community by providing and coordi- nating information, statistics, research and development, training, and support to help the justice community build the capacity it needs to meet its public safety goals; embracing local decision making and encouraging local innova- tion through strong and intelligent national policy leadership. Current law defines powers of OJP as follows (42 USC § 3712(a)): The Assistant Attorney General [for OJP] shall— (1) publish and disseminate information on the conditions and progress of the criminal justice systems; (2) maintain liaison with the executive and judicial branches of the Federal and State governments in matters relating to criminal justice; (3) provide information to the President, the Congress, the judiciary, State and local governments, and the general public relating to criminal justice; (4) maintain liaison with public and private educational and research insti- tutions, State and local governments, and governments of other nations relating to criminal justice; (5) provide staff support to coordinate the activities of the Office and the Bu- reau of Justice Assistance, the National Institute of Justice, the Bureau of Justice Statistics, and the Office of Juvenile Justice and Delinquency Prevention; and (6) exercise such other powers and functions [as may be defined or dele- gated]. (continued) 

42 JUSTICE STATISTICS Box 1-4 (continued) Section 5–A.2 describes the evolution of point (5) in this list in greater detail. Aside from the Bureau of Justice Statistics (BJS), major units within OJP include: • The Bureau of Justice Assistance (BJA) provides leadership in grant adminis- tration and policy development to support state, local, and tribal criminal justice strategies to achieve safe communities. • The Community Capacity Development Office (CCDO) works with local communities to analyze public safety and criminal justice problems, develop solutions, and foster local-level leadership to implement and sustain these solutions. For example, it oversees the “Weed and Seed” initiative—community- based activities to promote the arrest and sanction of offenders (“weed”) and crime prevention and community revitalization (“seed”). • The National Institute of Justice (NIJ) is the research, development, and evaluation agency of the U.S. Department of Justice, dedicated to researching crime-control and justice issues. • The Office of Juvenile Justice and Delinquency Prevention (OJJDP) provides national leadership, coordination, and resources to prevent and respond to juvenile delinquency and victimization. • The Office for Victims of Crime (OVC) is committed to enhancing the nation’s capacity to assist crime victims and to providing leadership in changing attitudes, policies, and practices to promote justice and healing for all crime victims. It provides federal funds to support victim compensation and assistance and training programs. • Most recently, the Office of Sex Offender Sentencing, Monitoring, Appre- hending, Registering, and Tracking (or SMART Office) was created by law in 2006. The assistant attorney general who oversees OJP operations is appointed by the president with the advice and consent of the Senate, as are the heads of BJA, BJS, NIJ, OJJDP and OVC. The CCDO director is appointed by the attorney general while the , SMART Office director is a presidential appointment without Senate confirmation. SOURCES: Organizational chart adapted from http://www.ojp.usdoj.gov/about/bureaus. htm (8/1/08) and http://www.usdoj.gov/dojorg.htm (1/14/07). Capsule descriptions of agencies adapted from http://www.ojp.usdoj.gov/index.htm (1/14/07) and individual bureau websites. eration by local police departments with the FBI’s National Incident-Based Reporting System. BJS also functions as a synthesizer of information from other federal agen- cies and from state governments. At the federal level, BJS acquires some of its data through partnerships with other agencies inside and outside of the Department of Justice, including the Bureau of Prisons and the Administra- tive Office of the U.S. Courts. It also acquires some of its data directly from state governments as, indeed, it is explicitly urged to do in its legal statement of purpose: “The Bureau shall utilize to the maximum extent feasible State 

INTRODUCTION 43 governmental organizations and facilities responsible for the collection and analysis of criminal justice data and statistics” (42 USC § 3731). In its relationship with the states, BJS has also cultivated a role as an ac- tive broker for research and data dissemination within the states. Through its State Justice Statistics program, BJS provides grant assistance to a strong network of state Statistical Analysis Centers (SACs), some of which are colo- cated with state government or law enforcement agencies and others of which are installed at academic institutions. These SACs act as a source of information to BJS as well as a conduit for dissemination of information among state policy makers (e.g., in fielding questions from state legislators). BJS funding also supports the Justice Research and Statistics Association, which provides coordination to the SACs and which operates a research journal. BJS is also frequently called upon to play a role as a “criminal justice in- vestigator,” typically when Congress mandates the collection of information on some aspect of the criminal justice system. Several of these are described in Box 3-3 and one major such request—for information on the occurrence of sexual violence in correctional facilities, as mandated by the Prison Rape Elimination Act of 2003—is described in detail in Section 5–A.1. On oc- casion, it is not Congress but the administration—the Justice Department— that creates an “investigatory” role for BJS, as with BJS’s role in coordinat- ing an 18-city tour to corroborate perceived increases in violent crime with accounts from local police officials (Rosenfeld, 2007). BJS has also been directed by legislation to produce and calculate the formulas used in grant programs administered by other OJP programs, such as the Bureau of Jus- tice Assistance’s Edward Byrne Memorial Justice Assistance Grant program (Hickman, 2005a). It has also assumed responsibility for producing certain compilations and summaries of state law, such as an overview series of state privacy and security laws that dates from the LEAA days (Bureau of Jus- tice Statistics, 2003b) and, more recently, summaries of state laws and rules governing firearm sales and transfers (Regional Justice Information Service, 2006). 1–C LIMITATIONS OF THE STUDY The preceding review of the varied roles of BJS and the expectations placed on the agency underscores the point that our panel’s charter to review the full suite of BJS programs and suggest strategic priorities is a very broad one. Accordingly, although we believe our review to be as comprehensive as possible, it is important to make two caveats up front before proceeding to the substance of our analysis. 

44 JUSTICE STATISTICS The first is that we intend our review to suggest directions for research, development, and study to further BJS’s objectives and priorities for data collection; it is neither intended as a set of specific budgetary priorities to reduce BJS costs nor as an uncontrolled “wish list” of topics that only a BJS with a massive funding increase could hope to accomplish. On several occasions, BJS Director Jeffrey Sedgwick described the work of our panel in the context of a broader examination of BJS’s mission and data series (Justice Research and Statistics Association, 2006a:12; see also Sedgwick, 2006): While we have routinely evaluated each of approximately four dozen data series (and received high marks for each), we have not evaluated whether we are doing the right things. . . . Put in econometric terms, we haven’t asked whether the value of the least important data we collect exceeds the value of the most valuable data we don’t collect. Consonant with this direction, we have approached our study with an eye toward gaps in substantive coverage that should be filled as well as collec- tions that should be suspended or dropped. However, to expand a note we struck in our interim report (National Research Council, 2008b:2), we rec- ognize the fiscal constraints on BJS but do not intend to be either strictly limited by them (assuming relatively flat funding for the indefinite future) or completely indifferent to them. We approach our work as suggesting a map of several possible avenues for a more effective BJS but not necessarily a single, unique path with specific tasks to meet a certain budget constraint. In that regard, we identify research questions and development issues that necessarily precede a priority-setting agenda. Our panel is not constituted to perform a full financial audit of BJS—and we do not think such a task is en- visioned in our charge—but a consequence is that our capacity to completely rank-order individual data collections and programs based on a cost-benefit analysis is necessarily limited. We return to these concepts, in particular, in Chapter 6 and our examination of priority setting in a climate of limited resources. The second caveat is a simpler limitation of space: The wide scope of BJS programs and the magnitude of the task of assessing the whole portfo- lio preclude us from delving too deeply into the details of any single data series. Pursuant to BJS’s requests, we provided a book-length treatment of one series—the NCVS—in our interim report. This report is not intended as a comprehensive sourcebook in which every BJS data series is granted the same level of review and analysis as we gave the NCVS; time is too short and the terrain is too broad for that level of detail on every series. Indeed, some of the measurement issues we touch on in our review have received extensive treatment from other National Research Council panels and workshops, in- cluding data needs in policing and prosecution (National Research Council, 2001b, 2004a) and measurement difficulties concerning the use of firearms and illegal drugs (National Research Council, 2001a, 2005a). In this report, 

INTRODUCTION 45 we provide a brief overview of BJS data series, but speak more in terms of statistical coverage in major theme areas (e.g., law enforcement or adjudica- tion, generally). 1–D REPORT CONTENTS We begin our analysis in Chapter 2 by discussing general perspectives on the justice system on which BJS is tasked to provide statistical measures. Chapter 3 provides a brief inventory of BJS’s data collections in the broad topic areas of victimization, law enforcement, corrections, and adjudication. BJS’s partnership with the states through its SACs, and its grant programs to state and local governments are outlined in Chapter 4. We describe and assess BJS’s programs and functions relative to expected principles and prac- tices of a federal statistical agency in Chapter 5. Chapter 6 articulates a set of four strategic goals for BJS, and includes summaries of the mapping of our detailed recommendations and findings to each strategic goal.