Ensuring the Quality, Credibility, and Relevance of U.S. Justice Statistics (2009)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

–A– Findings and Recommendations This appendix lists the panel’s findings and recommendations in this final report for ease of reference. Finding 2.1: The data on crime currently collected by BJS are primarily fo- cused on street crime. This focus on certain forms of violent and property crime does not account for important or emerging types of crime—notably, many forms of white-collar crime such as corporate fraud, health care fraud, financial institution fraud, money laundering, government fraud, consumer fraud, public corruption, and Internet crimes. The broad area of civil justice proceedings—distinct from criminal justice—is represented by one principal data series in BJS’s extensive portfolio, and is limited by its construction to cover only completed court cases (and not out-of-court settlements). BJS’s slate of cross-sectional series also does not readily provide for comprehen- sive analyses of contextual factors such as drugs and their impact on crime and violence. Finding 2.2: Responsibility within the U.S. Department of Justice for coor- dinating and organizing data collections on juveniles is generally assumed by the Office of Juvenile Justice and Delinquency Prevention (OJJDP), instead of BJS. Though BJS’s series do cover some segments of the juvenile popula- tion (e.g., juveniles housed in adult correctional facilities), the results of BJS and OJJDP studies are not well integrated. Within both BJS’s and OJJDP’s statistical coverage, there remain substantial gaps in data for juvenile offend- ers and victims with respect to their processing through the justice system “funnel.” 325 

326 JUSTICE STATISTICS Recommendation 2.1: Consistent with its legal mandate to collect, analyze, and disseminate statistical information on all aspects of the justice system, BJS should (a) document and organize the available statistics on forms of crime not covered by the NCVS, the FBI’s UCR and NIBRS data systems, and other major data series maintained by other statistical agencies, (b) pur- sue research on what new statistics could be feasibly and usefully developed, and (c) propose such new data collections as the research suggests to be both feasible and useful. BJS should strive to function as a clearinghouse of justice-related statistical information, including reference to data not di- rectly collected by BJS. Recommendation 2.2: In line with its original charge and to better docu- ment and understand the contribution of juveniles to street crime and vio- lence, the victimization of youth, and the consequences for youth and society of their victimization and offending, BJS should develop juvenile victimiza- tion, crime, and justice statistical series suitable for describing the patterns of offending and victimization of youth, longitudinal progression of youth through the juvenile and criminal justice systems, and reentry into the com- munity and criminal system. Taking on this responsibility would require additional resources. Finding 3.1: BJS currently gathers data about the criminal justice system but it does so on an institution-by-institution basis (police, courts, correc- tions) using varying units of analysis (crimes, individuals, cases) and some- times varying time periods and samples. This approach provides good cross- sectional assessments of parts of the system, but makes it difficult or impos- sible to answer questions about the flow of individuals from arrest through eventual exit from the system. Yet people exit the system at many different stages in ways that are ill-understood but consequential for the effective- ness and fairness of criminal justice system processes. The cross-sectional approach misses the interfaces between the institutions, such as the large but unknown number of individuals who are arrested but not prosecuted. Recommendation 3.1: BJS’s goal in providing statistics from basic adminis- trative data on corrections should be the development of a yearly count of correctional populations capable of disaggregation and cross-tabulation by state, offense categories, and demographic groups (age, race, gender, educa- tion). Recommendation 3.2: BJS should produce yearly transition rates be- tween steps in the corrections process capable of disaggregation and cross- tabulation by state, offense categories, and demographic groups. Recommendation 3.3: BJS should explore the possibilities of increasing the utility of their correctional data collections by facilitating the linkage of 

APPENDIX A 327 records across the data series. For example, the ability to link records from the Recidivism Studies or from NCRP to the Census of Adult Correctional Facilities (CACF) would increase the ability to understand how correctional facilities contribute to recidivism. Recommendation 3.4: BJS should develop an approach to measure the ex- periences of individuals through the criminal justice system on a prospective, longitudinal basis, beginning as early as practicable in the process (arrest) and ending with their eventual exit (ranging from early dismissal of charge through completion of sentence). Recommendation 3.5: BJS should develop an approach to measure the vic- timization experiences of individuals on a prospective, longitudinal basis, beginning from a focal victimization and following the victim forward in time measuring subsequent victimizations and possible consequences of vic- timization. The NCVS may be used to recruit respondents to a panel survey of crime victims. Recommendation 3.6: BJS should develop a panel survey of people under correctional supervision to understand how individuals move between insti- tutional and community settings, and to understand the social contexts of correctional supervision. Recommendation 3.7: To be useful, a BJS strategic plan must articulate a blueprint of interrelated data collection and product activities, including both current and potentially new data products. This blueprint would be used to evaluate new opportunities. Recommendation 3.8: BJS should make supplements a regular feature of the NCVS. Procedures should be developed for soliciting ideas for supplements from outside BJS and for evaluating these supplements for inclusion in the survey. Finding 3.2: The multitude of scattershot “census” studies of specific law en- forcement agency types (e.g., campus law enforcement, medical examiners, training academies) detracts from the appearance of a coherent measurement program in the area of law enforcement. Instead, the impression left is that these “censuses” are sporadic inventories or catalogs of particular agency types with no obvious internal consistency. Recommendation 3.9: To maximize both utility and timeliness of informa- tion, the LEMAS survey should be conducted as core-supplement design in the context of a continuous data collection. 

328 JUSTICE STATISTICS Recommendation 3.10: To improve the utility of censuses of law enforce- ment agencies, BJS should develop an integrated conceptual plan for their periodicity, publish a 5-year schedule of their publication, and integrate their measurement into the LEMAS as supplements. Recommendation 3.11: The NCVS (and its supplements) should be more effectively used as a tool for studying law enforcement, both in terms of the types of crime that are reported (and not reported) to police and the action that results from the reporting of a crime (e.g., the Police-Public Contact Survey). Finding 3.3: BJS’s current approach to data collection in adjudication lacks an effective basis in sampling. Recommendation 3.12: As court records become more accessible through computerized case management systems, BJS should implement more rigor- ous methods of probability sampling in its adjudication series. Recommendation 3.13: To inform future revisions to its adjudication port- folio and to more efficiently acquire and work with court data in the future (including longitudinal analysis), BJS should develop a research program to build representative samples of courts and to assess strategies for collection of case records. Recommendation 3.14: BJS should mount a feasibility study of the flow of individuals between correctional supervision and community settings. Re- peated interviews of samples of about-to-be-released prisoners that track their successes and failures in reintegrating with the community would en- hance understanding of this critical policy issue. Finding 4.1: BJS’s state Statistical Analysis Center (SAC) program has cul- tivated a strong federal-state relationship, relative to other federal statisti- cal agencies. Development of the SAC network—which provides points of contact across the justice system to facilitate research on individual data se- ries, dissemination of BJS information, and coordination of activities—has involved forging unique relationships adapted to state environments (for in- stance, whether the SAC is part of a state law enforcement department or is housed at a university). Recommendation 4.1: Through its Statistical Analysis Center and State Jus- tice Statistics programs, BJS should continue to develop its ties with the states, and more fully exploit the potential for using states as partners in data collections. 

APPENDIX A 329 Recommendation 4.2: Developments toward longitudinal and small-area measurement systems should involve state partners who are active in data collection and knowledgeable about state justice systems. Finding 4.2: The National Criminal History Improvement Program (NCHIP) is a grantmaking program but not directly a statistical collec- tion, even though it is administered by BJS. However, improved criminal history records are important for the prospects of longitudinal analysis of the criminal justice system. Analysis of the National Instant Background Check System serves as one approach to provide the data necessary to evaluate national policy on regulation of firearms purchases. Recommendation 4.3: BJS should actively utilize the NCHIP program to improve criminal history records necessary for longitudinal studies of crime. Finding 4.3: A full-fledged NIBRS would be a source of basic information on police responses to public complaints (911 calls), including whether or not a case is “cleared” by police through an arrest. Recommendation 4.4: To improve the timeliness of crime statistics, BJS should explore the development of a crime reporting system based on a probability sample of police administrative records. The goals of such a sys- tem would be national representativeness, high response, high data quality, timeliness and flexibility in terms of crime classification and analysis, and national statistics for the monitoring of crime trends. Finding 5.1: Under the terms of the Prison Rape Elimination Act of 2003, BJS was required to release the identity of selected responding institutions (i.e., facilities with the highest and lowest rates of sexual violence against inmates) for later regulatory action as part of a statistical program. Recommendation 5.1: Congress and the Department of Justice should not require, and BJS should not provide, individually identified data in support of regulatory functions that compromise the independence of BJS or require BJS to violate any of the principles of a federal statistical agency. Finding 5.2: The appearance of political interference in release of statistical information undermines public trust in that information and in the entire agency. Recommendation 5.2: The Department of Justice review of any BJS statis- tical product and related communications should not require changes to the content, the release schedule, or the mode of dissemination planned by BJS. 

330 JUSTICE STATISTICS Finding 5.3: The placement of BJS within the Office of Justice Programs has harmed the agency’s ability to innovate in data collections and expand the efficiency of achieving its statistical mission. It suffers from a zero-sum game in competition with programs of direct financial benefit to states and localities. Recommendation 5.3: BJS should be administratively moved out of the Of- fice of Justice Programs, reporting to the attorney general or deputy attorney general. Finding 5.4: Under current law, the director of the Bureau of Justice Statis- tics serves at the pleasure of the president; the director is nominated to an unspecified term by the president, with the advice and consent of the Senate (42 USC § 3732(b)). Recommendation 5.4: Congress and the administration should make the BJS director a fixed-term presidential appointee with the advice and consent of the Senate. To insulate the BJS director from political interference, the term of service should be no less than 4 years. Recommendation 5.5: The BJS director needs to reach out to other agen- cies within DOJ, forming partnerships to propose initiatives for information collection that are relevant to policy needs. Recommendation 5.6: The Department of Justice should build provisions for BJS collection of data and statistical information into its program initia- tives aimed at crime reduction. These are not intended as program evalua- tion funds, but rather as funds for the basic monitoring and assessment of the phenomena targeted by the initiative. Finding 5.5: BJS enjoys high credibility but often is critiqued for missing fine-grained data by geography or time. Recommendation 5.7: To effectively get input on contemporaneous topics of interest, BJS should regularly convene ad hoc stakeholder workshops to suggest areas of immediate data needs. Recommendation 5.8: BJS should establish an Advisory Group under the Federal Advisory Committee Act to provide guidance to BJS on the addition of new data collection efforts and the modification of current ones in light of needs identified by the group. Membership in the group should include, at a minimum, leaders and practitioners from each of the major subject matters covered by BJS data, as well as those with statistical and other types of aca- demic expertise in these subject matters. The members of the group should be selected by the BJS director and the group should provide the director with at least two reports each year that contain its recommendations. 

APPENDIX A 331 Recommendation 5.9: DOJ should take steps to ensure that congressional staff are aware of BJS data that could be used in developing legislation; DOJ and BJS should learn from congressional staff how their data are needed to inform/support legislation so that they can improve the utility of their current data and so that they can develop new data sets that could enhance policy development. Recommendation 5.10: To improve the utility and accuracy of the National Corrections Reporting Program (NCRP), BJS should work with correctional agencies to develop their own internal records to promote consistent data collections and expand coverage beyond the 41 states covered in the most recent NCRP. Finding 5.6: A recurring criticism of BJS data products is that their quality is highly valued but that they are not sufficiently timely to meet user needs. All statistical agencies are attempting to grapple with new data collection designs that offer more timely estimates. Recommendation 5.11: BJS should evaluate each of its data programs to inquire whether more timely estimates might be obtained by (a) making discrete data collections into more continuous operations and (b) issuing preliminary estimates, to be followed by final estimates. Finding 5.7: The credibility of BJS’s products is a function of its quality review procedures. Recommendation 5.12: BJS should articulate why some data collections are housed on external websites and describe the process by which links to external websites are allowed. BJS should articulate and justify the use of its insignia on external websites. Finding 5.9: The active investigation of new ways of measuring and under- standing crime and criminal justice issues is a critical responsibility of BJS. The agency has lacked the resources needed to fully meet this responsibility and, for some issues, has fallen behind in developing such innovations. Finding 5.10: BJS has lacked the resources to sufficiently produce new topi- cal reports with data it currently gathers. It also lacks the resources and staff to routinely conduct methodological analyses of changes in the quality of its existing data series and to fully document those issues. Instead, the BJS production portfolio primarily is limited to a routine set of annual, biannual, and periodic reports and for some topics, the posting of updated data points in online spreadsheets. 

332 JUSTICE STATISTICS Recommendation 5.13: BJS should carefully study changes in the NCVS survey design before implementing them. Recommendation 5.14: BJS should study the measurement of emerging or hard-to-reach groups and should develop more appropriate approaches to sampling and measurement of these populations. Recommendation 5.15: BJS must improve the technical skills of its staff, including mathematical statisticians, computer scientists, survey methodol- ogists, and criminologists. Finding 6.1: Relatively little is understood about how BJS data are used in developing policy and could be used in improving policy development. Recommendation 6.1: BJS must ensure that the nation has quality annual estimates of levels and changes in criminal victimization. Recommendation 6.2: Congress and the administration should ensure that BJS has a budget that is adequate to field a survey that satisfies the goal in Recommendation 6.1. Recommendation 6.3: More information about the needs of victims is es- sential to the compensation and assistance goals of the Office of Victims of Crime. Congress should allow additional funding for the collection and im- provement of victimization data to be obtained from funds obtained through the Victims of Crime Act. Recommendation 6.4: Additional resources made available for the NCVS should be used not only to increase the reliability of annual estimates but also to supplement the survey in ways that increase our understanding of criminal victimization.