– F –

Cautionary Tales from International Experience: Police-Report Crime Statistics in the United Kingdom

As described in Section 3.1, the United States represents an extreme case of decentralized authority both in terms of law enforcement and federal statistics—a combination with strong ramifications for collecting “national crime statistics.” It is also a combination that sharply limits the degree to which the United States can directly learn from the experience of other countries in reengineering their crime statistics infrastructures. The circumstances are typically so varied in comparing the U.S. to other nations that the comparison is at best illustrative. Yet this report would be incomplete without mention of at least one key illustrative experience, simply as a check on overconfidence. We argue in this report for completely rethinking the way the United States measures crime, including adoption of a new classification of crime, and using that classification as blueprint for implementing and coordinating a new crime statistics system according to the principles and sensitivities of a statistical agency. We believe we have been appropriately open about the difficulty of the task. The story of another country that made these kinds of changes and still experienced a major crisis of confidence in its crime statistics is useful to review, strictly illustrative as it may be.

Similar to the United States, crime statistics in the United Kingdom historically drew from two principal sources: the British Crime Survey (BCS, akin to the U.S. National Crime Victimization Survey) and the collation of police-report data from about 45 territorial and special-purpose police agencies. However, for years, full authority for both crime data streams was vested in the same government ministry—the Home Office, responsible for administration of the police forces themselves. Over the years, several reviews of UK crime statistics were commissioned due to perceived diminishing of trust in the resulting data, much more so for the police-report (offense count) statistics than the BCS but still pervasive. As it turned out, these concerns over erosion of trust coincided with a major structural change in management of UK official statistics. The United Kingdom Statistics Authority was created in April 2008 pursuant to law enacted the previous year, and it was formed as an entity independent of any government ministry (accountable directly to Parliament). In addition to taking on oversight of the nation’s statistical office (the Office for National Statistics, ONS), the Authority was also directed to certify that statistical series and products adhered to its Code of Practice for Official Statistics—and so to certify which statistical products would bear the label and designation as official “National Statistics” (and which would not).

As a major sector of official statistics, crime statistics quickly drew the Authority’s attention. An initial report in 2010 summarized user input and previous reform suggestions—and ultimately sided with maintaining the organizational status quo (United Kingdom Statistics Authority, 2010:4–5). Specifically, the review noted that data collection for the BCS was already contracted out to a private survey organization, making transfer of authority from the Home Office to ONS relatively easy, but observed that it was unclear whether transfer of the police-report data to ONS would significantly improve trust in those data. Likewise, it concluded that “practical considerations”—the existing ties between the national Home Office and the territorial and special-purpose police agencies—advised against an administrative transfer of police-report data. An initial assessment of UK crime statistics effectively granted them conditional designation as “National Statistics” subject to completion of five enhancements, chief among them the provision of “more information about the quality of the police recorded crime statistics, including the consistency with which crimes are recorded and classified” (United Kingdom Statistics Authority, 2011:3).

However, by December 2010, the Home Office itself recognized the need for a more definitive direction and drew up terms of reference for a fuller Statistics Authority assessment. In addition to comprehensive review of the “gaps, discrepancies and discontinuities” in both the BCS and police-report data, the review was explicitly directed by the Home Office to “consider which body outside the Home Office is best placed to have future formal responsibility for the publication of crime statistics”—detaching the publication function

from the compilation of the data—“and what should be the appropriate underpinning data collection arrangements.” The National Statistician’s review (United Kingdom Government Statistical Service, 2011) satisfied this request. It recommended that full authority for the BCS and that publication authority for all crime statistics be vested in the ONS and that collection/validation be formally detached from publication for police-report data. Effective April 2012, the primary recommendations were adopted: Authority for the renamed BCS, now the Crime Survey for England and Wales (CSEW), was transferred in full to ONS, while authority for police-report data was formally divided in function. The Home Office remained responsible for collecting police-report data, but was to channel compiled data to ONS—which was now a more public “owner” of the police-report data, analyzing and publishing them on a quarterly cycle. This elevated role of ONS in UK crime statistics was also the occasion for a reexamination of what constitutes a reportable offense in the data as well as the UK’s intricate and voluminous set of “counting rules” for crimes in police-report data.

The National Statistician’s review also called for assessment and strengthening of data quality and audit routines for police-report data, the regularly scheduled program of audits having been cancelled since 2007 due to resource constraints. Her Majesty’s Inspectorate of Constabulary (2012) performed an independent review of recording practices by territorial police, updating an analysis performed in 2000; though it found improvement in the consistency of recording/reporting offenses, the audit suggested wider-than-desired variability in recording offenses (or “no criming”/not recording some offense types, such as sexual assault). Select committees of the House of Commons also heard allegations of misreporting, and particularly the “no criming” of incidents in order to meet performance targets and show reductions in crime levels. In January 2013, ONS researchers published an analysis of crime trends contrasting police-report and BCS/CSEW data, which suggested that the level of inconsistency of accurate police reporting might be increasing with time (Flatley and Bradley, 2013).

The transfer in authority for crime statistics publication and the ongoing concern over potential misreporting led the Statistics Authority to initiate another assessment of UK police-report statistics. This time, in January 2014, the consequences were severe. Specifically, the Statistics Authority stripped police-recorded crime data of “National Statistics” designation, citing “accumulating evidence [that] the underlying data on crimes recorded by the police may not be reliable” and that ONS (as publisher of the data) “does not have sufficient information about the quality of recorded crime data [to] ensure that users are made fully aware of the limitations of the recorded crime statistics” (United Kingdom Statistics Authority, 2014a:2). The Authority listed 16 specific requirements that ONS (working with the Home Office and other authorities) would have to satisfy to reacquire the National Statistics label. By

and large, these requirements center around communication of uncertainty and data quality and the capacity of the resulting data to satisfy data user needs. For instance, the assessment said to “provide more information to users about [how] to interpret changes in the published crime statistics from one reference period to another,” “explain more fully how all the administrative data sources are used to produce statistics about crime,” and (mindful of ongoing audit by the Inspectorate of Constabulary) consider the general question of “whether statistics based on police recorded crime data can be produced to a level of quality that meets users’ needs” (United Kingdom Statistics Authority, 2014a:4). Another requirement is to make public information “about police forces’ progress in switching to use of the Data Hub”—a Home Office–created system by which local departments transfer their crime data directly to the Home Office for compilation—“and the reason why this has taken so long” (United Kingdom Statistics Authority, 2014a:5).

Subsequent to the stripping of National Statistics status, a new Crime Statistics Advisory Committee was empaneled to assist in the process of working through the requirements. The House of Commons’ Public Administration Select Committee (2014:3, 22) suggested approval of the Authority’s conclusions but concluded that the Authority, the ONS, and the Home Office had all been “far too passive in the face of concerns raised about [Police Recorded Crime (PRC)]; they have repeatedly missed opportunities to ensure the integrity and quality of PRC data.” In particular, the committee chided the Authority for having initially designated police-report data as National Statistics in 2011, and found it “deplorable that ONS can have overseen the production of crime statistics [with] what appears to have been very limited knowledge of the ‘quality assurance’ steps” performed on the data. Further, the committee recommended “the re-instatement of annual audits of crime recording practices,” noting that their suspension in 2007 was a “mistake,” and “deprecate[d] the use of targets” for reduction in crime tallies by local departments “in the strongest possible terms” as a detriment to data integrity.

Not long after the Statistics Authority stripped police-report data for England and Wales of its National Statistics designation, it did the same for police-report data for Scotland (United Kingdom Statistics Authority, 2014b). Two years later, the Scottish government was notified that its Recorded Crime in Scotland data were reinstated as National Statistics (Humpherson, 2016). As of mid-2017, however, the police-report data for England and Wales continue to be decertified as National Statistics.

One could look at the division of labor established in 2012 in the UK—substituting “FBI” for “Home Office” and “BJS” for “ONS”—and attempt some inference concerning our delineation of coordination and governance rules for crime statistics in Chapter 3. This is not at all what we intend by telling this story, and we do not support such easy inference; much like blithely comparing community crime levels absent any context, there are sufficient

differences between the United States and United Kingdom crime measurement circumstances that simple comparison is misleading. But some broader takeaway lessons are justified, from the general nature of the UK experience and from the substance of the UK Statistics Authority’s rationale for decertification of police-report data. To wit, the UK experience and the example of Ireland (discussed in Section 4.1.4 of Report 1)¹ support the conclusion that adoption of a rigorous classification and implementation of new systems are not panaceas on their own. Indeed, they are largely for naught without adequate attention to the quality of the underlying data entering the system or to whether the resulting data products are understood by data users/stakeholders and meet their needs.

___________________

¹ In brief: Ireland was one of the first countries (in recent years) to fully revisit and revise its classification of crime; its result was an important starting point for the United Nations Office on Drugs and Crime (UNODC) International Classification of Crime for Statistical Purpose (ICCS) on which our proposed classification is patterned. Implementation of the new classification was concomitant with transfer of responsibility for crime statistics publication from the national police (La Garda Síochána) to the Irish Central Statistical Office (CSO). However, a review found “systemic failures in recording practices” in the Police Using Leading Systems Effectively (PULSE) system—basically, Ireland’s UCR database, the recording system used to compile inputs from local police offices (Garda Síochána Inspectorate, 2014:9–11). Misclassification results were sufficiently large that the CSO formally suspended the publication of Irish police-report crime statistics for 6 months, only resuming their publication after completing their own review (Central Statistics Office, 2015) and instituting remedies and data improvements.

This page intentionally left blank.