The growing debate about policing in America arises from concern about horrific but extraordinary acts of police violence. These incidents and the clear racial disparities in criminal justice contact raise important questions about the ordinary practice of policing. Should police stop suspicious individuals and frisk them for weapons? Should departments use statistical techniques to predict crime and make decisions about where to deploy officers? Evaluating police practices requires measuring their benefits and their costs. Do police practices reduce crime? How do they affect communities? How do those effects vary within and among communities? However, community groups and municipal leaders outside law enforcement currently lack data and tools to measure the impact of policing strategies. Community stakeholders – including city governments, community groups, and non-governmental organizations – need rigorous tools to independently evaluate the costs and benefits of various policing strategies.
To assess the benefits and costs of policing, we need to know how police actions affect patterns of crime. Both components – police actions and crime – are hard to measure. Most crime is secret and police practices influence variation in recording of crimes. When departments hire more officers, or when they deploy more officers to certain neighborhoods, recorded crime may increase even if actual crime does not change. Additionally, police knowledge about crime is the result of reporting by civilians who trust the police. Many victims are reluctant to report crime to police because they think the police will be unable to help them, because they worry that police may suspect them of being criminals themselves, or because they fear retaliation from perpetrators or neighbors.
Our team specializes in collecting and analyzing data on events that are hard to measure. In the last year, the Human Rights Data Analysis Group has begun studying issues in U.S. police practice, focusing on three topics: homicides by police, predictive policing, cost-benefit analysis of policing. We have already created multiple new analyses of available data on crime and policing, assessing the accuracy of the number of killings by police and the effects of Predictive Policing. We propose to build on our work to create a scalable, sustainable, community-driven, technically rigorous assessment of the costs and benefits of various policing strategies.
Homicides by Police
In 2015, Kristian Lum (project lead) and Patrick Ball re-analyzed work published by the U.S. Department of Justice’s Bureau of Justice Statistics (BJS) to show that the BJS estimates substantially understate the number of homicides by police. The HRDAG analysis was cited in 538 Politics by Carl Bialik and in Boing Boing by Cory Doctorow.
A year later, in March 2016, Granta published Patrick Ball’s “Violence in Blue,” which explained the analysis in non-technical detail. Among the findings is that one-third of all Americans killed by strangers are killed by police. Another oft-cited statistic generated by HRDAG: serial killers, mass shooters, and “terrorists” combined are only approximately one-fifth the quantity of police homicides.
HRDAG has partnered with a team of analysts to illustrate for a broad audience how predictive policing increases racial disparities in policing.
Kristian and William Isaac have collaborated on a statistical model that demonstrates how bias works in predictive policing. They reimplemented the algorithm used by one of the more popular vendors who sell this technology to police departments. The analysis shows how the predictive models reinforce existing police practices because they are based on databases of crimes known to police.
As William said at a recent Stanford Law symposium, predictive policing tells us about patterns of police records, not patterns of crime. And as Patrick said recently at a talk at the Data and Society Research Institute, technology and massive samples tend to amplify, not ameliorate, selection bias.
The combination of these effects creates a technologically obscured tautology: the model predicts approximately where crimes were previously known. The model cannot predict patterns of crime that are different from the patterns already known to police. Because only half of crimes are known to police, the predicted patterns are likely to be substantially different from the true patterns. Predictive policing algorithms add no new information to what police already know. Indeed, these algorithms may amplify racially disparate policing outcomes.
Cost-benefit Analysis of Policing
Kristian is leading the technical work on our first sustained domestic project, looking at patterns of crime: crimes committed by civilians and crimes committed by police; and crimes known to police, and those unknown to police.
Evaluating police practices requires measuring their benefits and their costs. In doing this evaluation, we might ask, “Do police practices reduce crime?” But the larger and in our opinion more important question is “How do police practices affect communities? Does policing create communities where people feel safe and trust the police?” The main barrier to this analysis is a lack of reliable data. That is, most data about crime come from police records, which cover only about half of all crimes.
HRDAG is exploring new data sources, communication tools, and statistical approaches to create integrated measures to better understand the relationship between policing, crime, and civic engagement. This work will engage municipal leaders and city staff, community groups, and media.
“Violence in Blue” (Patrick Ball, Granta, March 2016).
“Estimating Undocumented Homicides with Two Lists and List Dependence” (Kristian Lum and Patrick Ball, April 2015).
“How many police homicides in the US? A reconsideration“ (Kristian Lum and Patrick Ball, April 2015).
“BJS Report on Arrest-Related Deaths: True Number Likely Much Greater“ (Kristian Lum and Patrick Ball, March 2015).
Patrick Ball at Data & Society Research Institute, April 2016.
If you’d like to support HRDAG in this project, please consider making a donation via Generosity/indiegogo.