Quantifying Police Misconduct in Louisiana
For Ayyub Ibrahim, data collection might involve using the US postal system, or even a long drive and a portable scanner. As a research associate at Innocence Project New Orleans, an organization dedicated to building an equitable, unbiased, and just legal system, Ayyub has been working to build the Louisiana Law Enforcement Accountability Database, known as LLEAD, which launched in October, 2022. The database is a public tool for transparency, consolidating information about police misconduct and use of force from more than 500 law enforcement agencies in the state of Louisiana. 
“Most of the documents produced exist only on paper. They’re not digitized,” says Ayyub. “The data’s out there for anyone who wants it, but there’s a massive hurdle in processing it.”
In Louisiana, different jurisdictions have different ways of hearing appeals for police disciplinary actions. Some are held within town council meetings, which are documented in the meeting’s resulting minutes. But the critical one or two paragraphs about the police officers and their cases may be buried among hundreds of other paragraphs about items on the city council agendas, from management details to budget discussions to economic development plans. The meeting minutes come in a large variety of document and file types, as well.
“If we hadn’t found HRDAG, a huge portion of our work wouldn’t exist.”
— Ayyub Ibrahim, Innocence Project New Orleans
In other jurisdictions, such as Baton Rouge Parish, a police civil service board hears the appeals. Regardless of where the hearings take place, many of the records of the hearings are not digitized. Some parishes respond to Ayyub’s request for the records by scanning and sending them—but other parishes don’t have that capacity. In the future, to obtain those records he might have to drive to the parish, locate the records in a bin or drawer, and scan them himself. Either way, he receives pushback from some disgruntled agencies, and some ignore him or cut him off.
“Some agencies take upwards of two years to send a two-page document,” he says. “Half of the time, the problem is that they don’t want to send the data. The other half of the time, they just don’t have the capacity to do a search for the records.”
Two years ago, in December 2020, the Innocence Project New Orleans and its partner, Public Data Works (PDW), contacted HRDAG about collaborating on LLEAD. HRDAG’s data scientist, Tarak Shah, contributes to the project by helping to classify, filter, extract, and standardize the records so that they can be useful in the database.
“I met Tarak through folks at Public Data Works, because of a similar situation at the Invisible Institute, where they’re dealing with massive troves of data,” says Ayyub. “This is a great partnership in that I’ve been able to leverage Tarak’s genius to apply the same principles on a smaller scale. I’ve begun learning how to process some of these documents in a similar fashion, and we’re building machine learning models.”
When Tarak came on board, Public Data Works and The Innocence Project New Orleans (IPNO) had a good plan for dealing with structured data, such as digitized spreadsheets, which originate with police agencies and are not terribly informative. But IPNO and PDW did not have a great plan for what to do with unstructured data—for example, letters, forms, and meeting minutes from town council meetings and civil service boards—which tend to be much richer in detail. Tarak works on code to process the unstructured data into database fields that empower searches and analyses, and he meets weekly with the team to review project updates, go over new data sources and analytical challenges, and to brainstorm analyses that can be done using the database.
Like Invisible Institute, IPNO has a team of volunteers who help process documents. Most of them are staff, law students, or legal interns. They spend a lot of time labeling documents in an effort to see if models can be created to process these kinds of documents for this project. “The jury is still out on that,” says Tarak.
“Machine learning is already working on the stuff we’ve extracted,” says Tarak. “Machine learning has been successful to the extent that several steps in the import pipeline are executed by a machine learning tool.”
Tarak hopes that the relationship with IPNO can be ongoing and evolving. Together, they create wish lists of the types of analyses they’d like to do. With the data that has been extracted so far, Ayyub is interested in analyses such as use-of-force patterns and trends in migration while under investigation. Tarak supports that, while focusing on some of the analytical challenges that accompany complex data like IPNO’s. Data generated across multiple jurisdictions and accountability mechanisms each have their own procedures and policies, so patterns and trends detected in this type of data could represent important substantive findings, or they could be artifacts of the way different agencies document their activities. Between the richness of the LLEAD, which incorporates a variety of sources of information to fill in any gaps, and IPNO’s existing base of expertise, he’s confident they’ll be able to address these challenges and produce important analyses.
“We want to be able to say useful, true things about different types of police conduct, and we’re also aware that we have multiple sources of data available to us, which we only kind of understand in a conceptual way,” he says. “We have a small representation of the negative interactions people have with police.”
The evolution currently underway is figuring out how to use machine learning to process new documents as they come in. So far, the project has only received documents from half the parishes in Louisiana. It’s likely that as the smaller parishes give access to their documents, the team will find the formats to vary and the process of extracting meaningful data to be more difficult. The team would love to build machine learning models that can process documents from every jurisdiction regardless of format.
“If we hadn’t found HRDAG, a huge portion of our work wouldn’t exist,” says Ayyub.
IPNO has use cases in which officers known to have engaged in misconduct have been tied to people wrongfully convicted and released from prison. He hopes to be able to use the data to tie newly discovered bad actors to other officers we already know about.
“What I’d really like is for these data to be useful in a wrongful conviction case,” says Ayyub. “If we can figure out how to make sense out of all this data, we might be able to get an innocent person released from prison. And because of the scale of these data, it could be many more.”
Image: David Peters, 2023.
HRDAG partner stories:
Quantifying Police Misconduct in Louisiana
Scraping for Pattern: Protecting Immigrant Rights in Washington State
Police Violence in Puerto Rico: Flooded with Data
Building Capacity In Colombia: Truth And Reconciliation
Police Accountability In Chicago: From Data Dump To Usable Data
Protecting the Privacy of Whistle-Blowers: The Staten Island Files