Making Missing Data Visible in Colombia
In 2016, when the Colombian government and left-wing rebels signed a ceasefire, the country came closer to peace than it had been in 50 years. But it was a confusing moment, says Valentina Rozo Ángel, an analyst and consultant at the Human Rights Data Analysis Group (HRDAG).
Over the 50 years of the armed conflict, right-wing government forces, right-wing paramilitaries, and left-wing guerilla organizations such as FARC (Revolutionary Armed Forces of Colombia) had staked out territories that they more or less controlled. But with the 2016 ceasefire, the main armed group demobilized, and this left a vacuum. With some regions “up for grabs,” new groups sought to gain influence, for example, community leaders who organized against oil companies or hydroelectric corporations, to oust them or prevent them from setting up operations in their regions. Some of these activists, whom Rozo refers to as “social movement leaders,” were murdered, and their deaths were reported on the news.
Rozo had been working as a researcher at Dejusticia, a think tank and advocacy organization that advances human rights and works on constitutional matters. In her role there, she started tracking the deaths of social movement leaders, or activists, that she saw reported on the news. She created a database that contained information about all the leaders killed since the 2016 peace agreement. And then she met Patrick Ball at a conference about transitional justice.
Ball is the director of research at the Human Rights Data Analysis Group (HRDAG) and has been urging the human rights community to pay more attention to missing data for most of his 30-year career as a data scientist. As a result of his three decades of work within truth commissions, non-governmental organizations, tribunals and United Nations missions, he has a keen sense of the ways in which human rights workers sometimes unintentionally overlook victims who are less visible, and thus not easily counted.
When Rozo and Ball met, he asked her if she’d thought about the fact that the social movement leaders accounted for in her database were those who had enough visibility to make the news.
“It was the first time someone had talked to me about missing data. That struck me,” says Rozo. “I was excluding the most vulnerable victims from the database—but what could I do about it?”
Dejusticia and HRDAG partnered, and Rozo learned about multiple systems estimation, also known as MSE. An economist by training, it was the first time she had used MSE to estimate the broader universe of activists who had been killed. (An estimation includes homicides of people who are not included in an enumerated list.) In 2018, Dejusticia and HRDAG jointly published a report about the underreporting of killings of social movement leaders in Colombia, and updated it in 2019. They have also co-authored a brief that addresses the importance of missing data and the trickiness of estimating it in a transitional justice process.
Rozo joined the Colombian Truth Commission in 2019 as an analytics lead, leading a team of researchers who were trying to answer questions related to the conflict using quantitative methods. She became part of what she calls an ecosystem of experts; HRDAG was a critical part of this ecosystem.
Rozo’s work at the Commission was interdisciplinary, and therefore challenging. Her colleagues included social scientists, doctors, activists, political scientists—“Quantitative work is always difficult to share because not everyone has the same foundation,” she says. “What might seem obvious to people who work with data is not obvious to many social scientists.”
One of her challenges was helping the diverse group of experts understand enough about statistics that they would trust the data scientists’ expertise. She also had to manage expectations about what data analysis can do, communicating findings clearly, but also with rigor.
“Some people believe data is magic,” she says. “They want you to run an analysis showing that A causes B, but we have to say ‘No, A is related to B.’”
HRDAG’s data analysis on the project was advanced, and included “statistical uncertainty,” which is standard in the field of data science. But, as Rozo says, “People don’t like uncertainty at all. In politics that means negotiating, so we sometimes only rely on what we can count [instead of estimate]. It’s difficult sometimes to include estimates, because of the uncertainty.”
As Rozo points out, this decades-old conflict is referred to as the “armed conflict,” although some people refer to it as a civil war. It’s part of Colombia’s identity, but she feels that it’s been normalized over many lifetimes. Most Colombians are not aware of its magnitude.
“I would like for all of us to learn a little bit about it,” she says. “I’ve entered that world, and I can’t leave it now.”
This story is part of HRDAG’s long-term work in Colombia, including our most recent collaboration, the JEP-CEV-HRDAG data integration and statistical estimation project.