Alejandro Valencia Villa is a Former Commissioner of the Colombian Truth Commission. (Letter in English, and letter in Spanish.)
Introduction
One of the most obvious and most difficult questions to answer when analyzing an armed conflict is determining the number of victims. In a conflict like Colombia’s, prolonged and with complex characteristics due to the different nature of the armed actors and because they committed a great variety and quantity of human rights violations and breaches of humanitarian law, the challenge is even greater. As if this were not enough, Colombia also had a large number of records of these violations and infract...
The need to establish the truth around events is central to goals of transitional justice, particularly securing accountability, establishing legitimate and effective justice mechanisms, and laying the foundations for a peaceful society.
Documentation to provide verifiable and widely accepted accounts of such events is a critical component of establishing this truth, or the multiple truths that may exist for a population. It is difficult to overstate the important role of documentation in transitional justice efforts. If some of what follows sounds familiar, echoing points of previous posts, it is no coincidence. Documentation, in a word, is the ...
James Johndrow, Patrick Ball, Maria Gargiulo, and Kristian Lum. (2020). Estimating the Number of SARS-CoV-2 Infections and the Impact of Mitigation Policies in the United States. Harvard Data Science Review. 24 November, 2020. © The Authors, 2020, CC BY 4.0. https://doi.org/10.1162/99608f92.7679a1ed
James Johndrow, Patrick Ball, Maria Gargiulo, and Kristian Lum. (2020). Estimating the Number of SARS-CoV-2 Infections and the Impact of Mitigation Policies in the United States. Harvard Data Science Review. 24 November, 2020. © The Authors, 2020, CC BY 4.0. https://doi.org/10.1162/99608f92.7679a1ed
Help us hold human rights violators accountable!
Some of the earliest large-scale human rights information projects happened in El Salvador. One was developed by Patrick Ball at the Salvadoran non-governmental Human Rights Commission, also known as Comision de Derechos Humanos de El Salvador (CDHES-ng). Between 1977 and 1990, more than 9,000 testimonies were taken in an effort to document the nature and scope of the bloody conflict between the army and the Farabundo Marti National Liberation Front (FMLN). Starting in 1991, Patrick worked with CDHES staff to organize the information in an early computer database. They linked reported human rights violations with the career structures of individual ...
<<Previous post: Collection, Cleaning, and Canonicalization of Data
Q8. What do you mean by "overlap," and why are overlaps important?
Q9. [In depth] Why is automated matching so important, and what process do you use to match records?
Q8. What do you mean by "overlap," and why are overlaps important?
MSE estimates the total number of violations by comparing the size of the overlap(s) between lists of human rights violations to the sizes of the lists themselves. By "overlap," we mean the set of incidents, such as deaths, that appear on more than one list of human rights violations. Accurately and efficiently identifying overlaps between ...
Patrick Ball, Ethan Hee-Seok Shin and Hyerin Yang (2018). There may have been 14 undocumented Korean “comfort women” in Palembang, Indonesia. Human Rights Data Analysis Group. 26 December 2018.© 2018 HRDAG. Creative Commons.
Patrick Ball, Ethan Hee-Seok Shin and Hyerin Yang (2018). There may have been 14 undocumented Korean “comfort women” in Palembang, Indonesia. Human Rights Data Analysis Group. 26 December 2018.© 2018 HRDAG. Creative Commons.
Bailey Passmore + Larry Barrett. 2025. Shots fired: Can technology really keep us safe from gunfire? Significance, Volume 22, Issue 4, July 2025, Pages 34–37. 27 May 2025. © Royal Statistical Society 2025. https://doi.org/10.1093/jrssig/qmaf042
An expensive American gunshot detection system claims it's necessary because humans don't always call the police to report gunfire. But opponents say it's fatally flawed. To investigate, Bailey Passmore and Larry Barrett analysed data on emergencies within the city of Chicago. Read more.
Bailey Passmore + Larry Barrett. 2025. Shots fired: Can technology really keep us safe from gunfire? Significance, Volume 22, Issue 4, July 2025, Pages 34–37. 27 May 2025. © Royal Statistical Society 2025. https://doi.org/10.1093/jrssig/qmaf042
An expensive American gunshot detection system claims it’s necessary because humans don’t always call the police to report gunfire. But opponents say it’s fatally flawed. To investigate, Bailey Passmore and Larry Barrett analysed data on emergencies within the city of Chicago. Read more.
Bing Wang has joined HRDAG as a Visiting Data Science Student until the summer of 2020.
I will use the skills and culture I learned from HRDAG’s team to understand how the conflict has affected the people in my country.
Patrick Ball is kicking himself for a decision he made almost 25 years ago. “I was clever, but I wasn’t smart,” he says ruefully, as he considers the labyrinth of tables and ASCII-encoded keystrings he used to design a database of human rights violations for the pioneering Salvadoran non-governmental Human Rights Commission (CDHES). Now I’m sitting in his office in San Francisco’s Mission District watching over his shoulder, and trying to keep up, as he bangs out code to decipher the priceless data contained in these old files. Created in 1991 and 1992, during the last days of El Salvador’s internal armed conflict, the files detail ...
We’re very happy to announce that our executive director, Patrick Ball, has been elected as a Fellow of the American Statistical Association (ASA), as announced by ASA President Nathaniel Schenker. Patrick is one of 63 new ASA Fellows to be honored this year in a ceremony at the Joint Statistical Meetings, which will take place this August 5 in Boston, Massachusetts. (more…)
Patrick Ball, director of research at the Human Rights Data Analysis Group, and a statistician who has conducted similar estimates of violent deaths in conflicts in other regions, said the study was strong and well reasoned. But he cautioned that the authors may have underestimated the amount of uncertainty caused by the ongoing conflict.
The authors used different variations of mathematical models in their calculations, but Dr. Ball said that rather than presenting a single figure — 64,260 deaths — as the estimate, it may have been more appropriate to present the number of deaths as a range from 47,457 to 88,332 deaths, a span that encompasses all of the estimates produced by modeling the overlap among the three lists.
“It’s really hard to do this kind of thing in the middle of a conflict,” Dr. Ball said. “It takes time, and it takes access. I think you could say the range is larger, and that would be plausible.”
I look at the beach and then at the table surrounded by nerds, deep in thought and conversation about Dirichlet priors, matching algorithms, and armed conflicts. This peculiar (in the best way) environment catalyzes a moment of reflection: how did I get here?
Four years ago, as a second-year statistics PhD student, I watched "Guatemala: The Secret Files" on PBS Frontline World. I listened to stories of family members who disappeared without answers or justice. Then the story shifted to the work being done by archivists and data experts at Guatemala's Historic Archive of the National Police. The scientists' pursuit of the truth energized me. I ...
In 2018, HRDAG collaborated on work in Guatemala, US criminal justice, and more.
Kevin Uhrmacher of the Washington Post prepared a graph that illustrates reported deaths over time, by number of organizations reporting the deaths.
