298 results for search: truth commissions/feed/content/colombia/copyright
Pulling Back the Curtain on LLMs & Policing Data
Structural Zero Issue 04
September 30, 2025
Artificial intelligence is transforming how we work with information. At HRDAG, that changes how I do my job every day. My most recent project was using LLMs to explore and parse vast quantities of data about police abuses in California.
In this newsletter, I’ll pull back the curtain on that work. I’ll describe how a diverse coalition gathered more than a million pages of documents about police misconduct in California and how LLMs helped us make sense of them in ways that wouldn’t have been possible before the advent of this technology.
In addition to understanding my work, I hope that this ...
HRDAG Retreat 2014
Ten data nerds gathered in a large hilltop beach house to analyze counts of killings from several war-torn countries. The time was January 16-20, 2014, the place was near San Francisco, the agenda was packed, and I was excited to be there.
Having defended my dissertation at Carnegie Mellon University just days before, I had often supposed that my thesis on a generalization of
log-linear models for capture-recapture might serve little other purpose than to fill a line on my curriculum vitae. This perception faded after a mid-2013 discussion with Patrick convinced me that HRDAG's data challenges could easily be the best match to my research ...
Welcoming Our 2019 Visiting Analyst
Valentina Rozo Ángel has joined our team as our new visiting analyst this fall.
Los asesinatos de líderes sociales que quedan fuera de las cuentas
Una investigación de Dejusticia y Human Rights Data Analysis Group concluyó que hay un subconteo en los asesinatos de líderes sociales en Colombia. Es decir, que el aumento de estos crímenes en 2016 y 2017 podría ser incluso mayor al reportado por las organizaciones y por las cifras oficiales.
Funding
HRDAG’s funding comes from private, international donors: the John D. and Catherine T. MacArthur Foundation, Open Society Foundations, an anonymous U.S.-based private foundation, Ford Foundation, Silicon Valley Community Foundation and individual donors. This funding supports both specific projects, as well as our scientific work generally in human rights data analysis.
For the entirety of its existence, HRDAG has been a project of non-profit organizations, first at the American Association for the Advancement of Science (AAAS), and then at Benetech, a non-profit Silicon Valley technology company. In February 2013, HRDAG ...
Talks & Discussions
2021 Rafto Prize Videos
.ugb-1c7c838 .ugb-video-popup__wrapper{height:460px !important;background-color:#000000;background-image:url(https://hrdag.org/wp-content/uploads/2022/12/Screen-Shot-2022-12-09-at-3.41.30-PM.png)}.ugb-1c7c838 .ugb-video-popup__wrapper:before{background-color:#000000;opacity:0.3}.ugb-1c7c838 .ugb-video-popup__wrapper:hover:before{opacity:0.6}.ugb-1c7c838 .ugb-block-title{color:#ffffff}.ugb-1c7c838 .ugb-block-description{color:#ffffff}@media screen and (max-width:768px){.ugb-1c7c838 .ugb-video-popup__wrapper{height:208px !important}}The Rafto Prize 2021 | Rafto Foundation Rafto Foundation | HRDAG team | 2021 | 4 ...
Remembering Scott Weikart
HRDAG’s core values all have a connection to Scott Weikart, 1951–2023.
Frequently Asked Questions
Multiple Systems Estimation
What is MSE?
What do you mean by statistical inference?
What is an overlap, and how do we know when lists overlap?
How does MSE find the total number of violations?
How was MSE originally developed?
How does the Benetech Human Rights Program use MSE?
1. What is MSE?
A: Multiple Systems Estimation, or MSE, is a family of techniques for statistical inference. MSE uses the overlaps between several incomplete lists of human rights violations to determine the total number of violations.
Return to Top
2. What do you mean by statistical inference?
A: ...
Multiple Systems Estimation: The Basics
Multiple systems estimation, or MSE, is a family of techniques for statistical inference. MSE uses the overlaps between several incomplete lists of human rights violations to determine the total number of violations. In this blogpost, and four more to follow, I’ll answer both conceptual and practical questions about this important method. (In posts to follow, questions that refer to specific statistical procedures or debates will be marked, "In depth.") (more…)
Data Archaeology for Human Rights in Central America: HRDAG Collaborates with UWCHR
Patrick Ball is kicking himself for a decision he made almost 25 years ago. “I was clever, but I wasn’t smart,” he says ruefully, as he considers the labyrinth of tables and ASCII-encoded keystrings he used to design a database of human rights violations for the pioneering Salvadoran non-governmental Human Rights Commission (CDHES). Now I’m sitting in his office in San Francisco’s Mission District watching over his shoulder, and trying to keep up, as he bangs out code to decipher the priceless data contained in these old files. Created in 1991 and 1992, during the last days of El Salvador’s internal armed conflict, the files detail ...
Reality and Risk in Our Mortality Study of the Peruvian TRC
HRDAG researchers and analysts at Peru's Truth and Reconciliation Commission (TRC) estimated conflict mortality due to violence using Capture-Recapture methods.
AI for Human Rights
From the article: “Price described the touchstone of her organization as being a tension between how truth is simultaneously discovered and obscured. HRDAG is at the intersection of this tension; they are consistently participating in science’s progressive uncovering of what is true, but they are accustomed to working in spaces where this truth is denied. Of the many responsibilities HRDAG holds in its work is that of “speaking truth to power,” said Price, “and if that’s what you’re doing, you have to know that your truth stands up to adversarial environments.”
Release of Yellow Book Calls on Salvadoran Military to Open Archives
With the release today of a civil war-era catalog of “enemies,” Salvadorans are calling for a new look at the 12-year civil war during which hundreds of citizens were victims of human rights violations such as torture, forced disappearance, and illegal imprisonment.
The recently leaked document, known as The Yellow Book, is a list created and (more…)
A Definition of Database Design Standards for Human Rights Agencies.
/wp-content/uploads/2013/01/Definition_of_Database_Design_Standards_1994.pdf
Patrick Ball. “A Definition of Database Design Standards for Human Rights Agencies.” © 1994 American Association for the Advancement of Science. [pdf]
Who Did What to Whom? Planning and Implementing a Large Scale Human Rights Data Project
/whodidwhattowhom/contents.html
Patrick Ball. Who Did What to Whom? Planning and Implementing a Large Scale Human Rights Data Project. © 1996 American Association for the Advancement of Science.
Guatemala 2011 – Developing Sampling Methods to Help Convict Perpetrators
During 36 years of internal armed conflict, which ended in 1996, an estimated 200,000 Guatemalans were killed or disappeared. HRDAG researchers returned to Guatemala in 2006 to analyze a sample of the estimated 46 million records discovered in the archive of the now disbanded Guatemalan National Police. HRDAG statisticians Daniel Guzmán, Romesh Silva, Patrick Ball and Tamy Guberek, together with Paul Zador and Gary Shapiro of the American Statistical Association, developed a multi-stage random sample of the archive to get a clearer picture of its contents. Sampled documents shed light on the disappearance of Guatemalan union leader Edgar Fernando ...
Trips to and from Guatemala
HRDAG has been working with the Historic Archive of the National Police in Guatemala (hereafter, the Archive) for the past seven years. The Archive contains a treasure trove of data recorded and kept by the Guatemalan National Police over the past century. When the Archive was first discovered in 2005, researchers there immediately recognized both the value and fragility of the tens of millions of documents. As a result, they reached out to HRDAG, and we reached out to volunteers at Westat to devise a plan to estimate the contents of the entire Archive as quickly as possible in case the documents were destroyed or access to them was limited. ...
The Art and Science of Coding AHPN Documents
The coding, from my perspective, is the heart of the project. I say this, because the coding team has the responsibility of selecting documents according to the random sample, recording the documents’ contents, and applying the criteria to convert that content into an entry in a quantitative database. Not to mention the fact that this team has the privilege of being in direct contact with the documents.
At present, because of advanced organizational processes, not everyone has a chance to hold an original document in their hands. The quantitative study had many advantages in this regard; since we started work in parallel with the archival ...
Learning Day by Day: Quantitative Research at the AHPN
Working at the Historic Archive of the National Police (AHPN) of Guatemala, there are many skills I learned on the job. My many years of work on the team that studies the recovered documents have been like a custom-made course in how to do quantitative research.
The Archive documents I study are the result of 36 years of creation during civil war (1960 to 1996). Many of these documents are simply administrative—but we are able to use them to understand patterns that occurred during the conflict, to get a sense of what mattered to the National Police and what didn’t. Our quantitative research shows us the Police behavior in broad strokes. ...
Celebrating Ten Years of Data from the AHPN
Ten years ago, in July 2005, human rights officers stumbled upon a nondescript warehouse in a commercial zone of Guatemala City and changed history. They had discovered an archive–its existence kept secret–belonging to the Guatemalan National Police, whose officers committed human rights atrocities on behalf of the government during the civil war.
Inside the building was the bureaucratic detritus typical of a large government agency: 80 million pages detailing shifts worked, tasks assigned, assignments fulfilled, workers’ whereabouts, and who was supervising whom. The documents, which were found stacked on dirty floors, shoved into bags, ...