Justice and impunity in Guatemala, algorithmic fairness in the US criminal justice system, and methodological advances

A 2018 year-end review from HRDAG

Catalina Sanchez testifies against General José Efraín Ríos Montt, overseer of genocide in Guatemala.

Photo by Elene Hermosa / Trocair, CC BY 2.0; modified by David Peters

Strength in numbers

Megan Price

We’ve completed some exciting and meaningful work in 2018, which you’ll find highlighted below. But this year I have been most gratified by our team’s growth. One of the pillars of the HRDAG mission is to build the field of quantitative analysts who think deeply and carefully about human rights, and thanks to your support we’ve been able to do that, by creating several new positions this year.

We recently hired a full-time data scientist, created a new data science fellow position, and put together our first formal development team. I’m excited about the sustained long-term increase in capacity this will mean for our organization, and I’m looking forward to taking on more projects that can help us fight for justice, support our partners and lead critical conversations around human rights. A year from now, I’ll be reporting on new progress and accomplishments enabled by this growth!

››› more

Reality Check

Thirty-five years and proof of genocide lead to an important though mixed victory in Guatemala.

HRDAG has never wavered providing consistent expert testimony across multiple trials. The first time was in 2013 when judges found General Ríos Montt guilty, though that verdict only stood for ten days. The second was in 2018 when the court unanimously concluded that the Guatemalan Army committed genocide during the de facto government of Ríos Montt.


Guatemala massacre

General José Efraín Ríos Montt oversees the most violent period of Guatemala’s 36-year armed conflict, resulting in the killings of thousands of citizens.

Photo: Erik Hungerbuhler CC BY-NC-SA 2.0

THE prosEcuTion TiMELinE


Guatemala massacre

The Truth Commission report includes HRDAG analysis of three databases consisting of interviews and reports of violations. Statistical estimates are central to its conclusion that 200,000 people were killed or ‘disappeared’ and the pattern of violence was consistent with genocide.


Guatemala massacre

HRDAG is invited to present expert witness testimony in the case against Ríos Montt, based on work presented in the Truth Commission report.

April 12, 2013

Guatemala massacre

Patrick Ball testifies in the Guatemalan Supreme Court for two hours. He is one of over 90 witnesses who testify for the prosecution during a nearly two-month trial.

Photo: Erik Hungerbuhler CC BY-NC-SA 2.0

May 10, 2013

Guatemala massacre

General Ríos Montt is found guilty of committing acts of genocide; Patrick’s testimony is quoted in the judges’ verdict.

Photo: LaJornada

May 20, 2013

Guatemala massacre

The Guatemalan Constitutional Court overturns the verdict on a legal technicality and annuls the final days of the trial.

Photo: Public Domain

March 2, 2018

Guatemala massacre

Patrick returns to the Guatemalan Supreme Court where he testifies twice in front of the High Risk Court, for a total of two hours.

Photo: Jo-Marie Burt

April 1, 2018

Guatemala massacre

Ríos Montt dies.

Photo: Así Sucede León

General Ríos Montt may have evaded accountability by dying before the verdict could be rendered in his final trial, but the Guatemalan justice system has concluded, multiple times, that genocide occurred under his leadership.

For the Truth Commission report in 1999 and again in preparing expert witness testimony in 2013, HRDAG researchers analyzed records from multiple databases to estimate the total number of victims killed or disappeared during Guatemala’s internal armed conflict. They also used census estimates to calculate homicide rates for members of the Mayan community and non-Mayans living in the same region.

This analysis found that members of the Mayan community suffered homicide rates between five and eight times higher than non-Mayans. This stark difference provided the basis for the truth commission’s conclusion “... that agents of the State of Guatemala, within the framework of counterinsurgency operations carried out between 1981 and 1983, committed acts of genocide against groups of Mayan people.”

The Guatemala project and the many others HRDAG pursued in the past twelve months illuminate what data can tell us — and why your support is so valuable to social justice.
Support our work with donations to hrdag.org/donate

More highlights from 2018

ALGORITHMIC FAIRNESS: Kristian Lum partnered with Liz Bender from the New York Legal Aid Society to organize a tutorial on “Understanding the Context and Consequences of Pre-Trial Detention” at the Conference on Fairness, Accountability, and Transparency (FAT*). A key presenter was Terrence Wilkerson who spoke about his first-hand experiences within the criminal justice system after being falsely accused of robbery. This tutorial brought an important perspective — that of a person impacted by the technology being discussed — to the conversation around risk assessment.

HRDAG equations

The measure of fairness — Multiple, contradictory, definitions help statisticians evaluate risk assessment models, like those used by the justice system when determining detention or bail.

Guatemala massacre

Extraordinary effort noted — ASA President Lisa LaVange congratulates Patrick, recipient of the 2018 Karl E Peace Award.

RECOGNITION: Patrick Ball received the Karl E Peace Award for Outstanding Statistical Contributions for the Betterment of Society. In presenting the award, Ron Wasserstein, executive director of the American Statistical Association, said, “Patrick Ball is an extraordinary individual whose work spans an area often ignored yet incredibly important to our statistics community and to the world at large. Since the early 1990s, Patrick has brought fresh approaches to data analysis of human rights crises. He has a unique ability to really target the biases in the collection, formats, and analysis of human rights data.”

METHODOLOGICAL ADVANCES: Lead statistician Kristian Lum contributed to new advances in record linkage and multiple systems estimation in two articles in the peer-reviewed academic journal Biometrika, the first with colleagues James Johndrow and David Dunson, the second with Johndrow and Daniel Manrique-Vallier. These important advances strengthen our analysis of human rights violations.

Guatemala massacre

Equations matter — The science of estimation has real world consequences.

Guatemala massacre

Casting light on losses — The August 2018 publication of “Asesinatos de líderes sociales en Colombia en 2016–2017: una estimación del universo”.

Photo: EFE/Leonardo Muñoz

COLOMBIA: As the war between the guerrillas, the Army, and paramilitary groups in Colombia winds down, violence against social movement leaders has intensified. We partnered with DeJusticia to analyze data from six organizations and estimate the total number of social movement leaders killed in 2016 and 2017. Based on this analysis, we conclude that there is a 50 percent probability that fatal violence against social movement leaders increased by 10 percent or greater between 2016–2017.

A year by the numbers



countries with active HRDAG projects this year


reports assessing situations of mass violence


requests for technical assistance



technical papers accepted in peer-reviewed publications; each paper improves the use of statistical modeling in human rights case



requests for media interviews


invitations to collaborate or participate in meetings, panels, convenings, or conferences


universities participating in active methodological collaborations


In 2018, we were able to pursue our mission with a highly skilled staff of 5, plus 5 advisory board members, 15 consultants, 3 summer interns, and 1 fall fellow.

HRDAG’s fiscal year is July 1–June 30



Beginning cash balance




Foundation grants



Revenue from contracts



Direct public support



Total income




Salaries and consultants



Travel and conferences



Rent, utilities, and technology



Supplies and other direct costs






Total expenses



Ending cash balance



HRDAG operates as a nonprofit project of Community Partners (communitypartners.org), a nonprofit organization that helps community leaders build and sustain effective programs that benefit the public good.

As our fiscal sponsor, Community Partners offers back-office services and the legal framework that allows nonprofit ventures to focus on their missions.

Your support makes our work possible

Human rights research — if done well — requires the dedication of a full-time team working with committed consultants. It requires investment in staff, in travel, in equipment and technology.

Our core team is small and focused. Key staff include executive director Dr Megan Price, director of research Dr Patrick Ball, lead statistician Dr Kristian Lum, data scientist Tarak Shah, and admin Suzanne Nathans, all based in San Francisco. We also work with colleagues who are based across the United States and Europe. These consultants make it possible to expand our capacity and rapidly respond to opportunities as they arise.

Primary funding comes from private, international human rights donors. The majority of our funding is not tied to specific projects, but rather supports our ongoing scientific work in human rights data analysis. A small but growing proportion of our funding specifically supports our work examining the use of predictive algorithms in the US criminal justice system.

Thank you to all our supporters and these major donors:

HRDAG equations

David & Anita Keller Foundation

Cooper Schneier Fund of The Minneapolis Foundation

The Anne R. Dow Family Foundation

Katherine Crecelius

Frank Schulenburg

Rick Storrs

Morgan Agnew

David Banks

Julie Broome

Cindy Cohn

Ruth Eckhouse

Alisha Walker

Oli Hall

Chris Palmer

Paul Fontes

Sudhir Bhaskar

Zennström Philanthropies


More than just words — Join us in celebrating 70 years of the Universal Declaration of Human Rights (1948–2018).

We provide analysis so that our partners — human rights advocates — can build scientifically defensible, evidence-based arguments. Our work has been used by truth commissions, international criminal tribunals, and non-governmental human rights organizations on five continents.

HRDAG equations

Put data science to work for justice and human rights. hrdag.org/donate