HRDAG built a machine-learning tool to strip the raw data of any potentially identifying information such as names and court case numbers. There was no "acceptable error rate."
Paula Amado and María Juliana Durán Fedullo reflect on how the Truth Commission may change Colombia’s history, finally officially acknowledging the 50-year conflict and its casualties, and reckoning with who did what to whom.
Kristian Lum spoke about "Understanding the Context and Consequences of Pre-Trial Detention" at the Conference on Fairness, Accountability, and Transparency (FAT*).
Identifiers being sequential could make possible estimations of the population of detained children.
In July 2009, HRDAG concluded a three-year project with the Liberian Truth and Reconciliation Commission (TRC) to help clarify Liberia’s violent history and hold perpetrators accountable. A military coup in 1979 sparked 24 years of civil war in Liberia where warring factions subjected civilians to severe human rights abuses. The TRC sought to determine whether these violations represented a systematic pattern or policy. This chapter describes how HRDAG developed a statistical analysis of the more than 17,000 victim and witness statements collected by the TRC and applied Ball’s “Who Did What To Whom?” methodology. HRDAG scientist Kristen ...
We’re very happy to announce that our executive director, Patrick Ball, has been elected as a Fellow of the American Statistical Association (ASA), as announced by ASA President Nathaniel Schenker. Patrick is one of 63 new ASA Fellows to be honored this year in a ceremony at the Joint Statistical Meetings, which will take place this August 5 in Boston, Massachusetts. (more…)
There may have been more undocumented World War II-era Korean "comfort women" than known.
Laurel Eckhouse, Kristian Lum, Cynthia Conti-Cook and Julie Ciccolini (2018). Layers of Bias: A Unified Approach for Understanding Problems With Risk Assessment. Criminal Justice and Behavior. November 23, 2018. © 2018 Sage Journals. All rights reserved. https://doi.org/10.1177/0093854818811379
Laurel Eckhouse, Kristian Lum, Cynthia Conti-Cook and Julie Ciccolini (2018). Layers of Bias: A Unified Approach for Understanding Problems With Risk Assessment. Criminal Justice and Behavior. November 23, 2018. © 2018 Sage Journals. All rights reserved. https://doi.org/10.1177/0093854818811379
Ayyub Ibrahim, Huy Dao, and Tarak Shah (2024). “Innocence Discovery Lab - Harnessing Large Language Models to Surface Data Buried in Wrongful Conviction Case Documents." The Wrongful Conviction Law Review 5 (1):103-25. https://doi.org/10.29173/wclawr112. 31 May, 2024. Copyright (c) 2024 Ayyub Ibrahim, Huy Dao, Tarak Shah. Creative Commons Attribution 4.0 International License.
Ayyub Ibrahim, Huy Dao, and Tarak Shah (2024). “Innocence Discovery Lab – Harnessing Large Language Models to Surface Data Buried in Wrongful Conviction Case Documents.” The Wrongful Conviction Law Review 5 (1):103-25. https://doi.org/10.29173/wclawr112. 31 May, 2024. Copyright (c) 2024 Ayyub Ibrahim, Huy Dao, Tarak Shah. Creative Commons Attribution 4.0 International License.
Anjana Samant, Noam Shemtov, Kath Xu, Sophie Beiers, Marissa Gerchick, Ana Gutierrez, Aaron Horowitz, Tobi Jegede, Tarak Shah (2023). The Devil is in the Details: Interrogating Values Embedded in the Allegheny Family Screening Tool. ACLU. Summer 2023.
Anjana Samant, Noam Shemtov, Kath Xu, Sophie Beiers, Marissa Gerchick, Ana Gutierrez, Aaron Horowitz, Tobi Jegede, Tarak Shah (2023). The Devil is in the Details: Interrogating Values Embedded in the Allegheny Family Screening Tool. ACLU. Summer 2023.
Tarak Shah (2020). How many people are infected with Covid-19? Significance. 09 April 2020. © 2020 The Royal Statistical Society.
Tarak Shah (2020). How many people are infected with Covid-19? Significance. 09 April 2020. © 2020 The Royal Statistical Society.
Patrick Ball, Ethan Hee-Seok Shin and Hyerin Yang (2018). There may have been 14 undocumented Korean “comfort women” in Palembang, Indonesia. Human Rights Data Analysis Group. 26 December 2018.© 2018 HRDAG. Creative Commons.
Patrick Ball, Ethan Hee-Seok Shin and Hyerin Yang (2018). There may have been 14 undocumented Korean “comfort women” in Palembang, Indonesia. Human Rights Data Analysis Group. 26 December 2018.© 2018 HRDAG. Creative Commons.
HRDAG has sampled and analyzed documents at Guatemala's AHPN and has testified against war criminals based on that analysis.
HRDAG is currently evaluating the quality and completeness of the Kosovo Memory Book of the Humanitarian Law Center (HLC) in Belgrade, Serbia. The objective of the Kosovo Memory Book (KMB) is to commemorate every single person who fell victim to armed conflict in Kosovo from 1998 to 2000, either through death or disappearance.
While building and reviewing their database, one of the things that HLC has to do is “record linkage,” a process also known as “matching.” Matching determines whether two records are the same people (“a match”) or different people (“a non-match”). Matching helps to identify whether two existing records refer ...
Working at the Historic Archive of the National Police (AHPN) of Guatemala, there are many skills I learned on the job. My many years of work on the team that studies the recovered documents have been like a custom-made course in how to do quantitative research.
The Archive documents I study are the result of 36 years of creation during civil war (1960 to 1996). Many of these documents are simply administrative—but we are able to use them to understand patterns that occurred during the conflict, to get a sense of what mattered to the National Police and what didn’t. Our quantitative research shows us the Police behavior in broad strokes. ...
Help us hold human rights violators accountable!
The HRDAG Tech Corner is where we collect the deeper and geekier content that we create for the website. Click the accordion blocks below to reveal each of the Tech Corner entries.
Sifting Massive Datasets with Machine Learning
Principled Data Processing
Ten data nerds gathered in a large hilltop beach house to analyze counts of killings from several war-torn countries. The time was January 16-20, 2014, the place was near San Francisco, the agenda was packed, and I was excited to be there.
Having defended my dissertation at Carnegie Mellon University just days before, I had often supposed that my thesis on a generalization of
log-linear models for capture-recapture might serve little other purpose than to fill a line on my curriculum vitae. This perception faded after a mid-2013 discussion with Patrick convinced me that HRDAG's data challenges could easily be the best match to my research ...
HRDAG analysis shows that the government figures are a gross underestimation of the drug-related killings in the Philippines.