<<Previous post: Collection, Cleaning, and Canonicalization of Data
Q8. What do you mean by "overlap," and why are overlaps important?
Q9. [In depth] Why is automated matching so important, and what process do you use to match records?
Q8. What do you mean by "overlap," and why are overlaps important?
MSE estimates the total number of violations by comparing the size of the overlap(s) between lists of human rights violations to the sizes of the lists themselves. By "overlap," we mean the set of incidents, such as deaths, that appear on more than one list of human rights violations. Accurately and efficiently identifying overlaps between ...
Patrick Ball, Ethan Hee-Seok Shin and Hyerin Yang (2018). There may have been 14 undocumented Korean “comfort women” in Palembang, Indonesia. Human Rights Data Analysis Group. 26 December 2018.© 2018 HRDAG. Creative Commons.
Patrick Ball, Ethan Hee-Seok Shin and Hyerin Yang (2018). There may have been 14 undocumented Korean “comfort women” in Palembang, Indonesia. Human Rights Data Analysis Group. 26 December 2018.© 2018 HRDAG. Creative Commons.
Bing Wang has joined HRDAG as a Visiting Data Science Student until the summer of 2020.
We’re very happy to announce that our executive director, Patrick Ball, has been elected as a Fellow of the American Statistical Association (ASA), as announced by ASA President Nathaniel Schenker. Patrick is one of 63 new ASA Fellows to be honored this year in a ceremony at the Joint Statistical Meetings, which will take place this August 5 in Boston, Massachusetts. (more…)
Last week HRDAG’s executive director, Patrick Ball, served as an expert witness for the prosecution in the trial of Hissène Habré, the ruler of Chad from 1982 to 1990. The trial is taking place in Dakar, Senegal, where the 73-year-old Habré has been living since 1990 when he fled Chad. He has already been sentenced to death in absentia in Chad.
Habré is being charged with war crimes, crimes against humanity, and torture that took place during his eight-year reign. The trial is happening at the Extraordinary African Chambers, which was inaugurated by Senegal and the African Union to try Habré. This is the first time that one country has ...
I look at the beach and then at the table surrounded by nerds, deep in thought and conversation about Dirichlet priors, matching algorithms, and armed conflicts. This peculiar (in the best way) environment catalyzes a moment of reflection: how did I get here?
Four years ago, as a second-year statistics PhD student, I watched "Guatemala: The Secret Files" on PBS Frontline World. I listened to stories of family members who disappeared without answers or justice. Then the story shifted to the work being done by archivists and data experts at Guatemala's Historic Archive of the National Police. The scientists' pursuit of the truth energized me. I ...
In 2018, HRDAG collaborated on work in Guatemala, US criminal justice, and more.
Kevin Uhrmacher of the Washington Post prepared a graph that illustrates reported deaths over time, by number of organizations reporting the deaths.
If this could be you, let us know. Also, please feel free to pass on this link to great people.
Job Title. Technical lead with a hacker's heart
Location. A cool office in SOMA, San Francisco. You need to be on-site with us.
What we do. The Human Rights Data Analysis Group (HRDAG) develops statistical techniques to measure human rights atrocities. Our work helps bring dictators to justice through data analysis of human rights atrocities around the world. Over more than 20 years, our small team has developed technology and statistical techniques to take disjoint, incomplete, and inaccurate information from conflict zones and process it to identify ...
Kristian Lum spoke about "Understanding the Context and Consequences of Pre-Trial Detention" at the Conference on Fairness, Accountability, and Transparency (FAT*).
In July 2009, HRDAG concluded a three-year project with the Liberian Truth and Reconciliation Commission (TRC) to help clarify Liberia’s violent history and hold perpetrators accountable. A military coup in 1979 sparked 24 years of civil war in Liberia where warring factions subjected civilians to severe human rights abuses. The TRC sought to determine whether these violations represented a systematic pattern or policy. This chapter describes how HRDAG developed a statistical analysis of the more than 17,000 victim and witness statements collected by the TRC and applied Ball’s “Who Did What To Whom?” methodology. HRDAG scientist Kristen ...
The Historic Archive of the Guatemalan National Police (hereafter the Archive) was discovered, quite by accident, in July 2005. Researchers immediately recognized both the importance and the fragility of the Archive's contents. As a result, in early 2006 the Archive team invited Patrick to evaluate the documents and help them answer a seemingly simple question: How can we learn about the contents of the Archive in a shorter period of time than is needed to systematically examine each individual document?
After inspecting the Archive, Patrick designed a multi-stage random sample of documents. In May 2006, Tamy Guberek, Daniel Guzmán, and ...
Laurel Eckhouse, Kristian Lum, Cynthia Conti-Cook and Julie Ciccolini (2018). Layers of Bias: A Unified Approach for Understanding Problems With Risk Assessment. Criminal Justice and Behavior. November 23, 2018. © 2018 Sage Journals. All rights reserved. https://doi.org/10.1177/0093854818811379
Laurel Eckhouse, Kristian Lum, Cynthia Conti-Cook and Julie Ciccolini (2018). Layers of Bias: A Unified Approach for Understanding Problems With Risk Assessment. Criminal Justice and Behavior. November 23, 2018. © 2018 Sage Journals. All rights reserved. https://doi.org/10.1177/0093854818811379
Kristian Lum, Chesa Boudin and Megan Price (2020). The impact of overbooking on a pre-trial risk assessment tool. FAT* '20: Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. January 2020. Pages 482–491. https://doi.org/10.1145/3351095.3372846 ©ACM, Inc., 2020.
Kristian Lum, Chesa Boudin and Megan Price (2020). The impact of overbooking on a pre-trial risk assessment tool. FAT* ’20: Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. January 2020. https://doi.org/10.1145/3351095.3372846 ©ACM, Inc., 2020.