It could make sense to use Rust as a data journalist for in-browser computations, and other thoughts from RustConf.
I had the pleasure of working with Patrick Ball at the HRDAG office in San Francisco for a week during summer 2016. I knew Patrick from two workshops he previously hosted at the University of Washington’s Centre for Human Rights (UWCHR). The workshops were indispensable to us at UWCHR as we worked to publish a number of datasets on human rights violations during the El Salvador Civil War. The training was all the more helpful because the HRDAG team was so familiar with the data. As part of an impressive career which took him from Ethiopia and Kosovo to Haiti and El Salvador among others, Patrick himself had worked on gathering and analysing ...
This post describes how we organize our work over ten years, twenty analysts, dozens of countries, and hundreds of projects: we start with a task. A task is a single chunk of work, a quantum of workflow. Each task is self-contained and self-documenting; I'll talk about these ideas at length below. We try to keep each task as small as possible, which makes it easy to understand what the task is doing, and how to test whether the results are correct.
In the example I'll describe here, I'm going to describe work from our Syria database matching project, which includes about 100 tasks. I'll start with the first thing we do with files we receive ...
The HRDAG Tech Corner is where we collect the deeper and geekier content that we create for the website. Click the accordion blocks below to reveal each of the Tech Corner entries.
Sifting Massive Datasets with Machine Learning
Principled Data Processing
There may have been more undocumented World War II-era Korean "comfort women" than known.
UPDATE (21 Dec 2014): Juan Cole is reporting that the Kurdish militia (the peshmerga) have retaken Shingal (also known as Sinjar) mountain where many Yezidi people have been trapped since 3 August 2014. They are now moving to liberate other Yezidi towns south of the mountain. The Yezidi people trapped on the mountain are now free. There is no word yet on the thousands of Yezidi people enslaved by ISIS.
ORIGINAL (19 Nov 2014): Farhad (not his real name) got the call from ISIS on his personal cell phone just after lunch: we have your sister, and we will give her back if you pay us $6000, plus $1500 for the driver.
Carrying little more than his ...
Illuminating Data's Dark Side: Big data create conveniences, but we must consider who designs these tools, who benefits from them, and who is left out of the equation.
HRDAG’s analysis and expertise continues to deepen the national conversation about police violence and criminal justice reform in the United States. In 2015 we began by considering undocumented victims of police violence, relying on the same methodological approach we’ve tested internationally for decades. Shortly after, we examined “predictive policing” software, and demonstrated the ways that racial bias is baked into the algorithms. Following our partners’ lead, we next considered the impact of bail, and found that setting bail increases the likelihood of a defendant being found guilty. We then broadened our investigations to examine ...
En ese artículo respondemos a una crítica del estudio de mortalidad que realizamos para la Comisión de la Verdad y Reconciliación en 2003.
The World According to Artificial Intelligence: Targeted by Algorithm (Part 1)
The Big Picture: The World According to AI explores how artificial intelligence is being used today, and what it means to those on its receiving end.
Patrick Ball is interviewed: “Machine learning is pretty good at finding elements out of a huge pool of non-elements… But we’ll get a lot of false positives along the way.”
The modular nature of the workflow and use of Git allowed us to work on different parts of the project from across the country.
Trina Reynolds-Tyler's internship at HRDAG helped her use data science to find patterns in state-sanctioned violence.
Help us hold human rights violators accountable!
Herb led and mentored a generation of statisticians working in human rights.
In 2001, President Alejandro Toledo, called for the establishment of the Comision de la Verdad y Reconciliacion (CVR) (Truth and Reconciliation Commission) to investigate human rights abuses in Perú between 1980 and 2000. Dr. Patrick Ball was invited to work with the Commission to evaluate the CVR technical work and make recommendations on the information management process and analytic strategies. HRDAG consultant Jana Asher worked with Dr. Ball and CVR staff members David Sulmont (Director Informations Systems) and Daniel Manrique (Database Expert) to present evidence of the violations in a report to the CVR.
The work included new estimates of ...
2023
10/19/2023 - Will you support HRDAG’s role in transformative justice?
09/25/2023 - The fruit of long collaborations
06/28/2023 - HRDAG publishes largest dataset in the history of human movement (special announcement)
06/15/2023 - Connecting with community
03/22/2023 - The risks of using AI to predict risk
2022
12/15/2022 - HRDAG and international trials
12/10/2022 - HRDAG on Human Rights Day
12/01/2022 - Will you help HRDAG examine police misconduct?
11/25/2022 - We’re thankful for your support
11/10/2022 - HRDAG and criminal justice
10/31/2022 - HRDAG's decade of work in Syria
10/13/2022 - Will you ...
“The 36th Rafto Prize was awarded to the Human Rights Data Analysis Group (HRDAG) for their work on uncovering large-scale human rights violations. By using statistics and data science, HRDAG documents human rights violations that might otherwise go undetected. Their approach has enabled courts to bring perpetrators to justice and given closure to affected victims and their families.”
HRDAG’s core values all have a connection to Scott Weikart, 1951–2023.
William Isaac and Kristian Lum. Predictive policing violates more than it protects. USA Today. December 2, 2016. © USA Today.
William Isaac and Kristian Lum. Predictive policing violates more than it protects. USA Today. December 2, 2016. © USA Today.