“In 2016, two researchers, the statistician Kristian Lum and the political scientist William Isaac, set out to measure the bias in predictive policing algorithms. They chose as their example a program called PredPol. … Lum and Isaac faced a conundrum: if official data on crimes is biased, how can you test a crime prediction model? To solve this technique, they turned to a technique used in statistics and machine learning called the synthetic population.”
How do police officer booking decisions affect tools relied upon by judges?
Much of the work we do at HRDAG involves estimating the number of undocumented deaths using a statistical technique called multiple systems estimation (MSE, described in more detail here). One of our goals is to make this class of methods more broadly available to human rights researchers. In particular, we are finding that Bayesian approaches are extremely valuable for MSE. Accordingly, we are pleased to offer a new R package called dga (“decomposable graphs approach”) that performs Bayesian model averaging for MSE.
The main function in this package implements a model created by David Madigan and Jeremy York. This model was designed to ...
In this story about how data are transforming the nonprofit world, Patrick Ball is quoted. Here's an excerpt:
"Data can have a profound impact on certain problems, but nonprofits are kidding themselves if they think the data techniques used by corporations can be applied wholesale to social problems," says Patrick Ball, head of the nonprofit Human Rights Data Analysis Group.
Companies, he says, maintain complete data sets. A business knows every product it made last year, when it sold, and to whom. Charities, he says, are a different story.
"If you're looking at poverty or trafficking or homicide, we don't have all the data, and we're not going to," ...
One year ago, HRDAG cast out on its own as an independent nonprofit—and this first year has been busy, productive, and exciting. We’re indebted to our Advisory Board for their valuable contributions and to our funders for their generosity and participation in our mission. Highlights of the past year include contributing testimony to three court cases, publishing two reports on conflict-casualties in Syria, presenting over a dozen talks (many of which are available on our talks page), traveling to over half a dozen countries to testify, collaborate with partners, and participate in conferences/workshops, hiring a new technical lead, and bringing in ...
Everyone I had the pleasure of interacting with enriched my summer in some way.
I joined the Benetech Human Rights Program at essentially the same time that HRDAG did, coming to Benetech from years of analyzing data for large companies in the transportation, hospitality and retail industries. But the data that HRDAG dealt with was not like the data I was familiar with, and I was fascinated to learn about how they used the data to determine "who did what to whom." Although some of the methodologies were similar to what I had experience with in the for-profit sector, the goals and beneficiaries of the analyses were very different.
At Benetech, I was initially predominantly focused on product management for Martus, a free ...
The interview poses questions about Lum's focus on artificial intelligence and its impact on predictive policing and sentencing programs.
The summer of 2002 in Washington, DC, was steamy and hot, which is how I remember my introduction to HRDAG. I had begun working with them, while they were still at AAAS, in the late spring, learning all about their core concepts: duplicate reporting and MSE, controlled vocabularies, inter-rater reliability, data models and more. The days were long, with a second shift more often than not running late into the evening. In addition to all the learning, I also helped with matching for the Chad project – that is, identifying multiple records of the same violation – back when matching was done by hand. But it was not long after I arrived in Washington ...
Kristian Lum (2017). Limitations of mitigating judicial bias with machine learning. Nature. 26 June 2017. © 2017 Macmillan Publishers Limited, part of Springer Nature. All rights reserved. Nature Human Behavior. DOI 10.1038/s41562-017-0141.
.
Kristian Lum (2017). Limitations of mitigating judicial bias with machine learning. Nature. 26 June 2017. © 2017 Macmillan Publishers Limited, part of Springer Nature. All rights reserved. Nature Human Behavior. DOI 10.1038/s41562-017-0141.
Patrick Ball and Megan Price (2019). Using Statistics to Assess Lethal Violence in Civil and Inter-State War. Annual Review of Statistics and Its Application, Volume 6. 7 March 2019. © 2019 Annual Reviews. All rights reserved. https://doi.org/10.1146/annurev-statistics-030718-105222.
Patrick Ball and Megan Price (2019). Using Statistics to Assess Lethal Violence in Civil and Inter-State War. Annual Review of Statistics and Its Application. 7 March 2019. © 2019 Annual Reviews. All rights reserved. https://doi.org/10.1146/annurev-statistics-030718-105222.
/wp-content/uploads/2013/01/Definition_of_Database_Design_Standards_1994.pdf
Patrick Ball. “A Definition of Database Design Standards for Human Rights Agencies.” © 1994 American Association for the Advancement of Science. [pdf]
Patrick Ball expanded his use of multiple systems estimation (MSE) to clarify the history of a deadly conflict in Kosovo. The violence began in 1989 when Serbian President Slobodan Milošević revoked Kosovo's autonomous status within the Republic of Serbia triggering fighting between Kosovar Albanians and the Yugoslav government. Allegations of widespread and systematic human rights violations were made against Serbian forces and NATO intervened to repel Serb forces from Kosovo. Ball and Scheuren gathered data from Albanian border crossings and other sources in the region. They used this information to examine the claim by the Yugoslav government ...
In Responsible Data Reflection Story #7—from the Responsible Data Forum—work by HRDAG affiliates Anita Gohdes and Brian Root is cited extensively to make the point about how quantitative data are the result of numerous subjective human decisions. An excerpt: “The Human Rights Data Analysis Group are pioneering the way in collecting and analysing figures of killings in conflict in a responsible way, using multiple systems estimation.”
The primer addresses what pretrial risk assessment is and what the research supports.
Megan Price and Patrick Ball. 2015. Statistical Journal of the IAOS 31: 263–272. doi: 10.3233/SJI-150899. © IOS Press and the authors. All rights reserved. Creative Commons BY-NC-SA.
In our database deduplication work, we’re trying to figure out which records refer to the same person, and which other records refer to different people.
We write software that looks at tens of millions of pairs of records. We calculate a model that assigns each pair of records a probability that the pair of records refers to the same person. This step is called pairwise classification.
However, there may be more than just one pair of records that refer to the same person. Sometimes three, four, or more reports of the same death are recorded.
So once we have all the pairs classified, we need to decide which groups of records refer to the ...