Researchers from Rice University and Duke University are using the tools of statistics and data science in collaboration with Human Rights Data Analysis Group (HRDAG) to accurately and efficiently estimate the number of identified victims killed in the Syrian civil war.

…

Using records from four databases of people killed in the Syrian war, Chen, Duke statistician and machine learning expert Rebecca Steorts and Rice computer scientist Anshumali Shrivastava estimated there were 191,874 unique individuals documented from March 2011 to April 2014. That’s very close to the estimate of 191,369 compiled in 2014 by HRDAG, a nonprofit that helps build scientifically defensible, evidence-based arguments of human rights violations.

But while HRDAG’s estimate relied on the painstaking efforts of human workers to carefully weed out potential duplicate records, hashing with statistical estimation proved to be faster, easier and less expensive. The researchers said hashing also had the important advantage of a sharp confidence interval: The range of error is plus or minus 1,772, or less than 1 percent of the total number of victims.

“The big win from this method is that we can quickly calculate the probable number of unique elements in a dataset with many duplicates,” said Patrick Ball, HRDAG’s director of research. “We can do a lot with this estimate.”

HRDAG director of research Patrick Ball is quoted in this New York Times article about a paper that models death tolls in Gaza.

From the article: “Intentionally inconsistent tracking can also influence the final tally, notes Megan Price, a statistician at the Human Rights Data Analysis Group. During the Iraq War, for example, officials worked to conceal mortality or to cherry pick existing data to steer the political narrative. While wars are handled differently from pandemics, Price thinks the COVID-19 data could still be at risk of this kind of manipulation.”

In this video, Trina Reynolds-Tyler of Invisible Institute talks about her work with HRDAG on the missing persons project in Chicago and Beneath the Surface.

William Isaac is quoted.

From the article: Esta entidad, que existe desde 1991, es liderada por su fundador, Patrick Ball, un científico que acumula una experiencia de más de 25 años realizando análisis cuantitativos en los lugares y en las situaciones más convulsos del planeta. Sobre su colaboración con el proyecto del predictor de fosas clandestinas en México, único en el mundo, Ball afirmó en entrevista:

“Cuando hablamos de crímenes de lesa humanidad estamos hablando de instituciones, de organizaciones grandes, cometiendo miles o centenares de miles  de violaciones a víctimas distribuidas sobre una geografía enorme. Para entender los patrones en esas violaciones, la estadística puede brindar una mirada sobre quiénes son los responsables materiales e intelectuales, quiénes son las víctimas y dónde o cuándo pasaron esas violaciones. Pero la estadística no es contabilidad, pues no estamos hablando solamente de las violaciones que podemos ver, sino que también debemos calcular las violaciones no observadas, las escondidas, invisibles, para incluir en nuestro análisis la totalidad de las violaciones”.

Ball analyzed the data reporters had collected from a variety of sources – including on-the-ground interviews, police records, and human rights groups – and used a statistical technique called multiple systems estimation to roughly calculate the number of unreported deaths in three areas of the capital city Manila.

The team discovered that the number of drug-related killings was much higher than police had reported. The journalists, who published their findings last month in The Atlantic, documented 2,320 drug-linked killings over an 18-month period, approximately 1,400 more than the official number. Ball’s statistical analysis, which estimated the number of killings the reporters hadn’t heard about, found that close to 3,000 people could have been killed – more than three times the police figure.

Ball said there are both moral and technical reasons for making sure everyone who has been killed in mass violence is counted.

“The moral reason is because everyone who has been murdered should be remembered,” he said. “A terrible thing happened to them and we have an obligation as a society to justice and to dignity to remember them.”

In this NPR story, HRDAG’s Patrick Ball comments on first-of-its-kind projections.

“In 2016, two researchers, the statistician Kristian Lum and the political scientist William Isaac, set out to measure the bias in predictive policing algorithms. They chose as their example a program called PredPol.  … Lum and Isaac faced a conundrum: if official data on crimes is biased, how can you test a crime prediction model? To solve this technique, they turned to a technique used in statistics and machine learning called the synthetic population.”

NPR Illinois has covered the new National Police Index, created by HRDAG’s Tarak Shah, Ayyub Ibrahim of Innocence Project, and Sam Stecklow of Invisible Institute.

Earlier this year, the Canadian Tamil Congress connected with HRDAG to bring its campaign to Toronto’s annual Tamil Fest, one of the largest gatherings of Canada’s Sri Lankan diaspora.

Ravichandradeva, along with a few other volunteers, spent the weekend speaking with festival-goers in Scarborough about the project and encouraging them to come forward with information about deceased or missing loved ones and friends.

“The idea is to collect thorough, scientifically rigorous numbers on the total casualties in the war and present them as a non-partisan, independent organization,” said Michelle Dukich, a data consultant with HRDAG.