BJS Report on Arrest-Related Deaths: True Number Likely Much Greater
(This post is co-authored by Patrick Ball and Kristian Lum.)
Today the Bureau of Justice Statistics (BJS) released a report on their effort to document “all deaths that occur during the process of arrest in the United States.” The analysis estimates that the Arrest-Related Deaths (ARD) program covers only 34-49% of these deaths. A parallel program by the FBI (the Supplementary Homicide Reports, SHR) is estimated to cover approximately the same proportion of deaths. Even taking into consideration both programs, 28% of all police homicides remain unreported.
In order to estimate the total number of homicides that appear on neither the ARD or SHR databases, the report relies upon a statistical technique that we at HRDAG regularly employ to estimate the number of undocumented conflict-related deaths: multiple systems estimation. The BJS analysts had only two databases available to them (the ARD and the SHR); HRDAG generally uses three or more databases.
Using only two databases requires the assumption that deaths are reported to both systems independently. That is, one must assume that the appearance of a homicide in one database makes that homicide no more or less likely to appear in the other (this is called list independence; see our explanation here). From our experience with conflict mortality estimation, we believe this assumption is unlikely to be true. We have typically found substantial positive list dependence in analyses with multiple lists. By positive list dependence, we simply mean that if a homicide is reported to one list (e.g. ARD), it is more likely than average to have also been reported to the other (e.g. SHR).
In the context of police homicides, there are several plausible ways in which positive list dependence might occur. Positive list dependence may be induced by characteristics within the victim population, for example, social visibility. The BJS report states that 26 out of 50 states relied solely on public media to identify homicides for the ARD. Those homicides that were visible enough to have attracted media attention may be wealthier or otherwise more socially prominent people relative to victims who did not attract media attention. Therefore, people more likely to have been noticed and documented by the media may have been more likely to be documented by the SHR as well. Thus, a victim with high social visibility would be likely to appear on both lists.
Positive list dependence may also arise as a function of the data collection process. For example, if both sources draw some of their information from a common source, victims reported to the common source would be very likely to appear on both lists. This report mentions that four states identify homicides using the National Violent Death Reporting System. We do not know how the SHR obtains all of its data, but if this same source also feeds into it, this could cause positive list dependence.
Two-system estimates such as the one BJS published today underestimate the total number of undocumented cases when there is positive list dependence. Because it is likely that the two databases used in this analysis are positively dependent, it is therefore very likely that these estimates are only a lower bound. In reality, it is likely that much more than 28% of all police homicides go unreported by the ARD and the SHR. (The BJS analysts are well aware of this limitation, and they express it in Section 2.2, p8, of the Technical Report, NCJ 248543.)
Said differently, the estimated annual average of 928 police homicides per year should be understood as a bare minimum. The true number is likely to be much greater.
We congratulate the authors of the report on an extremely thorough, transparent, and well reasoned implementation of record linkage and multiple systems estimation. We do a lot of estimates like this, and we know how substantial this task is.
HRDAG estimates total killings in places where it is impossible to establish a comprehensive database of killings committed by the government, places like Guatemala during its internal armed conflict, Kosovo during the war between Yugoslavia and NATO, and Perú during the civil war. It is profoundly worrisome to think that the United States has joined this group of countries obfuscating state violence. In this regard, we welcome remarks by the FBI and the Attorney General that an accurate and comprehensive database will be established.
Meanwhile, while local and federal officials are unable or unwilling to comprehensively document police abuses, reports like this one will continue to be necessary. If we cannot list every victim, at least they should be counted.