New Research Shows Community Engagement Improves the Validity and Reliability of Artificial Intelligence
Community involvement can improve the validity and reliability of artificial intelligence (AI), argue HRDAG Executive Director Dr. Megan Price and Citizens and Technology Lab founder Dr. J. Nathan Matias in new research published this week.
AI is transforming the way scientists analyze complex data. Computer scientists, social scientists, and other professionals are using AI to parse and understand datasets related to healthcare, policing, education, insurance, and much more.
But while AI may help researchers, computers alone may miss important issues.
“How Public Involvement Can Improve the Science of Artificial Intelligence,” published in the peer-reviewed journal Proceedings of the National Academy of Sciences of the United States of America (PNAS), dives into some of the public participation models, discusses the potential pitfalls, and offers a literature review about the impact of engaging the public in AI research. Matias and Price also share their own experiences evaluating AI using public participation in areas like police oversight and social media algorithms.
Matias and Price note that while AI excels at parsing data, computers alone cannot determine which questions are meaningful—or which measurements truly reflect reality. For example, AI in healthcare is often implemented in the hope of improving patient health. But how can “patient health” be measured? If researchers look solely at healthcare costs—something easy to measure and track over time—they may mistake that as a stand-in for health itself. But costs can go down even when health is also declining, such as when patients stop going to the doctor or when important screening tests aren’t ordered. Human involvement, including from patients and healthcare providers, can help identify when the metrics are off-course.
Matias and Price identify five domains where public engagement can directly improve the science of AI evaluation:
Equipoise: When science is invoked by a group of people who disagree with each other or are uncertain over a course of action, it allows those people to commit to a shared process and accept the outcome–a condition called equipoise.
Measurement: Public participation can help ensure that AI systems measure the right things. For example, when members of the public were engaged in reviewing police records, they were able to identify instances of sexual assault that might otherwise be categorized as something less harmful, such as an improper search. (See Beneath the Surface, a project led by Invisible Institute and supported by HRDAG).
Explanation: Humans can help explain why and how AI systems arrive at the conclusions and outputs that they generate—which can otherwise seem obtuse.
Inference: One of the main utilities of AI is looking for patterns and then predicting what might happen next, but sometimes that means that AI can exacerbate bias in the initial data set. HRDAG and the ACLU learned the value of involving the public to correct bias in evaluating the Allegheny Family Screening Tool (AFST), in a project that found that certain families in Pennsylvania were being unfairly and inappropriately categorized as “high risk” in the welfare system by an AI-enabled tool.
Interpretation: Affected communities also contribute unique insights to the interpretation of scientific findings generated by AI systems.
This peer-reviewed paper can serve as a resource for researchers who are considering adding community feedback to an AI research project or who want to take it a step further and conduct participatory research. The authors share stories and summaries of the state of knowledge, both of which should help researchers, communities, funders, and regulators decide when and how to do participatory evaluation of AI.
In the age of machine learning, reliability doesn’t come from data alone—it comes from a deep collaboration between humans and machines. As Matias and Price conclude: “the public can serve as creative partners in the scientific and policy interpretation of evaluation findings. These evaluations may not resolve disputes entirely, but they can advance science and enable negotiations to be based on a more accurate, reliable understanding of AI systems in use.”
Image: Alexandre Dulaunoy “Kids, Adults and Computers at Hack4Kids” CC BY-SA 2.0 https://www.flickr.com/photos/adulau/