Ragnedda and Ruiu urge AI audit 'power test,' not just fairness

Two media scholars writing in Tech Policy Press argue that the AI audit regimes taking shape in New York, Brussels and Washington risk certifying discriminatory systems with impressive technical documentation. Massimo Ragnedda of the University of Sharjah and Maria Laura Ruiu of the American University of Sharjah call this pattern "bias laundering" and propose a four-part power test to sit alongside the fairness metrics that dominate current practice.

Their anchor case is a 2019 Science study of a commercial US healthcare algorithm that used cost as a proxy for medical need. Because less money had historically been spent on Black patients, "Black patients assigned the same risk score were considerably sicker than white patients." Swapping in a closer measure of need would have lifted the share of Black patients selected for additional support from 17.7 percent to 46.5 percent. "The model was not simply inaccurate," the authors write. "It was competently predicting the wrong thing."

The four questions they want audits to answer are who defined the problem, who controls the system, who receives the benefits and bears the errors, and who can contest the decision. On the last: "Audits should verify that affected people can obtain relevant information, challenge data and classifications, reach a responsible human decision-maker." They note that New York City's bias audits for automated employment tools, the EU AI Act's fundamental-rights impact assessments, the NIST framework, UNESCO's Ethical Impact Assessment and the UK Algorithmic Transparency Recording Standard already touch parts of this territory, but stop short of interrogating who set the objective in the first place.

"An audit that begins after an institution has chosen the system's objective may certify an unequal policy with impressive technical documentation," Ragnedda and Ruiu warn. Two of the AI experts we track shared the piece within days of publication.