AI Governance Conflates Bad Data With Flawed AI Systems

The Minab school strike is the case study that anchors this piece, and it is a heavy one. In February, a US Tomahawk missile struck the Shajareh Tayyebeh girls' school in Minab, Iran, killing 165 people, most of them girls between the ages of 7 and 12. A preliminary military investigation found that the strike resulted from outdated targeting data in a Defense Intelligence Agency database, with the school mislabeled as a military facility for at least a decade. Writing in TechPolicy.Press, Michael A. Santoro uses this tragedy to make a precise diagnostic point: this was not an AI system failure. The article notes that a human analyst consulting the same database would have reached the same catastrophic conclusion. The problem was the data.

That distinction turns out to matter enormously for governance, and Santoro argues the current policy conversation lacks the vocabulary to make it. He identifies two distinct failure categories. The first is as old as governance itself: "decisions are only as good as the information on which they are based." When data is faulty, outdated, or incomplete, any decision-maker (human or algorithmic) is likely to get it wrong. The second category is different in kind: AI systems themselves are not yet perfected. They are subject to brittleness when encountering situations outside their training distribution, opacity that makes it impossible to reconstruct a decision chain, and limitations inherent in the current state of the technology rather than in the quality of the data they receive.

Collapsing these two categories produces three governance errors. Accountability gets misdiagnosed, since different actors bear responsibility depending on which failure type occurred. Reform efforts get misdirected: organizational fixes are needed for data problems, technical research for system limitations, and confusing the two produces neither. The third error is a kind of intellectual dishonesty, making accountability demands that assume current AI systems are already reliable enough to bear the weight those frameworks place on them.

The amplification dynamic is what makes the distinction especially consequential. As Santoro writes, a flawed input into a manual decision may produce a single flawed outcome; the same flawed input into an algorithmic system can produce many flawed outcomes, faster and at scale. That scaling property is what turns a data-integrity gap into a mass casualty event.

What the article does not resolve is how to handle failures involving both bad data and system-level limitations simultaneously, or which body would enforce separate accountability tracks in practice. Santoro's narrower claim is more modest and harder to argue with: governance frameworks should require that both questions be asked separately before any investigation concludes, "was the data right?" and "is the system ready?"