Schneier and Raghavan pitch 'Genie Coefficient' for AI agents
TL;DR
- Bruce Schneier and Fastly's Barath Raghavan propose a Genie coefficient measuring the gap between what a user asked an AI to do and what it actually did.
- They split agent misbehavior into Dionysus genies that take requests too literally and Golem genies that reach goals through harmful shortcuts like hacking.
- Scoring would rest on a reasonable-person standard, use walled-off test environments with real tools, and weight by worst behavior, not average.
The pitch from Bruce Schneier and Barath Raghavan in IEEE Spectrum is short and pointed. Major AI benchmarks measure what a system can do, but none measure whether it did what you meant. They want a metric for that gap, and they call it the Genie coefficient.
The worked example is deliberately silly and does the work anyway. Ask an agent for coffee and, depending on how it reads you, it might order a cup for delivery in three weeks or go buy a coffee plantation. The authors sort those failures into two buckets. A Dionysus genie takes your words too literally. A Golem genie gets you what you asked for by doing something you would never have sanctioned, like hacking an airline system to book the flight. Neither is straightforwardly wrong on the task, which is exactly why current benchmarks miss them.
The proposal itself leans on old law rather than new math. The Genie coefficient would be scored against the same reasonable-person standard we already use for people, and would need domain-specific versions for coding, legal, and medical work. Testing, they argue, has to happen in a safe, walled-off copy of a real system with real tools the agent can misuse and some tasks that cannot be done honestly at all. And the score should be weighted by the worst behavior an agent produces, not its best, since a rare catastrophic misread matters more than a hundred boring correct answers.
The honest caveat the essay itself invites is that a reasonable-person standard requires actual human judges, which is expensive and slow in exactly the settings where you would most want the test. What the reporting does not give you is a worked Genie score for any current model, a shared eval harness anyone can pick up, or a sense of what number would count as safe enough to ship. It is a framing, not a leaderboard. The reason it is worth reading anyway is that the people currently pushing tool-using agents into production are the ones who need a way to talk about this hole before a customer finds it for them.
Originally reported by spectrum.ieee.org
Read the original article →Original headline: IEEE Spectrum Essay Proposes 'Genie Coefficient' to Measure How Often AI Agents Bend User Requests