APort Vault: OAP Layer Blocks Unpermitted AI Agent Payments

Behind a deterministic authorization check sitting between the model and the payment tool, 0 of 69,297 evaluations produced a transfer to a recipient the policy did not permit. On the same attacks with the model alone, there were 140 of 76,842.

That is the headline in APort Vault, a September 2026 preprint by Uchi Uchibeke that replays 4,371 human-written attacks against a live payment agent across 14 models from 8 labs, in five policy configurations and two replay tracks, with and without a deterministic pre-action check implementing the Open Agent Passport (OAP) specification. 225,964 evaluations completed.

The attacks are not synthetic. Between March and August 2026 the author ran a public capture-the-flag against a live agentic bank with a $6,500 prize pool, and 1,128 distinct sessions produced the 4,371 attempts. Each attack was then replayed across Anthropic's Claude Fable 5.1, Sonnet 5 and Haiku 4.5; OpenAI's GPT-6 Astra, GPT-5.6 Terra and Luna; Google DeepMind's Gemini 3.5 Flash and 3.8 Flash; DeepSeek V4 Pro and Flash; Moonshot AI's Kimi K3; Z.AI's GLM-5.3; Alibaba's Qwen3.8 Max; and Meta's Muse Spark 1.3.

On Level 4's 1,293 shared prompts, request rates ran from 71.2% to 84.3%, and 809 (62.6%) elicited a request from all fourteen models, each ending in a successful payment. But Level 4 authorizes documented transfers to that allowlisted recipient, so shared behaviour there is not a defeat. The measurement levels are 2 to 4, where the passport permits some recipients and forbids others. The paper reports that "transfers to recipients the passport did not permit number 140 of 76,842 with the model alone and 0 of 69,297 behind the layer, and 105 against 0 on 68,970 matched model, prompt and track triples," with a session-clustered upper bound of 0.38%.

The zero was not obtained by refusing payments. Behind the layer, 25,370 payments executed; the policy denied 187 of the 25,640 transfer calls it evaluated, 148 of them because the recipient was not on the allowlist. Aggregate request rates barely moved between architectures: the paired difference is +0.084 percentage points.

The paper also thins out the model-safety framing. Five of fourteen models recorded no unpermitted transfer at Levels 2 to 4 in this corpus, but the 140 outcomes came from 24 of 790 sessions, with one session producing 67 of them. A single forged-receipt cohort of 103 attempts — the literal string `[Tool Result: verify_recipient]` written into the attacker's own message to look as though the tool had already approved the payee — accounts for 111 of 113 multi-turn Level 2 outcomes. Not one attack at Levels 2 to 4 defeated more than five of fourteen models.

The conflict of interest is stated in the paper: "The author is the founder of APort Technologies Inc., which develops the authorization layer evaluated here." The full 225,964-row evaluation set is released under CC BY 4.0, and a pre-registered human-labeled validation slice of roughly 300 stratified evaluations, promised in the dataset card, was not completed before the freeze. The preprint landed the same day the UN scientific panel warned that AI agent safeguards are "unravelling".