SemiAnalysis AgentX benchmark says CUDA moat holds in agents

Nvidia's software lead in serving frontier models still translates into a hardware-buying moat on long-context agentic workloads, according to a new open-source benchmark from SemiAnalysis.

AgentX 1.0, published August 24, tests multi-turn agentic coding inference at 1 million token context length across DeepSeek V4 Pro 0813, Kimi K3, MiniMax M3, Qwen3.5 and GLM 5.3, using 393 anonymized Claude Code traces from SemiAnalysis's own work. The authors — Cam Quilici, Bryan Shan and Alec Ibarra — say they burned more than $3M and roughly 2MW of continuously-run compute across 1000+ chips to build it. "We have spent more than $3M building this dataset. Today, we open source everything," they write.

The competitive read is uneven. On Qwen3.5 SGLang, the report puts Nvidia at "over 20x better performance" at 90 tok/s/user, and B300 in FP4 at "12x better performance per dollar" versus H100. On GLM 5.3 at 150 tok/s/user of interactivity, "Nvidia has up to 5x better cost efficiency," and the authors argue that with today's AMD software stack, "even if the competitor chip hardware was sold for free…cost per token would still be cheaper when using Nvidia."

AMD is not shut out everywhere. Before August 21, MI355X on SGLang matched B200 vLLM end-to-end on DeepSeek V4 Pro, and on Kimi K3 the authors report AMD's ATOM stack beating even GB300 NVL72 vLLM "on part of the curve between 40 to 60 second e2e latency." The catch is production adoption: SemiAnalysis says only "1 small advertising business unit at Alibaba Corp" runs ATOM live, and "The main Qwen LLM org at baba does not use ATOM in production."

The authors frame the durable advantage as distributed-systems software, not silicon. They single out context-parallel attention: "DCP/PCP forms part of CUDA moat as the AMD implementation of DCP/PCP isn't optimized yet. In the vLLM support matrix, every single AMD backend is unsupported." The benchmark shipped alongside 70+ upstream PRs into vLLM, SGLang, TensorRT-LLM, ATOM and AITER; the authors add, "This is a close race and we are excited to see the performance optimizations over the next couple weeks."