Φ-Bench: Claude Opus 5 Tops 36.53% on LLM Infrastructure Tasks

The best frontier model on Φ-Bench, a new benchmark that asks whether large language models can build the infrastructure that runs them, scored 36.53%. That model, Claude Opus 5, led an eight-model field that included Kimi K3 at 28.12%, GPT 5.6 Sol at 24.51%, and DeepSeek V4Pro at 13.31%.

The paper on arXiv, from a team led by Leilei Ding, packages 85 tasks derived from 2,260 papers and 1,852 artifacts across nine top-level categories: training, inference and serving, compression, kernel, I/O, hardware and edge, data infrastructure, system optimization, and system assurance. Tasks come in three shapes: 55 kernel-function completions, 20 long-horizon implementations, and 10 end-to-end optimizations. The paper frames the workload as "open-ended, long-horizon LLM infrastructure engineering."

Category-level numbers are uneven. On end-to-end optimization, Claude Opus 5 reached 62.94%. On hardware and edge, the best model managed 5.4%. The authors read the top-line result as "substantial room for improvement."

The premise the benchmark tests comes from the paper's own opening line: LLMs "have demonstrated remarkable capabilities in reasoning and code generation, raising the prospect that they could assist in developing and optimizing the very infrastructure that powers them."