FrontierPhysics: top agent clears 27.4% of physics research tasks

The best AI agent passed 27.4% of 168 rollouts on an end-to-end physics research benchmark, and no other configuration did better. That score — 46 of 168 for Claude Opus 5.5 — comes from FrontierPhysics, 56 tasks built by 42 physicists from their own work across nine subfields including condensed matter, AMO, astrophysics, fluid dynamics, and high energy physics.

The tasks are not quizzes. Each agent has to review the literature, develop a research plan, conduct the experiment or simulation, and produce a structured output plus a scientific manuscript. A deterministic verifier checks the numbers; an expert-authored rubric grades the rest, with binary blocker gates that zero the whole score if scientific validity fails.

Across 22 frontier models in 26 configurations, Claude Opus 5.5 leads. The paper's own diagnosis of the pattern is flat: "frontier agents' main weaknesses are insufficiently rigorous reasoning and limited expert intuition, rather than gaps in physics knowledge or execution errors."

Manual review of 168 GPT-6 Astra rollouts carries the same picture in numbers. 48% of failures were tagged as non-rigorous thinking, with sub-categories including "Circular self-verification" and "Failure to consider edge cases." 34% were missing physics expert intuition, listed as "Inability to identify error sources" and "Poor debugging decisions." Physics knowledge gaps came in at 11%. Execution errors, 5%.

To check they had not just built an impossible rubric, the authors scored their own published papers against it. The result was 96.2% on average. 18 of the 56 tasks are drawn from published work, four of those from Nature.

The release lands into a crowded agent-evaluation beat — our tracker has logged 532 agent stories in the last 90 days, with Claude Opus 5.5 appearing across several of them this week.