Found first: a primary source the press has not covered yet.
A new benchmark called SWE-bench Science tests coding agents on real bugs from scientific GitHub repositories, and the strongest system evaluated does not reach half. The paper, submitted August 20, 2026 by Zhipeng Xu, Jiahao Lu, Yining Zheng, Yuxin Wang, and Xipeng Qiu, reports that Claude Code with Opus-5 (max) achieves a pass@1 rate below 50% across 119 tasks spanning 20 scientific domains.
What the source says
The benchmark draws 119 tasks from 98 GitHub repositories across 20 scientific domains, organized into three paradigms: Issue-driven, Expert-exploratory, and Engineering-integration. The authors identify four recurring failure patterns: agents lack the scientific domain knowledge the task requires, exploration and repair tend to be surface-level or misdirected, fixes leave coverage gaps or fail to integrate with surrounding systems, and agents cannot generalize scientific reasoning from one context to another. On guidance, the paper finds a split result: well-grounded scientific context can improve average performance, but poorly aligned guidance induces anchoring without improving exact match success rates.
Why it matters
Scientific software is increasingly the instrument of record in research. A bug in a genomics pipeline or a numerical simulation library does not merely crash a program; it can corrupt the data underlying published conclusions without any visible error. The below-50% ceiling for the strongest available system gives a concrete measure of how far short current agents fall. The guidance finding adds a separate caution: providing an agent with more context is not a reliable path to better outcomes, and poorly chosen context can make performance worse.