Agentic coding is genuinely useful now, and there are some impressive reports of AI agents doing science. But how well and how reliably can they handle tasks scientists actually want to hand off, ones that bottleneck progress? How do we even measure that?? New paper🧵 arxiv.org…
AI Weekly's analysis
→
- Coding agents solved individual stages of a fly optogenetics pipeline but could not complete the full end-to-end discovery run.
- Agents struggle most when there is no predefined criterion to iterate on and must use their own scientific judgment, a key open challenge.
- The study flags challenges absent from standard benchmarks: computational resource management and generalization to large held-out datasets.
Read full analysis →