↻
Kanishka Misra reposted
Maria Antoniak
@mariaa.bsky.social
Lillian highly recommends this paper: aclanthology.org/2026.acl-lon... about errors made by coding agents
AI Weekly's analysis
→
- The best LLM coding agent on RExBench completed roughly 33% of 12 realistic research-paper extensions autonomously, with no hints.
- Adding human-written hints lifted the best success rate, but it still stayed below 44% across all agents tested.
- The benchmark evaluates 12 LLM agents across two frameworks, aider and OpenHands, on expert-authored extension tasks with automated scoring.
Read full analysis →
View on Bluesky →